Excluding and Including Sitecore Assemblies in Helix Publishing Pipeline Solution

If you have a Sitecore Visual Studio solution inspired by Helix Example and tried source control the Sitecore assemblies or Hotfix assemblies and are unable to copy it to the deploy folder then this post explains how we can publish the same to the deploy folder and we will try to understand how it works.

Sitecore Helix is a set of official guidelines and recommended practices for Sitecore Development. I believe we often take reference and reuse code from Helix Example to develop the new Sitecore VS solution for a new project. Good to read the intended usage. and recent Update on Development Practices for Headless Projects.

Challenge:

Since the Webroot bin has the Sitecore assemblies already, we usually don’t need to send Sitecore assemblies or the assemblies that are already there in the bin to avoid unnecessary version conflicts in deployment and to keep build artifacts lighter. When it comes to the Sitecore Hotfix patch case, we plan to source control it if we don’t want to deploy it to each of the servers manually. Here, we need to allow Sitecore assemblies being part of the Hotfix, to deploy to the publish folder. Consider Sitecore solution based on Sitecore Unicorn-based Helix Examples Repository. With its current setup, Sitecore assemblies do not get deployed. Hence, even if Sitecore Hotfix dlls are referenced, it does not get deployed i.e., not being copied to the publish output folder.

This post does not target any specific Sitecore version, but we will understand this behavior and its solution in the Helix publishing pipeline solution.

Solution:

We will understand how the Sitecore assemblies are not being deployed and, we will explore how we can allow the selected Sitecore assemblies to be deployed to the output folder.

Taking reference to https://github.com/Sitecore/Helix.Examples, let’s clone this repository.

git clone https://github.com/Sitecore/Helix.Examples.git

How are Sitecore Assemblies not Published to the Deploy Folder?

Let’s consider a unicorn-based solution. Open Helix.Examples\examples\helix-basic-unicorn\BasicCompany.sln.

Note the solution supports Sitecore version 10.1 now when I write this post. To play with a higher version, consider upgrading the solution and also upgrading Sitecore.Assemblies.Platform NuGet package to the same desired Sitecore version.

Build the solution.

Note: If you get an error –

Copying file Helix.Examples\examples\helix-basic-unicorn\src\Feature\BasicContent\website\..\serialization\templates\BasicContent\Section Header Folder\__Standard Values.yml to obj\Debug\Package\PackageTmp\App_Data\serialization\Feature\BasicContent\templates\BasicContent\Section Header Folder\__Standard Values.yml failed. Could not find a part of the path 'obj\Debug\Package\PackageTmp\App_Data\serialization\Feature\BasicContent\templates\BasicContent\Section Header Folder\__Standard Values.yml'

Or a similar error then it could be because the file path exceeds more than 255 characters. Hence, try to move the repository root folder Helix.Examples of a shorter parent folder path e.g., C:\Helix.ExamplesPractice.

And rebuild again. Should succeed now.

Browse to the folder Helix.Examples\examples\helix-basic-unicorn\docker\deploy\website\bin. Even though most of the projects in this solution have referenced the Sitecore.Kernel.dll and some other dlls, we do not find any Sitecore Dll deployed to Helix.Examples\examples\helix-basic-unicorn\docker\deploy\website\bin.

This is because this VS solution has a reference to Nuget packages – Sitecore.Assemblies.Platform and RichardSzalay.Helix.Publishing.WebRoot.

Sitecore.Assemblies.Platform Nuget package provides the SitecoreAssemblies item group for assemblies that ship with the main Sitecore platform roles (CM/CD/etc..), which should be excluded from deployment.

RichardSzalay.Helix.Publishing.WebRoot package includes dependent web projects in the ASP.NET Publishing Pipeline.

These two packages together help to exclude the Sitecore assemblies that ship with Sitecore.

Open the following file to know the list of Sitecore assemblies.

%USERPROFILE%\.nuget\packages\sitecore.assemblies.platform\10.1.0\build\ Sitecore.Assemblies.Platform.targets

This file defines the SitecoreAssemblies ItemGroup which holds the list of Sitecore Assemblies.

Consider SitecoreAssemblies as a collection variable for simplicity purposes.

Open file – Helix.Examples\examples\helix-basic-unicorn\src\Environment\Website\Website.wpp.targets

WPP stands for Web Publishing Pipeline, and hence this is the Web Publishing Pipeline target file.

This one is website project-specific and applies to all the publish profiles. We can also have profile-specific wpp.targets file.

Note the file name convention, it is [project name].wpp.targets. We can have .wpp.targets files project-specific also. These files have instructions to exclude or include files for deployment.

Website.wpp.targets

File name given with Include attribute means it adds file name to the SitecoreAssembliesToExclude ItemGroup to exclude from the deployment. This SitecoreAssembliesToExclude ItemGroup is being used by instruction in the below file.

%USERPROFILE%\.nuget\packages\richardszalay.helix.publishing.webroot\1.5.6\build\Helix.Publishing.Plugins\AssemblyLists.targets

<Project>
  <PropertyGroup>
    <ExcludeSitecoreAssemblyLists Condition="'$(ExcludeSitecoreAssemblyLists)'==''">true</ExcludeSitecoreAssemblyLists>
  </PropertyGroup>

  <UsingTask TaskName="RichardSzalay.Helix.Publishing.Tasks.ParseAssemblyLists" 
              AssemblyFile="$(MSBuildThisFileDirectory)..\RichardSzalay.Helix.Publishing.Tasks.dll"
              Condition="'$(ExcludeSitecoreAssemblyLists)'=='true'"
              />

  <PropertyGroup Condition="'$(ExcludeSitecoreAssemblyLists)' == 'true'">
    <ExcludeFilesFromPackageDependsOn>
      $(ExcludeFilesFromPackageDependsOn);
      ExcludeSitecoreAssembliesFromPublish
    </ExcludeFilesFromPackageDependsOn>
  </PropertyGroup>
    
  <Target Name="ExcludeSitecoreAssembliesFromPublish" DependsOnTargets="ParseSitecoreAssemblyLists">

    <ItemGroup>
      <_SitecoreAssembliesToExclude Remove="@(_SitecoreAssembliesToExclude)" />
      <_SitecoreAssembliesToExclude Include="@(SitecoreAssembliesToExclude)" />
      <_SitecoreAssembliesToExclude Remove="@(SitecoreAssembliesToInclude)" />
    </ItemGroup>

    <ItemGroup>
      <ExcludeFromPackageFiles Include="@(_SitecoreAssembliesToExclude -> 'bin\%(Filename)%(Extension)')" />
      <ExcludeFromPackageFiles Include="@(_SitecoreAssembliesToExclude -> 'bin\%(Filename).pdb')" />
      <ExcludeFromPackageFiles Include="@(_SitecoreAssembliesToExclude -> 'bin\%(Filename).xml')" />
    </ItemGroup>
  </Target>
  
  <!-- This is a pretty naive implementation but it's fine for the lists we have -->
  <Target Name="ParseSitecoreAssemblyLists">

    <ParseAssemblyLists Files="@(SitecoreAssemblyListsToExclude)"
                        Condition="'@(SitecoreAssemblyListsToExclude)'!=''">
      <Output TaskParameter="Output" ItemName="SitecoreAssembliesToExclude"/>
    </ParseAssemblyLists>
    
  </Target>


</Project>

The above target file contributes to building the solution.

ExcludeSitecoreAssemblyLists

It has a property ExcludeSitecoreAssemblyLists which is set to true if nothing is set. Since it is true, 2 targets ExcludeSitecoreAssembliesFromPublish and ParseSitecoreAssemblyLists in this file, are executed.

Targets group tasks together in a particular order and allow the build process to be factored into smaller units. For example, one target may delete all files in the output directory to prepare for the build, while another compiles the inputs for the project and places them in the empty directory. (Reference)

ExcludeSitecoreAssembliesFromPublish depended on the ParseSitecoreAssemblyLists target. ParseSitecoreAssemblyLists is executed first and then ExcludeSitecoreAssembliesFromPublish. SitecoreAssemblyListsToExclude can be used to supply the list of Sitecore Assemblies from the CSV file. At this moment, SitecoreAssemblyListsToExclude is empty, and hence nothing is set to SitecoreAssembliesToExclude from SitecoreAssemblyListsToExclude. Moving to the ExcludeSitecoreAssembliesFromPublish.

SitecoreAssembliesToExclude items are included in _SitecoreAssembliesToExclude ItemGroup which later is included in ExcludeFromPackageFiles. Note the third entry – SitecoreAssembliesToInclude. This can hold the list of assemblies we want to copy to the deploy folder. Here, this list is removed from _ SitecoreAssembliesToExclude. Hence the SitecoreAssembliesToExclude is updated and devoid of the list of assemblies mentioned by SitecoreAssembliesToInclude. As a result, the removed list of assemblies is not filtered from getting copied to the deploy folder.

ExcludeFromPackageFiles is a WPP mechanism to exclude the files from the deployment. To exclude the folder(s) itself ExcludeFromPackageFolders can be used.

To Summarize,

_SitecoreAssembliesToExclude = _SitecoreAssembliesToExclude - _SitecoreAssembliesToExclude

_SitecoreAssembliesToExclude = _SitecoreAssembliesToExclude + SitecoreAssembliesToExclude

_SitecoreAssembliesToExclude = _SitecoreAssembliesToExclude – SitecoreAssembliesToInclude

ExcludeFromPackageFiles = _SitecoreAssembliesToExclude

All the types of files .dll, .xml, and .pdb after the assembly file name are excluded from the deploy folder.

We can set up SitecoreAssemblyListsToExclude as another layer to exclude the list of assemblies if really needed.

Following is the syntax we can add in the Website.wpp.targets or any .targets file.

<!-- In ProjectName.wpp.targets or PublishProfile.wpp.targets -->
<ItemGroup>
  <!-- Assembly lists -->
  <SitecoreAssemblyListsToExclude Include="Assembly Lists\Sitecore.Platform.Assemblies.xxversionxx.csv " />
  <SitecoreAssemblyListsToExclude Include="Assembly Lists\Sitecore.XConnect.Platform.Assemblies.xxversionxx.csv" />
</ItemGroup>

Following is the syntax if any specific assembly needs to be excluded.

<ItemGroup>
  <!-- Or individual assemblies -->
  <SitecoreAssembliesToExclude Include="Sitecore.Kernel.dll" />
</ItemGroup>

Reference: https://github.com/richardszalay/helix-publishing-pipeline#excluding-sitecore-assemblies

We can download the assembly CSV file list from the download page of a particular Sitecore version. Check the Assembly list link under Release information.

e.g. https://dev.sitecore.net/Downloads/Sitecore_Experience_Platform/101/Sitecore_Experience_Platform_101.aspx

Reference: Generating Assembly Lists for SXA (or anything) to exclude during deployment

Note: Earlier the solution was using the CSV but then later updated to use the Sitecore.Assemblies.platform package (reference commit).

Within the Visual Studio solution, one can always set the copy local to false in properties of the concerned local dll reference. But then we need to ensure we do it for all the occurrences.

Include Sitecore Assemblies or Sitecore Hotfix assemblies in to deploy folder.

Let’s go back to the file.

%USERPROFILE%\.nuget\packages\richardszalay.helix.publishing.webroot\1.5.6\build\Helix.Publishing.Plugins\AssemblyLists.targets

At the following line, the list of assemblies set in SitecoreAssembliesToInclude is removed from the list of assemblies set to _SitecoreAssembliesToExclude.

<_SitecoreAssembliesToExclude Remove="@(SitecoreAssembliesToInclude)" />

As a result, upon build, only those assemblies are excluded which are mentioned by _SitecoreAssembliesToExclude.

<ItemGroup>
    <ExcludeFromPackageFiles Include="@(_SitecoreAssembliesToExclude -> 'bin\%(Filename)%(Extension)')" />
    <ExcludeFromPackageFiles Include="@(_SitecoreAssembliesToExclude -> 'bin\%(Filename).pdb')" />
    <ExcludeFromPackageFiles Include="@(_SitecoreAssembliesToExclude -> 'bin\%(Filename).xml')" />
</ItemGroup>

Moreover, to quickly validate if the above file is contributing to the build, comment the following line and build the solution.

<_SitecoreAssembliesToExclude Include="@(SitecoreAssembliesToExclude)" />

We can see all the Sitecore assemblies deployed to – Helix.Examples\examples\helix-basic-unicorn\docker\deploy\website\bin. Hence we need to modify the SitecoreAssembliesToExclude list i.e. remove the desired assemblies that we need to publish to the deploy folder. Check approach 3 and 4 from the next section.

Once you validated, clean up the bin folder and uncomment the above line.

Allow Sitecore Assemblies or Hotfix assemblies to deploy folder.

To source control the dll files from the Hotfix package, create a folder in the Website project for instance Hotfixlib, copy all the Hotfix DLLs, and locally reference them in the Website project. This will help deploy these dlls to the output folder. We considered the website project to reference the hotfix dlls because this picks in the last i.e., it overrides the dlls from other projects to the deployment folder once allowed. Also, we have a website project last in the build order. Now to allow the dlls to be copied to the deploy folder we have multiple places or ways we can supply the same solution. We can do any of the following approaches.

• 1. To allow a copy of specific Sitecore Assembly like Sitecore.Kernel.dll to deploy folder we can add the following line in the file – Helix.Examples\examples\helix-basic-unicorn\src\Environment\Website\Website.wpp.targets

This can be added in the first ItemGroup element or in a separate one, both works. Ensure we restart the Visual Studio after the Website.wpp.targets file is updated else it will not work.

• 2. We can update the same in the Helix.Examples\examples\helix-basic-unicorn\src\Environment\Website\Website.csproj file.

<ItemGroup>
  <SitecoreAssembliesToInclude Include="Sitecore.Kernel.dll" />
</ItemGroup>

In case of multiple dlls, entry for each dll can be made as below.

<ItemGroup>
  <SitecoreAssembliesToInclude Include="Sitecore.Kernel.dll" />
  <SitecoreAssembliesToInclude Include="Sitecore.Mvc.dll" />
</ItemGroup>

• 3. We can create a separate ItemGroup collection called SitecoreHotfixAssemblies in Helix.Examples\examples\helix-basic-unicorn\src\Environment\Website\Website.wpp.targets.

<ItemGroup>
  <SitecoreHotfixAssemblies Include="Sitecore.Kernel.dll" />
  <SitecoreHotfixAssemblies Include="Sitecore.Mvc.dll" />
</ItemGroup>

Duplicate and comment on the following line.

<SitecoreAssembliesToExclude Include="@(SitecoreAssemblies)" />

Update the duplicated line with Exclude=”@(SitecoreHotfixAssemblies)” so it becomes as below.

<SitecoreAssembliesToExclude Include="@(SitecoreAssemblies)" Exclude="@(SitecoreHotfixAssemblies)"/>

• 4. If the list is too long, then we can maintain it in a separate target file. For instance, create a file named Hotfix.targets and move the ItemGroup element with SitecoreHotfixAssemblies to this file as shown below.

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <ItemGroup>
        <SitecoreHotfixAssemblies Include="Sitecore.Buckets.Client.dll" />
        <SitecoreHotfixAssemblies Include="Sitecore.Buckets.dll" />
        <SitecoreHotfixAssemblies Include="Sitecore.Client.dll" />
        <SitecoreHotfixAssemblies Include="Sitecore.Client.xml" />
    </ItemGroup>
</Project>

Finally, in place of the moved ItemGroup element from the Website.wpp.targets file, import the Hotfix.targets file as below.

<Import Project="Hotfix.targets" />

And keep following as it is on the Website.wpp.targets file.

<SitecoreAssembliesToExclude Include="@(SitecoreAssemblies)" Exclude="@(SitecoreHotfixAssemblies)"/>

Run the following command in the bin directory from the Hotfix package to obtain the entries with the dlls. Copy the output to the .targets file under the ItemGroup element.

foreach ($dllname in (ls).Name){"<SitecoreHotfixAssemblies Include=""$dllname"" />"}

Demo

By default, Sitecore Assemblies are excluded from the publish to deploy folder.

Based on a separate .targets file including the list of hotfix assemblies approach.

Note: If you come across an already setup project or a legacy one and if the above changes are made, still the configured assemblies are not getting deployed then do check if SitecoreAssemblyListsToExclude is set in any of the files like .targets, .pubxml, or .csproj. Because this might still be getting assemblies from some CSV files, which is considered by the SitecoreAssembliesToExclude in %USERPROFILE%\.nuget\packages\richardszalay.helix.publishing.webroot\1.5.6\build\Helix.Publishing.Plugins\AssemblyLists.targets. Good to get rid of it or remove the desired assembly’s entry from the referenced CSV file.

Hope this helps to understand the different files involved in excluding and including the Sitecore assemblies in the Helix publishing pipeline solution.

Happy Sitecore Learning 😊