Sitecore Unicorn Serialization Paths Too Long in Azure

We recently ran into a mysterious issue when running a Unicorn sync in an Azure environment that turned out to be a classic “path is too long” issue in disguise. Here is what we saw in our logs:

ERROR ERROR: Some directories could not be loaded. (9 inner failures)
Tree failed to load: master:/sitecore/content/blah/blah (guidguid-guid-guid-guid-guidguidguid) (the error may be on a child of this item, see the details below)
... (more of these)
Tree failed to load: master:/sitecore/content/blah/blah (guidguid-guid-guid-guid-guidguidguid) (the error may be on a child of this item, see the details below) (Unicorn.DeserializationAggregateException)
at Unicorn.Loader.DeserializeFailureRetryer.RetryAll(ISourceDataStore sourceDataStore, Action`1 retrySingleItemAction, Action`1 retryTreeAction)
   at Unicorn.Loader.SerializationLoader.LoadAll(IItemData[] rootItemsData, IDeserializeFailureRetryer retryer, IConsistencyChecker consistencyChecker, Action`1 rootLoadedCallback)
   at Unicorn.SerializationHelper.SyncTree(IConfiguration configuration, Action`1 rootLoadedCallback, Boolean runSyncStartPipeline, IItemData partialSyncRoot)
   at Unicorn.SerializationHelper.SyncConfigurations(IConfiguration[] configurations, IProgressStatus progress, ILogger additionalLogger
... (more of these)
INNER EXCEPTION
Tree failed to load: master:/sitecore/content/blah/blah (guidguid-guid-guid-guid-guidguidguid) (the error may be on a child of this item, see the details below) (Rainbow.Storage.Sc.Deserialization.DeserializationException)
No stack trace available.
INNER EXCEPTION
Value cannot be null.
Parameter name: reference (System.ArgumentNullException)
at Sitecore.Diagnostics.Assert.ArgumentNotNull(Object argument, String argumentName)
   at Unicorn.Loader.DeserializeFailureRetryer.AddItemRetry(IItemData item, Exception exception)
   at Unicorn.Loader.SerializationLoader.LoadOneLevel(IItemData rootSerializedItemData, IDeserializeFailureRetryer retryer, IConsistencyChecker consistencyChecker)
   at Unicorn.Loader.SerializationLoader.LoadTreeInternal(IItemData root, IDeserializeFailureRetryer retryer, IConsistencyChecker consistencyChecker)

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

Through trial and error, we ended up isolating a few YAML files that had several interesting qualities:

• They had a deep path
• The filename had been appended with a Guid

Rainbow (not Unicorn) has a few things that it does to ensure that you don’t end up with crazy long paths, as the content tree can easily end up with very deep paths or lengthy item names that would be too long for a file system to mirror. When browsing through code, it is quite helpful to have the serialized content files match up with the content tree as much as possible, but because Rainbow is helpful it has two strategies to prevent you breaking your own stuff.

• If the file path to the serialized item would be longer than the value in the config setting Rainbow.SFS.SerializationFolderPathMaxLength, Rainbow will replace the relative path of the item with the Guid of the parent item. So if you have/a/really/long/folder/structure/then/it/may/be/replaced/with/just guidguid-guid-guid-guid-guidguidguid. By default this value is 110 in Rainbow, but Habitat sets it to 150 which seems to work pretty well for a Helix based site.
• If the item name is longer than the value in the config setting Rainbow.SFS.MaxItemNameLengthBeforeTruncation, Rainbow will truncate the item name to just those characters. IMPORTANT: If after truncating the file is not uniquely named, Rainbow will append the item id. By default this value is 30 in Rainbow, but Habitat bumps it to 50.

Rainbow will not let you have values for the folder length and truncation length that are too long, but unfortunately it also reserves the 36 characters for the Guid that it might need to append. This forces you to have shorter lengths for your max path and truncation, unfortunately. I have half a mind to submit a PR to allow you to have the option to replace the file name with the item id instead of appending the id, as that would give you an extra 36 characters to play with in your path and file name. 50 characters is pretty good for ensuring unique file names, so I would likely give most of the extra characters to the path. In our particular project, we had some very lengthy paths to being with due to the multi-tenant nature of our Helix solution, so we were only able to spare 20 characters for the item name so that our serialization path could be long enough to not have every single item be in a guid parent folder.
The problem we ran into was that while our local file paths were under the Windows limits, after a TFS build and a deployment to Azure (all of which succeeded) the resulting file path exceeded 260 characters. Somehow every step along the way succeeded and the actual files were deployed, but Rainbow threw an error when trying access the files. We tried manually deleting the files with Kudu and were finally exposed to the underlying error. Our solution was to refactor our build scripts for establishing local symlinks with the tenant serialization folders so that they would have much shorter paths, along with shortening the deployment folder paths to ensure that all paths were kept well under the limits. Hopefully my colleague Andy Cohen will detail that process in a future blog post!