When building Sitecore implementations one thing you must always think of is: What is the cache strategy and how it will be implemented? For example will html cache be set in standard values of templates? What is the size of database cache? What about prefetch cache?
Here at Perficient we always take these questions into consideration when dealing with Sitecore implementations.
We like to benefit as much as possible from atomic design principles and our SCORE accelerator does help a lot in that sense. You can assemble your pages, however it makes more sense to achieve the expected design. You can have presets of templates with SCORE components and setup cache using standard values to make sure those are applied for all items.
Now one thing we implemented recently posed an interesting challenge for our team. We could not rely solely on standard values cache because we had templates which were generic content pages and our content authors needed all flexibility to setup the page however they wanted. In this scenario we could not setup all templates with all variations of those pages. The goal was to have any type of page with any organization of components and that organization would vary a lot between them so again we couldn’t setup in standard values.
Another important thing is that html cache is something that impacts Sitecore in ways that normally a more technical person(developer) needs to be involved when making those decisions. Normally content authors are specialists is content, in assembling it and matching designs, but deciding what type of cache to use and understand the implications in any of those choices can be tough for them due to the implications those decisions may have in the system.
In a scenario like this how can one balance having Sitecore cache to improve performance but at the same time keep atomic design flexibility?
With that in mind we figured that we needed to automate applying the cache in such way that:
- A) Html cache is applied automatically for all pages
- B) Each tenant site could have different cache settings and variations
- C) We needed a way to opt out of cache when conditions applied
- D) Not all renderings should have cache so we needed a way to set which ones should or should not have it setup
PS: When we had this situation we were baking internally an auto cache implementation in SCORE which has already been released, and if you are using any version after 3.0.4 these are built in for you. It’s a very interesting implementation which supports Donut caching with Sitecore. If you are not using SCORE I would suggest you to give it a try. But if you are interested in what we built keep reading…
Before we get to the auto cache implementation it’s important to understand a bit more on one key Sitecore pipeline: mvc.renderRendering. This pipeline comes out of the box with Sitecore and it is patched through the Sitecore.Mvc.config file. This pipeline runs every time a rendering is rendered. So if you have 10 renderings in a page this pipeline gets executed 10 times. That’s important because anything added to it cannot be time consuming or heavy processing because it will slowdown tremendously the website. There are 3 processors which are important to understand what they do if you want to apply auto cache:
Before we get to the auto cache implementation it’s important to understand a bit more on one key Sitecore pipeline: mvc.renderRendering. This pipeline comes out of the box with Sitecore and it is patched through the Sitecore.Mvc.config file. This pipeline runs every time a rendering is rendered. So if you have 10 renderings in a page this pipeline gets executed 10 times. That’s important because anything added to it cannot be time consuming or heavy processing because it will slowdown tremendously the website. There are 3 processors which are important to understand what they do if you want to apply auto cache:
Sitecore.Mvc.Pipelines.Response.RenderRendering.SetCacheability
This processor is what sets in the RenderRenderingArgs whether the rendering is cacheable or not. For example when a content authors goes into the rendering parameters popup and selects it’s cacheable with Vary By Data for example this will set the RenderRenderingArgs in the property Caching.Cacheable to be true. This property is used in the RenderFromCache processor to either render from cache or not. So again if you are thinking of hooking an auto cache you can add a custom processor prior to this to essentially set that property as per your conditions
Sitecore.Mvc.Pipelines.Response.RenderRendering.GenerateCacheKey
This processor is used to determine the cache keys that are generated based on which cache variations are selected for that particular rendering. For example if Vary by Data is set it adds a cache key for the component datasource so if there are two instances of the same component on a page and they have different datasources, the processor RenderFromCache will determine that it needs to render both instances since they have different datasources.
Sitecore.Mvc.Pipelines.Response.RenderRendering.RenderFromCache
This is the processor which validates whether there is an html cached for the cachekey. If there is then it renders from cache otherwise it will execute the render itself for that particular component.
Now that we have reviewed these processors let’s start breaking down the implementation.
Configure opt-in and opt-out
The first thing to implement the auto cache is setup a way determine which renderings will opt-in and out of auto cache. We will use a configuration file for that but it could also be done through Sitecore items with the only caveat that if the calls become expensive to the database it will create a performance issue.
The file is a typical patch config file which can be read using Sitecore’s API. Here is how it looks like:
<cacheSettings>
<site name="Tenant1">
<!--List of all renderins that will be auto cached-->
<IncludeCacheRenderings>
<add ID="{D4ED0D11-17DF-49EA-9A86-D6B50FE658CC}" toolTip="Reference Component" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
<add ID="{DE0D280E-2138-4DA6-A9D5-5D67ABB591D8}" toolTip="Reference a Snippet" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
<add ID="{45EE100C-7D3C-4E57-8600-78BA96B19365}" toolTip="One Column Fixed Layout" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
<add ID="{452AD13D-29A5-406E-879E-FB2A28489D24}" toolTip="Page layout - One Column wide screen" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
<add ID="{21D93CC0-304F-47A3-A412-89410FD714B1}" toolTip="Page layout - 2 Column Equal" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
</IncludeCacheRenderings>
<ExcludeCacheRenderings>
<add ID="{462C3495-FE96-4E19-93BE-FDA208548A0A}" toolTip="MyCustomRenderingExclude 1" ></add>
<add ID="{0C298C22-790B-4B08-B6BC-415D6A7F2BDD}" toolTip="MyCustomRenderingExclude 1"></add>
</ExcludeCacheRenderings>
</site>
<site name="Tenant2">
<!--List of all renderins that will be auto cached-->
<IncludeCacheRenderings>
<add ID="{21D93CC0-304F-47A3-A412-89410FD714B1}" toolTip="Page layout - 2 Column Equal" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
<add ID="{487AEE7D-7A9F-47FE-A9DF-D5953A1A14FB}" toolTip="Page layout - 3 Column Equal" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
<add ID="{1F776F0D-E392-4B3D-A6D4-5FCBFECE56D2}" toolTip="Stripe" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
<add ID="{2EB2D682-0098-48C4-B314-F4F37B1DB932}" toolTip="Picture Stripe" VaryByPage="true" VaryByData="true" VaryByDevice="false" VaryByLogin="false" VaryByParam="false" VaryByQueryString="false" VaryByUser="false" ></add>
</IncludeCacheRenderings>
<ExcludeCacheRenderings>
<add ID="{462C3495-FE96-4E19-93BE-FDA208548A0A}" toolTip="MyCustomRenderingExclude 1" ></add>
<add ID="{0C298C22-790B-4B08-B6BC-415D6A7F2BDD}" toolTip="MyCustomRenderingExclude 1"></add>
</ExcludeCacheRenderings>
</site>
</cacheSettings>
Note that in the configuration above the setup is done per tenant. Also on each tenant we can setup what are the renderings and it’s variations for auto cache that need to be auto cached(section IncludeCacheRendering). The ID property for each tenant actually reflects the ID of the rendering definition item in Sitecore itself.
Also we have set for each rendering what are the possible cache variations(Vary by Data, Vary by Device, etc).
Also notice we have created a section “ExcludeCacheRenderings” which flags that the rendering needs to be removed from cache, in other words always render instead of reading from the cache. We will see bellow how the opt-out works.
Custom processor to apply auto cache
Now it’s time to apply to our renderings all the configs we setup previously. To do so we create a custom processor which inherits from RenderRenderingProcessor and we patch it before SetCacheability:
public class SetCacheableRenderings : RenderRenderingProcessor
{
public override void Process(RenderRenderingArgs args)
{
Assert.ArgumentNotNull(args, "args");
if (!Sitecore.Context.PageMode.IsNormal || args.Rendered || args.Rendering?.Caching == null || args.Rendering?.RenderingItem==null)
{
return;
}
ApplyCache(args);
}
protected virtual void ApplyCache(RenderRenderingArgs args)
{
var renderings = CacheSettings.Instance.GetIncludedCacheRenderingsPerSite(Sitecore.Context.Site.Name);
//Ids of renderings that need to be cached
var Ids = CacheSettings.Instance.GetCachedRenderingsList(renderings).ToLower();
//if the rendering is part of the list of renderings that are cachable it sets it
if (!string.IsNullOrWhiteSpace(Ids) && Ids.Contains(args.Rendering.RenderingItem.ID.ToString().ToLower()))
{
var rendering = renderings.FirstOrDefault(c => c.Id == args.Rendering.RenderingItem.ID.ToString());
if (rendering != null)
{
args.Rendering.Caching.Cacheable = true;
args.Rendering.Caching.VaryByUser = rendering.VaryByUser;
args.Rendering.Caching.VaryByData = rendering.VaryByData;
args.Rendering.Caching.VaryByDevice= rendering.VaryByDevice;
args.Rendering.Caching.VaryByLogin= rendering.VaryByLogin;
args.Rendering.Caching.VaryByParameters = rendering.VaryByParam;
args.Rendering.Caching.VaryByQueryString = rendering.VaryByQueryString;
//vary by page is Score custom so needs to be set in the parameters
args.Rendering.Parameters[Score.Custom.Presentation.RenderingParameters.VaryByPageAttrName] = rendering.VaryByPage ? "1" : "0";
}
}
}
}
This processor will read the configuration file and apply the auto cache and it’s variations to each rendering as per defined in config.
Create custom processor to extent cache key
Now we need to make sure that the cache key is properly updated for all renderings with no datasource. The reason why we update we set only the ones with no datasource is because Score already handles scenarios for renderings with datasource so we don’t need to worry about that. On the processor bellow note that we are only applying the cache key if it is cacheable and it is not yet rendered.
public class ExtendCacheKeyRenderingUniqueId : RenderRenderingProcessor
{
/// <summary>
/// Processes the specified arguments.
/// </summary>
/// <param name="args">The arguments.</param>
public override void Process(RenderRenderingArgs args)
{
if (!CacheSettings.Instance.IsCacheEnabled || !Context.PageMode.IsNormal || args.Rendered
|| args.Rendering == null
|| !args.Cacheable
|| args.Rendering.RenderingItem == null)
{
return;
}
//applies only for processors with no datasource
if (String.IsNullOrWhiteSpace(args.Rendering.DataSource))
{
args.CacheKey = string.Concat(args.CacheKey ?? string.Empty, "_#uniqueRenderingId:", args.Rendering.UniqueId);
}
else
{
//Exception to Score's ExtendCacheKeyWithRenderingUniqueId processor. This is needed for a scenario where two components share the same datasource so a new variation in cache key is required
if (!(args.CacheKey.IndexOf("_#uniqueRenderingId:") >= 0))
{
args.CacheKey = string.Concat(args.CacheKey ?? string.Empty, "_#uniqueRenderingId:", args.Rendering.UniqueId);
}
}
}
}
This processor can be patched after Sitecore’s GenerateCacheKey one.
Patch instead a new RenderFromCache processor
Now we will copy Sitecore’s RenderFromCache processor code(just decompile it) and add a few things to it and in this case we will patch this new processor instead of Sitecore’s one.
The first thing to note below is we have a check for whether the rendering html contains a token. This token we will apply on the custom processor ApplyCacheToken (below). If that token exists it will force Sitecore to render the rendering(by returning false). This is how we opt-out. All those renderings we configured to be excluded will have this token within their html. Note that we used an html comment for that because that will be present in the render presentation(markup). For this post we won’t bother removing it but I’d say it’s something to consider if you don’t want to have these tokens added to the markup on your site.
public class RenderFromCache: Sitecore.Mvc.Pipelines.Response.RenderRendering.RenderFromCache
{
public override void Process(RenderRenderingArgs args)
{
Assert.ArgumentNotNull((object)args, nameof(args));
if (!Context.PageMode.IsNormal)
{
base.Process(args);
return;
}
if (args.Rendered)
return;
string cacheKey = args.CacheKey;
if (!args.Cacheable || string.IsNullOrEmpty(cacheKey) || !this.Render(cacheKey, args.Writer, args))
return;
args.Rendered = true;
args.Cacheable = false;
args.UsedCache = true;
}
protected virtual bool Render(string cacheKey, TextWriter writer, RenderRenderingArgs args)
{
HtmlCache htmlCache = Context.Site.ValueOrDefault<SiteContext, HtmlCache>((Func<SiteContext, HtmlCache>)(site => CacheManager.GetHtmlCache(site)));
if (htmlCache == null)
return false;
string html = htmlCache.GetHtml(cacheKey);
if (String.IsNullOrWhiteSpace(html))
return false;
if (html.Contains($"<!-- myToken:"))
{
//The container has a component that should be cleared from the cache so returns false so the whole section is rendered
return false;
}
Log.Debug("Cache Debug info: Rendering from html cache. "+args.Rendering.RenderingItem.ID.ToString());
//writes from cache
writer.Write(html);
return true;
}
}
Create a processor to apply cache token
As mentioned before we need a processor to apply the cache token. This processor can be patched after RenderFromCache. In this case the first time it gets executed it will render the rendering and add to the cache so the second time the page is loaded is where we want to either render from cache itself or to render again.
This processor is used not only to exclude from cache when the rendering html contains the token, but also to exclude when the rendering has personalization setup or multi-variant tests. Unless one wants to cache the personalization variations(which we are not covering in this post but can very well be done). For the sake of this implementation let’s assume that we don’t render from cache personalizations nor multi-variant tests so we always render it from the rendering engine instead of from cache.
public class ApplyCacheToken : RenderRenderingProcessor
{
public override void Process(RenderRenderingArgs args)
{
Assert.ArgumentNotNull(args, "args");
if (!Sitecore.Context.PageMode.IsNormal || args.Rendered || args.Rendering?.Caching == null || args.Rendering?.RenderingItem==null)
{
return;
}
RenderingContext renderingContext = RenderingContext.CurrentOrNull;
Sitecore.Mvc.Presentation.Rendering rendering = renderingContext.Rendering;
// rendering.RenderingItem is null when presentation details points to a rendering that is no longer in Sitecore
// RenderingXml property is only set on renderings that were bound to a placeholder via presentation details
if (rendering == null || rendering.RenderingItem == null ||
rendering.Properties["RenderingXml"].IsWhiteSpaceOrNull())
{
Log.Debug("Cache Debug info: RenderingXml is null or rendering.RenderingItem is null when presentation details points to a rendering that is no longer in Sitecore ", args);
return;
}
var renderings = CacheSettings.Instance.GetExcludeCacheRenderingsPerSite(Sitecore.Context.Site.Name);
//Ids of renderings that need to be cached
var Ids = CacheSettings.Instance.GetExcludedCachedRenderingsList(renderings).ToLower();
Log.Debug($"Cache Debug info: cached renderings {Ids}", args);
//if the rendering is part of the list of renderings that are cache excluded or have personalization or has mvt set
if (!String.IsNullOrWhiteSpace(Ids) && (Ids.Contains(args.Rendering.RenderingItem.ID.ToString().ToLower()) || args.Rendering.Properties["RenderingXml"].Contains("<rule") || args.Rendering.Properties["RenderingXml"].Contains("mvt=")))
{
Log.Debug($"Cache Debug info: Rendering cacheable {args.Rendering.RenderingItem.ID.ToString()}", args);
ApplyToken(args);
}
}
public static void ApplyToken(RenderRenderingArgs args)
{
if (RenderingContext.CurrentOrNull == null)
{
Log.Debug($"Cache Debug info: Could not retrieve rendering context so cannot add tokens to rendering", args);
return;
}
//creates the token and adds as a wrapper so it's ready to be read by the RenderFromCache processor
var token = new CacheToken(RenderingContext.CurrentOrNull);
args.Disposables.Add(new Wrapper(args.Writer, token));
}
}
public class CacheToken : IMarker
{
private RenderingContext _renderingContext { get; set; }
public CacheToken(RenderingContext context)
{
_renderingContext = context;
}
public virtual string GetStart()
{
var token = "<!-- myToken:";
Log.Debug($"Cache Debug info: Token cache being set {token}", this);
return token;
}
public virtual string GetEnd()
{
return String.Empty;
}
}
One important thing to understand here is let’s assume you have a 2 column container component which within it has 2 other components like a button and an image component. In this implementation when you add the token to the two column what will happen with the button and image? Will they be rendered from cache or not?
In this implementation it will clear the cache for all 3 because the html of the 2 column will already have the html of the button and the image so it opts-out everything within it.
That’s it. Now once the pages are rendered auto cache will start taking effect. There are a few tools that used together help a lot checking the cache being built:
- A) Sitecore cache page(/sitecore/admin/cache.aspx): here you can detect for each tenant the cache size increasing as pages are consumed
- B) Sitecore stats page(/sitecore/admin/stats.aspx): here you can also see the cache size increasing but broken down per rendering
- C) Sitecore debug: It will show you whether a rendering is rendered from cache or not
d) Pipeline.debug marketplace module: This module helps a lot as you can debug the pipeline execution and figure out whether the values you are expecting to see in the args are properly set on each processor so definitely something useful.
Other things to consider
There are a few more things to be considered when building an auto cache. For example if there are too many pages with too many components within them with auto cache setup, what will happen to memory usage? It will go up really fast and of course we don’t want to have memory spiking at 100% because of it so one thing to keep in mind is this approach needs to be tested in a lower environment which needs to mirror production as much as possible and it needs to have a proper load test execution to validate if real production traffic will not cause these spikes.
Another important thing to keep in mind is the site definition htmlCacheSize property which needs to be updated to make sure that the tenant has enough memory allocated for html cache. If this is not configured what will happen when it reaches the threshold it will clear what has been cached to add new entries to it. To identify what’s the cache size one can also get that through the load test I mentioned above and monitor cache usage to have a good sense on what the limit is.
