Solr search queries of your index are blazing fast. At least they should be, right? Have you ever run into a Solr search query that takes an abnormal amount of time? If so, then this blog post is for you. There are a few tricks you can employ to speed the query up. Even if your query was already quick, these tips can still bring you some performance enhancement. Without further ado, let’s dive into these tips by improving an example query.
Let’s provide some context. This is the Solr search query we will be improving:
var personnel = context.GetQueryable<PersonnelSearchResultModel>().FirstOrDefault(x => x.Login == user.login);
This line of code is querying an index containing personnel information such as name, email, login, phone number, etc. Here is the definition of the object we are querying for:
public class PersonnelSearchResultModel
{
[IndexField("email")]
public string Email { get; set; }
[IndexField("login")]
public virtual string Login { get; set; }
[IndexField("active")]
public virtual bool Active { get; set; }
[IndexField("hiredate")]
public virtual DateTime HireDate { get; set; }
[IndexField("firstname")]
public virtual string FirstName { get; set; }
[IndexField("lastname")]
public virtual string LastName { get; set; }
[IndexField("middlename")]
public virtual string MiddleName { get; set; }
[IndexField("mobilephone")]
public virtual string MobilePhone { get; set; }
// CUSTOM PROPERTIES THAT ARE NOT STORED IN SOLR
public bool IsFavorite { get; set; }
public string SpecialDisplayName { get; set; }
public string SpecialTitle { get; set; }
}
This is the template we are working with from Sitecore:
And finally, here is the index config file we are working with:
<personnelSolrIndexConfiguration ref="defaultSolrIndexConfiguration">
<fieldMap ref="defaultSolrIndexConfiguration/fieldMap">
<fieldNames hint="raw:AddFieldByFieldName">
<field fieldName="firstname" returnType="string" />
</fieldNames>
</fieldMap>
</personnelSolrIndexConfiguration>
Performing the search (which contains 2900+ Personnel items) takes this long according to the logs:
11:03:28 INFO Starting Personnel Query 11:03:30 INFO Personnel Query completed in 2512.6272ms
This is painfully slow. This is over 2 seconds! This seriously impacts the performance of the page load. Let’s fix this problem.
1. Change .FirstOrDefault() to .Where().ToList().FirstOrDefault()
This may seem counterintuitive at first, but changing the LINQ query to this translates to a faster query due to Sitecore having to deserialize the Solr content into objects. After switching out this code, the logs showed a faster query.
11:15:21 INFO Starting Personnel Query 11:15:22 INFO Personnel Query completed in 1278.3236ms
Now our C# code looks like this:
var personnel = context.GetQueryable<PersonnelSearchResultModel>().Where(x => x.Login == user.login).ToList().FirstOrDefault();
Wow! This nearly doubled the speed of the query. However, this still isn’t good enough. This query still needs some improvement.
2. Explicitly list out the return types of the indexed fields in the index config file
Even though Solr will index everything for you automatically, there is a performance cost if Sitecore doesn’t explicitly know the return type of any given attribute. Solr will have to use reflection to determine the return type. Using reflection for say 30 attributes can lead to a lot of time being eaten up. You can tell if Solr is having trouble determining the return type by taking a look at the search logs. This is a small snippet of what the search logs look like for the query above:
11:20:29 WARN Processing Field Name : email. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'email', return type 'String' and field type '' 11:20:29 WARN Processing Field Name : login. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'login', return type 'String' and field type '' 11:20:29 WARN Processing Field Name : lastname. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'lastname', return type 'String' and field type '' 11:20:29 WARN Processing Field Name : lname. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'lname', return type 'String' and field type '' 11:20:29 WARN Processing Field Name : login. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'login', return type 'String' and field type '' 11:20:29 WARN Processing Field Name : middlename. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'middlename', return type 'String' and field type '' 11:20:29 WARN Processing Field Name : mobilephone. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'mobilephone', return type 'String' and field type '' 11:20:29 WARN Processing Field Name : workphone. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name workphone, return type 'String' and field type '' 11:20:29 WARN Processing Field Name : workphone2. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name workphone2, return type 'String' and field type '' 11:20:29 WARN Processing Field Name : prefix. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'prefix', return type 'String' and field type '' 11:20:29 WARN Processing Field Name : previousemployer. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'previousemployer', return type 'String' and field type ''
As you can see, even simple return types such as a string give Sitecore difficulty. Sitecore is trying to map all of the properties found in Solr to my PersonnelSearchResultModel even though we are not requiring to have those attributes in the model. For example, Sitecore is trying to map workphone2 to its proper return type even though we are not asking for it in the model. Note that we do not see problems regarding the firstname field because we listed its return type in the index config already.
What’s the fix? We need to explicitly tell Sitecore the expected return types for these attributes found in the Personnel item in the index config like so:
<personnelSolrIndexConfiguration ref="defaultSolrIndexConfiguration">
<fieldMap ref="defaultSolrIndexConfiguration/fieldMap">
<fieldNames hint="raw:AddFieldByFieldName">
<field fieldName="firstName" returnType="string" />
<field fieldName="active" returnType="bool" />
<field fieldName="email" returnType="string" />
<field fieldName="login" returnType="string" />
<field fieldName="hiredate" returnType="datetime" />
<field fieldName="lname" returnType="string" />
<field fieldName="middlename" returnType="string" />
<field fieldName="mobilephone" returnType="string" />
<field fieldName="prefix" returnType="string" />
<field fieldName="previousemployer" returnType="string" />
<field fieldName="workphone" returnType="string" />
<field fieldName="workphone2" returnType="string" />
</fieldNames>
</fieldMap>
</personnelSolrIndexConfiguration>
This cleans up the search logs of these warnings of the attributes not listed out in the index config. Let’s take a look at our speed in the logs now:
11:42:21 INFO Starting Personnel Query 11:42:21 INFO Personnel Query completed in 450.1276ms
We are getting some serious performance enhancements here! Taking a second look back at the search logs, however, we are still seeing some reflection problems from Solr as seen below:
11:45:17 WARN Processing Field Name : IsFavorite. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'isfavorite', return type 'bool' and field type '' 11:45:17 WARN Processing Field Name : SpecialDisplayName. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'specialdisplayname', return type 'String' and field type '' 11:45:17 WARN Processing Field Name : SpecialTitle. Resolving Multiple Field found on Solr Field Map. No matching template field on index field name 'specialtitle', return type 'String' and field type ''
We are still seeing warnings for the custom attributes we have made on our model even though those custom attributes are not found on the Personnel item in Sitecore. This leads us to my next point.
3. Remove all attributes in your search result that are not found in the Sitecore item that you are querying
You need to remove custom attributes that are not found in the index. Else Solr will use reflection to determine its return type as seen in the logs above. If you need both the search result and the custom attributes combined together to pass into your view, I recommend creating a container model that houses both the search result and your custom attributes. Your LINQ query will still only query for the search result, but you will then assign its return object to the search result attribute on your container model. Let’s create that container model for our code above. This is what our final code looks like after all these performance enhancements from this post.
Our new view model we are passing to the view:
public class PersonnelViewModel
{
public PersonnelSearchResultModel Personnel { get; set; }
// THESE ATTRIBUTES ARE REMOVED FROM THE PERSONNELSEARCHRESULTMODEL
public bool IsFavorite { get; set; }
public string SpecialDisplayName { get; set; }
public string SpecialTitle { get; set; }
}
Our LINQ query found in the controller action:
model.Personnel = context.GetQueryable<PersonnelSearchResultModel>().Where(x => x.Login == user.login).ToList().FirstOrDefault();
Our index config now looks like this:
<personnelSolrIndexConfiguration ref="defaultSolrIndexConfiguration">
<fieldMap ref="defaultSolrIndexConfiguration/fieldMap">
<fieldNames hint="raw:AddFieldByFieldName">
<field fieldName="firstName" returnType="string" />
<field fieldName="active" returnType="bool" />
<field fieldName="email" returnType="string" />
<field fieldName="login" returnType="string" />
<field fieldName="hiredate" returnType="datetime" />
<field fieldName="lname" returnType="string" />
<field fieldName="middlename" returnType="string" />
<field fieldName="mobilephone" returnType="string" />
<field fieldName="prefix" returnType="string" />
<field fieldName="previousemployer" returnType="string" />
<field fieldName="workphone" returnType="string" />
<field fieldName="workphone2" returnType="string" />
</fieldNames>
</fieldMap>
</personnelSolrIndexConfiguration>
Let’s take a look into the logs to see how fast this query is now.
11:55:43 INFO Starting Personnel Query 11:55:43 INFO Personnel Query completed in 278.5632ms
Nice! We have greatly improved the speed from 2512.6272ms to 278.5632ms. Improving this query will produce very noticeable performance enhancement to the page load of your site. I hope this blog post helps save you from a slow site!
