Solr is increasingly becoming a defacto search engine for Sitecore Implementations because of it’s performance, features, customizability and scalability. However, if Solr is not configured or used appropriately, applications sometimes notice performance problems.
What are signs of Solr search performance issues on Sitecore?
Is the performance of your Solr search implementation sluggish and crawling along at a snail’s pace? Are you getting unexpected out-of-memory errors, suffering from spontaneous recovery mode issues or experiencing slow indexing performance?
Solr search configuration issues are complex and the prescriptions for fixing them are sometimes subtle. Through our deep experience with Solr configurations at SearchStax, we have learned that the solrconfig.xml settings are the root cause of many performance issues. The default values suggested by the system are sometimes poor choices and can lead to sustained high CPU usage or high JVM memory loading.
Too-frequent hard commits, with their massive load on CPU and disk I/O, are the worst offender. Fortunately, that problem and many others are easy to fix by following best practices to optimize or tune Solr search performance on Sitecore.
Follow these five SearchStax recommendations to optimize your solrconfig.xml settings and avoid the most-common Solr search performance issues with Sitecore.
1. Set the autoSoftCommit feature to 2 minutes
A “soft commit” achieves near-realtime (NRT) indexing by opening a new searcher after a specific time interval. It is faster and less expensive than a hard commit as it does not write the index to disk, or start a new transaction log. However, even though it is less expensive than a hard commit, it still utilizes resources and should be set for as long as it is reasonable for the application. We recommend setting this feature to two minutes or 120,000 milliseconds.
<autoSoftCommit> <maxTime>120000</maxTime> </autoSoftCommit>
2. Set the autoCommit feature to 5 minutes
A “hard commit” writes the in-memory index to the disk. It closes the current transaction log and opens a new one. It is best not to do this too frequently since it requires significant time. We recommend committing after at least five minutes.
If autoSoftCommit is enabled, which it is by default, that opens a new searcher on every soft commit. Since soft commits happen more frequently than hard ones, there is no need to have hard commits open searchers, too, and so we recommend setting the openSearcher flag as false.
<autoCommit> <maxTime>300000</maxTime> <openSearcher>false</openSearcher> </autoCommit>
As your collection sizes become large, Solr might not be able to create a snapshot of the index within 5 minutes. In those cases, for regular backups, you might have to consider increasing the commit settings higher – at 5 minutes for soft commits and 10 minutes for hard commits.
3. Use autowarmCount = 0 for All Cache Settings
The query caches–filterCache, queryResultCache, and documentCache–store query results for faster retrieval on subsequent searches. We recommend setting the autowarmCount to zero for all of the cache settings.
<filterCache class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0"/> <queryResultCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/> <documentCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/>
4. Set maxRamMB to 200
The queryResultCache has an additional optional setting, maxRamMB. This feature can be set to make sure that the cache does not consume too much memory, which can happen in cases where there are large stored fields returned as part of the query result.
We also recommend using the default value of 200 for queryResultMaxDocsCached.
5. Use the Default Values of True for Lazy Fields and Sorted Query
The enableLazyFieldLoading setting allows Solr to load fields that are not directly requested later as needed. This can boost performance if most queries request only a small subset of fields.
We recommend leaving the enableLazyFieldLoading with the default value of true.
There can be cases where the same query is requested many times, but with different sort field / ordering. Setting useFilterForSortedQuery in these scenarios can give performance improvement as Solr would use a filter to satisfy the search and cache the results.
We recommend that useFilterForSortedQuery should be left as true.
Related Best Practices for Tuning Solr Performance
If you want to learn more about best practices for tuning Solr performance, check out these additional resources:
- Performance optimization for the xConnect Search Service when using Solr: A Sitecore knowledge-base article on how to configure xConnect when indexing the Sitecore xDB collection in Solr.
- Solr Out of Memory (OOM): Causes and Solutions: A SearchStax blog post on how users accidentally over-allocate memory, which can slow down performance and potentially crash Solr.
- Solr-as-a-Service for Sitecore: Learn more about the benefits of using Solr-as-a-Service on Sitecore.