Mar. 01, 2023
Dipsy Kapoor
|
Is the performance of your Solr search implementation sluggish and crawling along at a snail’s pace? Are you getting unexpected out-of-memory errors, suffering from spontaneous recovery mode issues or experiencing slow indexing performance?
Solr search configuration issues are complex and the prescriptions for fixing them are sometimes subtle. The solrconfig.xml settings can be the root cause of many performance issues. The default values suggested by the system are sometimes poor choices and can lead to sustained high CPU usage or high JVM memory loading.
Too-frequent hard commits, with their massive load on CPU and disk I/O, are the worst offender. Fortunately, that problem and many others are easy to fix by following best practices to optimize or tune Solr search performance.
Follow these five recommendations to optimize your solrconfig.xml settings and avoid the most-common Solr search performance issues:
A “soft commit” achieves near-realtime (NRT) indexing by opening a new searcher after a specific time interval. It is faster and less expensive than a hard commit as it does not write the index to disk, or start a new transaction log. However, even though it is less expensive than a hard commit, it still utilizes resources and should be set for as long as it is reasonable for the application.
We recommend setting this feature to two minutes or 120,000 milliseconds.
<autoSoftCommit>
<maxTime>120000</maxTime>
</autoSoftCommit>
A “hard commit” writes the in-memory index to the disk. It closes the current transaction log and opens a new one. It is best not to do this too frequently since it requires significant time.
We recommend committing after at least five minutes.
If autoSoftCommit is enabled, which it is by default, that opens a new searcher on every soft commit. Since soft commits happen more frequently than hard ones, there is no need to have hard commits open searchers, too, and so we recommend setting the openSearcher flag as false.
<autoCommit>
<maxTime>300000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
The query caches–filterCache, queryResultCache, and documentCache–store query results for faster retrieval on subsequent searches.
We recommend setting the autowarmCount to zero for all of the cache settings.
<filterCache class="solr.FastLRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
<queryResultCache class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
<documentCache class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
The queryResultCache has an additional optional setting, maxRamMB. This feature can be set to make sure that the cache does not consume too much memory, which can happen in cases where there are large stored fields returned as part of the query result.
We also recommend using the default value of 200 for queryResultMaxDocsCached.
The enableLazyFieldLoading setting allows Solr to load fields that are not directly requested later as needed. This can boost performance if most queries request only a small subset of fields.
We recommend leaving the enableLazyFieldLoading with the default value of true.
<enableLazyFieldLoading>true</enableLazyFieldLoading>
There can be cases where the same query is requested many times, but with different sort field / ordering. Setting useFilterForSortedQuery in these scenarios can give performance improvement as Solr would use a filter to satisfy the search and cache the results.
We recommend that useFilterForSortedQuery should be left as true.
<useFilterForSortedQuery>true</useFilterForSortedQuery>
SearchStax Cloud is a fully-managed SaaS solution that automates, manages, maintains and scales Solr search infrastructure in public or private clouds. Schedule a Demo to see the power of hosted Solr for yourself or Contact Us to speak with one of our search experts.
The Stack is delivered bi-monthly with industry trends, insights, products and more
Copyrights © SearchStax Inc.2014-2024. All Rights Reserved.