SearchStax

The SearchStax® Frequently Asked Questions page includes the following approved question and answer about our Apache Solr Cloud services.


Document contains immense term

While indexing, several clients have encountered a verbose error like this one:

solr.log:153:2021-07-01 20:20:34.784 ERROR (qtp496729294-18) [c:a-sitecore-master-index s:shard1 r:core_node2 x:a-sitecore-master-index_shard1_replica_n1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Exception writing document id sitecore://master/{9bda6ef2-56f2-43d5-a804-94f741184cd5}?lang=en&ver=1&ndx=sitecore_master_index to the index; possible analysis error: Document contains at least one immense term in field="additionalfields_sm" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[78, 87, 86, 79, 101, 86, 73, 120, 85, 108, 81, 52, 100, 72, 86, 90, 90, 106, 77, 50, 97, 71, 78, 67, 99, 86, 77, 118, 82, 68]...', original message: bytes can be at most 32766 in length; got 45116. Perhaps the document has an indexed string field (solr.StrField) which is too large

The key is this phrase:

Document contains at least one immense term in field=<field_name> (whose UTF8 encoding is longer than the max length 32766)

This is a Lucene index error message (Solr uses Lucene indexes) indicating that an indexed string field value contained more than 32766 characters. A string field is indexed as a single monolithic value. Since it makes no sense to attempt a perfect character-to-character match against a 32K string, this almost always indicates that the field type is set incorrectly in the Solr schema.

To remedy this situation, consider these strategies:

  • Change the field definition in the schema so the field is not indexed.
  • Change the field type to “text” or some other tokenized field.
  • Create a customized tokenizer to handle the field as you see fit.

The Internet contains many blog discussions of this error. The bloggers suggest multiple possible modifications to the field definition.


We love to answer questions!

Please contact the SearchStax Support Desk immediately if you have any question about Solr Cloud deployments.

Return to Frequently Asked Questions.