Solr Log Files – SearchStax
SearchStax® Managed Solr log files help you tune Solr performance and isolate potential bugs in your application. The SearchStax Help Center references many typical errors and other log entries very specifically.
The Solr log files may be viewed in a limited way from the Solr dashboard. Recent solr.log files may be downloaded from Managed Solr. There is a special log in Managed Solr that records your team’s actions in the Managed Solr Dashboard. Some other logs are available through the agency of SearchStax support engineers.
Time Stamps in Solr.log Files
The date/time stamps in solr.log files are in UTC (Coordinated Universal Time).
Solr Server Logs
Each Solr instance maintains a solr.log file, managed by Apache Log4j.
This file accumulates log messages at the INFO, WARN, and ERROR levels until the size of the file reaches 33.55 MB or until Solr is restarted. At that point, Solr closes the file, renames it solr.log.N, and opens a new solr.log file. You can see the files in the Managed Solr dashboard, Dedicated Infrastructure > Deployment > Deployment Details > Logs screen.
Solr.log is the live file, containing the most-recent log entries. The rollover files are numbered sequentially, so the one with the highest number is the most-recent one.
The files in this list are a snapshot of the Solr log files on the server. This snapshot is updated at the top of every hour. This means that when you experiment with your system, you must wait until the top of the hour to see changes in the logs. (If you are in India, the logs become available at the bottom of every hour.)
The topic of “log retention” using Log4j can be a bit complicated. Please see Configuring Logging for all the details.
As expressed in Managed Solr, a new solr.log file is initiated each time a Solr instance is started. That file continues to record log messages until either Solr stops, or until the size of solr.log exceeds 33.55 MB. The solr.log file of a small, stable system can accumulate log messages for weeks or months without rolling over.
When solr.log reaches 33.55 MB, it rolls over.
- The original solr.log file is automatically renamed solr.log.1.
- A new solr.log file is created and begins to record messages.
The next time the log rolls over:
- The current solr.log file is renamed solr.log.2.
- A new solr.log file is initiated.
Note that the solr.log contains the current log messages. The most recent archived messages are in the file with the largest extension number.
This process continues until there is one solr.log file and ten numbered rollover files:
|solr.log||Current entries. “Current” might mean hours to months, depending on log message volume.|
|solr.log.1||Oldest entries still retained by the system.|
|solr.log.2||A little newer than solr.log.1.|
|solr.log.3||Newer than solr.log.2.|
|solr.log.4||Newer than solr.log.3.|
|solr.log.5||Newer than solr.log.4.|
|solr.log.6||Newer than solr.log.5.|
|solr.log.7||Newer than solr.log.6.|
|solr.log.8||Newer than solr.log.7.|
|solr.log.9||Newer than solr.log.8.|
|solr.log.10||Newest rollover entries from solr.log.|
Managed Solr retains a maximum of ten rollover files. Therefore, when solr.log rolls over for the eleventh time, something has to give.
- Solr.log.1, the oldest rollover file, is deleted. This is automatic and is not recoverable.
- All of the remaining rollover files are decremented, so solr.log.2 becomes the new solr.log.1 and so forth. At the bottom of the list, solr.log.10 is renamed to be solr.log.9.
- The current solr.log file is saved as the new solr.log.10.
This process is dynamic, so note that the log messages you saw in solr.log.6 last week might have migrated to solr.log.5 by now. If you need to reference messages in a log file, you should save a physical copy of the file.
When debugging a troubled Solr deployment, there isn’t much point in looking back more than a few days. Therefore, one downloads the solr.log file (current) and sometimes also the rollover file with the largest number (which is often solr.log.10). These two files contain a hefty sample of recent log messages.
Note that you must examine the logs from each of the Solr nodes in your cluster. Solr problems sometimes occur on only one node at a time.
The remarks in this section assume that you want to modify the default logging behavior of Log4j in Managed Solr.
You can adjust Solr logging in the Solr Dashboard:
Solr Dashboard log changes affect only one server
Managed Solr connects to the Solr Dashboard of one Solr node chosen randomly by a load balancer. Changes to logging levels affect only one server of the cluster. When you return to the Solr Dashboard to undo your log settings later, you might not be connected to the same node. This can make it difficult to back out your changes.
To cleanly reset all servers to default logging, perform a rolling restart of the cluster. Each server returns to default logging as it restarts.
You can monitor significant Solr events on an adjacent screen in the Solr dashboard:
See Configuring Logging in the Apache Solr documentation.
Managed Solr Activity Log
Managed Solr automatically records all significant user actions that occur in the Managed Solr dashboard. To view them, click Activity Log in the left-side navigation panel.
Activity Log entries are retained indefinitely.
Help with Log Files
Premium customers may ask SearchStax support for help tracking down memory and performance issues. This often involves enabling logging on servers that the SearchStax end-user cannot reach.
For this type of help please reach out to the SearchStax Support Desk.