SearchStax

The SearchStax® Frequently Asked Questions page includes the following approved question and answer about our Apache Solr Cloud services.


How Much Free Disk Space Do I Have?

The Managed Solr backup feature requires abundant free disk space in order to succeed. Backups may fail intermittently when the size of the index(es) is more than a third of the disk space allocated to each node of the cluster. As the index size approaches half of the disk space, backups will fail consistently. See Why do Backups Fail?

But how can you tell how much disk space is available? In early versions of Solr, this was pretty mysterious. From Solr 6.0 onward, however, you can pull this information out of the Solr Dashboard, the Managed Solr Dashboard, and the Pulse graphs. You just have to know where to look for it.

Allocated Disk Space

To determine the amount of disk space allocated to each node of your deployment, look on the Deployment Details page of the Managed Solr Dashboard. The HW Specs column shows the original amount of disk space on the node. In this case it is 8.0 GB.

SearchStax backup disk space

Note, however, that it is possible to add disk capacity to a deployment (see the section near the end of this page), but that the added space is not reflected in this display.

Disk Usage

To determine the amount of disk space in use on a node, open up the Solr Dashboard by clicking on the HTTP Solr Endpoint of the deployment. (You might need to make adjustments to your IP Filter and Solr Basic Auth security settings.)

Navigate to the Cloud > Nodes display. The data table reveals the Disk Usage of each node in the cluster. In this case, 3.5 GB of the disk is in use.

SearchStax backup disk space

In most deployments, this number is a good proxy for the size of the index(es), but it also includes some disk space devoted to configurations, logs, and JAR files.

Disk Space from Pulse

The Pulse graph of Disk Usage reveals some more detail. This is a graph from a newly-created NDN1 node. Initially, the disk usage was 0.43 GB.

SearchStax backup disk space

We then restored a 13-million-record collection from a backup. (This was the same index as shown in previous examples.) Total disk usage grew to 3.95 GB. (Pulse shows the upper limit of available disk space as a dashed line at 7.75 GB.)

SearchStax backup disk space

Note that 3.95 GB – 0.43 GB = 3.52 GB, which is in good agreement with the estimate we obtained from the Solr Dashboard.

Will the Backup Succeed/Fail?

Here is a heuristic for predicting backup success/failure. If 1.5 times the occupied disk space is greater than the remaining disk space (allocated – occupied), then the backup attempt will fail. For example,

1.5 x 3.5 GB > (8.0 GB - 3.5 GB)

so the backup attempt failed.

SearchStax backup disk space

This failure resulted in automatic email to the users of the account, notifying them that the system will need more disk space.

The failure threshold varies from one deployment to another, and even from one node to another. This is a rule of thumb only.

Adding Disk Space

It is possible to add disk space to the nodes of a deployment by making a request of SearchStax Support. We can add disk space in blocks of 50GB for $25/node/month.

For instance, we added 50 GB to the example deployment shown above. We could see the added memory in the Pulse graph:

SearchStax Managed Solr Backups

The bump in the graph is the temporary disk usage increase due to the next backup attempt. The system needed almost 10 GB for this event. This one succeeded. The final backup file was just a little larger than the occupied disk space.


We love to answer questions!

Please contact the SearchStax Support Desk immediately if you have any question about Solr Cloud deployments.

Return to Frequently Asked Questions.