Managed Search Help Center

The SearchStax Managed Search service backup feature requires abundant free disk space in order to succeed. Backups may fail intermittently when the size of the index(es) is more than a third of the disk space allocated to each node of the cluster. As the index size approaches half of the disk space, backups will fail consistently. See Why do Backups Fail?

But how can you tell how much disk space is available? In early versions of Solr, this was pretty mysterious. From Solr 6.0 onward, however, you can pull this information out of the Solr Dashboard, the Managed Search Dashboard, and the Monitoring graphs. You just have to know where to look for it.

Allocated Disk Space

To determine the amount of disk space allocated to each node of your deployment, look on the Deployment Details page of the Managed Search Dashboard. Note the Plan of the deployment, and then open the New Deployment form. Find the Disk Space figure for this plan. In this case it is 16 GB.

Note, however, that it is possible to add disk capacity to a deployment (see the section near the end of this page), but that the added space is not reflected in this display.

Disk Usage

To determine the amount of disk space in use on a node, open up the Solr Dashboard by clicking on the HTTP Solr Endpoint of the deployment. (You might need to make adjustments to your IP Filter and Solr Basic Auth security settings.)

Navigate to the Cloud > Nodes display. The data table reveals the Disk Usage of each node in the cluster. In this case, 8.7 GB of the disk is in use.

In most deployments, this number is a good proxy for the size of the index(es), but it also includes some disk space devoted to configurations, logs, and JAR files.

Disk Space from Monitoring Graphs

The graph of Disk Usage reveals some more detail. This is a graph from an NDN2 node containing 195 million records of test data. It shows disk usage as 9.57 GB, which is in rough agreement with the previous figure.

The graph shows the upper limit of available disk space as a dashed line at 15.58 GB.

Will the Backup Succeed/Fail?

Here is a heuristic for predicting backup success/failure. If 1.5 times the occupied disk space is greater than the remaining disk space (allocated – occupied), then the backup attempt will fail. For example,

1.5 x 8.7 GB > (15.58 GB - 8.7 GB)

so the backup attempt failed.

This failure resulted in automatic email to the users of the account, notifying them that the system will need more disk space.

The failure threshold varies from one deployment to another, and even from one node to another. This is a rule of thumb only.

Adding Disk Space

It is possible to add disk space to the nodes of a deployment by making a request of SearchStax Support. We can add disk space in blocks of 50GB for $25/node/month.