SearchStax NextGen Solr Clusters: When Less is More

In the next few days, SearchStax is releasing a new product line, the “NextGen” Dedicated  Clusters (NDC). This was announced a couple months ago in this blog. Some of our clients found our three-node production systems too pricey, so we responded by bringing out more reliable five-node systems that actually cost less per month — and deliver additional benefits.
No, it’s not economy of scale or sleight-of-hand. NextGen clusters really cost less than our Standard Dedicated Clusters. Less is more.

History: SearchStax has always offered two types of Solr Cloud deployments. The Dedicated Node (DN) series offers single-node Solr deployments for experimentation, testing, and proof-of-concept projects. Single-node systems are quite economical. They start around $20/month and go up from there.

SearchStax Solr Dedicated Node

Our Standard Dedicated Cluster (DC) series supports robust three-node production systems. Pricing on the DC systems cannot be called “economical.” They are big servers designed for heavy duty, and there are at least three of them in each deployment.

SearchStax Solr Dedicated Cluster

SearchStax uses Apache ZooKeeper to manage configuration files across a cluster. In both DN and DC deployments, each Solr instance shares space with a ZooKeeper instance on the same server. That’s a single ZooKeeper for a DN system, and a three-instance “ensemble” for a production system. By design, ZooKeeper must have three (or more) instances to deliver fault-tolerant management of a production system. Since a production system requires three servers for ZooKeeper, we co-located three Solr instances on the same servers. Those were our Standard Dedicated Clusters.

Problem: As our company gained experience in the Solr Cloud marketplace, we realized that some clients don’t need three Solr servers to achieve a fault-tolerant production environment. For many clients, even some large organizations using enterprise-grade CMS like Sitecore, two Solr servers will do the job. Unfortunately, a two-server system cuts ZooKeeper back to two instances, which simply doesn’t work. ZooKeeper must have three servers.

SearchStax Solr NextGen Cluster

Solution: The NextGen Dedicated Clusters use twin Solr servers matched with three independent ZooKeeper VMs. A typical NextGen cluster, therefore, has five servers.

Fortunately, ZooKeeper servers can be very small, sometimes called micro-machines. That’s why a five-server NextGen cluster is more economical than our three-node Standard clusters.

Benefits: The NextGen clusters are both more economical and more reliable!

  • Customers with lower query loads were over-provisioned with three-node clusters. NextGen clusters are a better fit for these clients. (You can easily add Solr nodes if you need them.)
  • In all NextGen plans, ZooKeeper instances run on their own dedicated nodes. That provides better performance and reliability because Solr and ZooKeeper aren’t competing for CPU time.
  • In a Standard Dedicated Cluster, when a node went down it stopped a Solr instance and a ZooKeeper instance at the same time. In the new architecture, a server failure stops only one of them. This greatly simplifies automatic recovery.

Take-away: Providing Solr infrastructure and management is not an easy feat. It requires constant attention, effort, love and care. This is why using an experienced service like SearchStax is usually more efficient than trying to do it yourself.