SearchStax NextGen Solr Clusters: When Less is More

SearchStax NextGen Solr Clusters: When Less is More

In the next few days, SearchStax is releasing a new product line, the “NextGen” Dedicated  Clusters (NDC) for our Solr-as-a-Service offering. This was announced a couple months ago in this blog. Our current three-node high availability production systems are reliable, but pricey as they are configured with three full-sized Apache Solr nodes — so our response was to engineer an even more reliable five-node system that actually cost less per month — and deliver additional benefits.

No, it’s not economy of scale or sleight-of-hand. NextGen clusters really cost less than our Standard Dedicated Clusters. Less is more.

History Behind NextGen Solr Clusters

SearchStax has always offered two types of Solr Cloud deployments. The Dedicated Node (DN) series offers single-node Solr deployments for experimentation, testing, and proof-of-concept projects. Single-node systems are quite economical. They start around $20/month and go up from there.

SearchStax Solr Dedicated Node

Our Standard Dedicated Cluster (DC) series supports robust three-node production systems. Pricing on the DC systems cannot be called “economical.” They are big servers designed for heavy duty, and there are at least three of them in each deployment.

SearchStax Solr Dedicated Cluster

SearchStax uses Apache ZooKeeper to manage configuration files across a cluster. In both DN and DC deployments, each Solr instance shares space with a ZooKeeper instance on the same server. That’s a single ZooKeeper for a DN system, and a three-instance “ensemble” for a production system. By design, ZooKeeper must have three (or more) instances to deliver fault-tolerant management of a production system. Since a production system requires three servers for ZooKeeper, we co-located three Solr instances on the same servers. Those were our Standard Dedicated Clusters.

Problem with Three Node Clusters

As our company gained experience in the Solr Cloud marketplace, we realized that some clients don’t need three Solr servers to achieve a fault-tolerant production environment. For many clients, even some large organizations using enterprise-grade CMS like Sitecore, two Solr servers will do the job. Unfortunately, a two-server system cuts ZooKeeper back to two instances, which simply doesn’t work. ZooKeeper must have three servers.SearchStax Solr NextGen Cluster

NextGen Dedicated Solr Clusters

The NextGen Dedicated Clusters use twin Solr servers matched with three independent ZooKeeper VMs which results in a typical NextGen cluster with five servers.

Fortunately, ZooKeeper servers can be very small, sometimes called micro-machines. That’s why a five-server NextGen cluster is more economical than our three-node full-sized Standard clusters.

Benefits of NextGen Clusters

The NextGen clusters are both more economical and more reliable!

  • Better Fit at Lower Cost – Customers with lower query loads were over-provisioned with three-node clusters. NextGen clusters are a better fit for these clients. (You can easily add Solr nodes if you need them.)
  • Better Performance and Reliability – In all NextGen plans, ZooKeeper instances run on their own dedicated nodes. That provides better performance and reliability because Solr and ZooKeeper aren’t competing for CPU time.
  • Simplifies Automatic Recovery – In a Standard Dedicated Cluster, when a node went down it stopped a Solr instance and a ZooKeeper instance at the same time. In the new architecture, a server failure stops only one of them. This greatly simplifies automatic recovery.

Key Take-aways

Providing Solr infrastructure and management is not an easy feat. It requires constant attention, effort, love and care. This is why using an experienced service like SearchStax is usually more efficient than trying to do it yourself.