What are My Options for Implementing Solr?
For search-based applications, there have typically been two options to develop, manage and maintain a Solr Infrastructure:
- Do-it-yourself or DIY
- Buy it by purchasing a completed solution from a third party
SearchStax brings an interesting third option to the equation. Our Managed Solr Search or Solr-as-a-Service combines the ease of buying a solution with the power, flexibility and control benefits of DIY.
What’s the Biggest Reason to Use SearchStax Managed Solr Search?
Your developers’ time is a very valuable commodity. They should spend their time focused on value-added activities such as building better search experiences instead of dealing with the mundane operational details of deploying, managing and scaling Solr infrastructure.
The SearchStax Solr-as-a-Service solution gives developers their time back and automates a huge chunk of this operational work without making them cede any flexibility or give up any control over their Solr infrastructure.
From a time standpoint, SearchStax provides significant time savings compared to DIY.
Here’s a summary of our internal analysis comparing SearchStax vs. Do-it-Yourself for the initial Solr setup and ongoing Solr support:
|SearchStax Total Time||Do-it-Yourself Total Time|
|Setting Up Solr||30 to 115 Minutes||9 to 47 Days|
|Ongoing Solr Support||15 to 60 Minutes||23 to 72 Days|
Using the time savings above and $125 per hour as a typical all-in cost for a DevOps resource (including salary, taxes, overhead, management, learning time and Rework), the cost differential between SearchStax and Do-it-Yourself is dramatic. And this doesn’t factor in the opportunity costs of doing something productive for your customer, like using the time to deliver a better customer search experience.
So, how do we do it? Let’s look at the tasks and functions involved in setting up and supporting a Solr environment to see all of the functions supported by our solution and how hard it can be to do it by yourself.
What’s Involved in Setting up Solr in the Cloud?
There are five key steps to set up Solr in a cloud infrastructure:
Now that you are ready to start setting up your search infrastructure, let’s build out the initial development cluster. SearchStax Managed Solr Search gives you complete control over the specific parameters of your cluster, but at a definitive time and effort savings. When you do it yourself, standing up a cluster can range from a couple of days to weeks depending on the complexity of the application, organization and how many hiccups pop up. With SearchStax, that timeframe is shrunk down to minutes.
Monitoring and Alerting
Once your cluster is built and running the way you want it, you’ll need to implement a monitoring system. You need to ensure that you are aware of the health of not only your servers and cluster as a whole but also individual metrics that may have an adverse impact on your end-user search performance or stability of your cluster.
Most search applications will create a large log file pretty quickly. Without proper log management in place, this can quickly fill your drive. In addition, to get easy access to those logs, you either have to put something in place for your developers to request access to the logs or develop a user access management system so that they can log in to the servers and get the logs
Backup Setup and Testing
Regularly backing up your application is a best practice in our industry. In case of data corruption, unexpected hardware malfunction or accidental situations, you can go back to your previously stored backups and restore your entire application quickly. For search systems with large amounts of data, re-indexing of your entire index can take many hours if not up to days. And sometimes, reindexing is not an option.
Building Disaster Recovery
We all hope disasters don’t happen, but smart companies are better prepared for it. Whether that’s because of ensuring zero downtime for your customers or service level agreement (SLA) guarantees or compliance, a disaster recovery site in a separate data center and/or location is advised and can be time-intensive to define and set-up. Check out the Disaster Recovery options from SearchStax.
What’s Involved in Ongoing Support of Solr in the Cloud?
Once the initial setup and Solr deployment is completed, there are several ongoing management and support activities that must be addressed.
Any application that’s built to be successful is eventually going to need to scale. Whether that’s because of more data that you want to bring into your Solr deployments or if it’s more query traffic or use cases that start to put more query load on your clusters. Whether that’s vertical scaling or horizontal scaling, you’ll have to ensure that the process of scaling is designed in a way that you can easily scale up or down your clusters. More so, you’d like not to have any downtime or performance impact to your end-users while this activity is being performed.
Apache Solr version upgrades consist of major and minor version upgrades. Major Solr version upgrades can contain changes that are not backward-compatible with existing applications. However, minor version upgrades can be generally applied without having an impact on your application.
Backup and Recovery
As your cluster configuration changes over time, you need to evolve your backup and recovery procedures and/or scripts that you’ve put in place. These changes can be either due to node changes or due to the fact that you have a unique collection / shard / replication setup that needs to be backed up appropriately.
After you set up monitoring, you’ll need to have a process in place to respond to the things that your monitoring uncovers as you support your production application. Working with numerous applications and systems, we’ve seen many issues such as performance degradation, SSL certificate updates, GC pauses, authentication issues, JVM heap changes, OS level security patches, node replacements, disk drive failures, network failures, zookeeper queue issues, replica recovery, security audits, and more.
What Should We Do Next?
Building your own search application can be a very time-consuming process. Even when you are “done”, there is still work to do in the form of responding to issues and events, applying security patches and upgrades, and keeping software versions up to date.
One of our customers recently summed up the SearchStax Managed Solr Search vs. DIY argument:
In our whitepaper, Why SearchStax is Better than DIY Solr Infrastructure, we provide a detailed step-by-step comparison between using SearchStax and DIY.