Feb. 19, 2018
admin
|
Overview
Search Automation
- Allocating Hardware
- Choosing and installing an Operating System
- Configuring Disk layouts including swap space, system volumes, and data volumes
- Setting up a zookeeper cluster
- Configuring Solr parameters
- Setting up and configuring a load balancer
- Setting up monitoring and alerting
- Many more fiddly details
Doing the initial setup can easily take an entire day or more, even for experienced engineers. And once the system is set up, there is another matter of maintaining the cluster, attending to issues like OS patch upgrades, disk failures, node crashes, etc.
Fortunately, software such as SearchStax can ease the pain of setting up a cluster by leveraging modern automation and cloud technology. With SearchStax, a cluster can be setup in minutes by filling out some simple forms and pushing a button.
Point-and-click cluster creation is an amazing feat of modern computation. However, it turns out that automation techniques can be leveraged even further. By providing a provisioning API, a cluster can be set up by an automation job, requiring zero human intervention.
Why would one want to set up a cluster via an API? The primary use case for programmatic cluster creation is for integration of testing automation. While production search environments are long-lived, testing environments tend to be of an ephemeral nature: a cluster is created, tests are run, and the cluster is torn down. Of course, semi-permanent dedicated clusters can be provisioned for test environments. This has several downsides:
- There is a cost to having hardware up and running while it is not being utilized
- Long-running clusters suffer from bit rot – they can become outdated, and accumulate crusty data and configuration
- Once a cluster is set up, it is non-trivial to reconfigure it. For example adding additional nodes or changing storage capacity require specialized knowledge and careful attention to detail.
SearchStax provides a REST API that makes it easy to integrate cluster provisioning with an automated testing pipeline. In the next section, we walk through a typical sequence of API calls that such a job might perform.
API
Obtain Credentials
The first step in the process is to obtain credentials from SearchStax. Given a username and password, a token is retrieved
Request
curl -X POST \
https://app.searchstax.com/api/rest/v1/obtain-auth-token/ \
-d 'username=admin&password=s3cr3t'
Response
{
"token": "b09c"
}
List Deployments
Request
curl -X GET \
https://app.searchstax.com/api/rest/v2/account/MyAccount/deployment/ \
-H 'Authorization: Token b09c'
Response
{
"count": 1,
"next": null,
"previous": null,
"results": [
{
"name": "mydeployment1",
"uid": "ss123456",
"application": "Solr",
"application_version": "6.6.2",
"tier": "Silver",
"http_endpoint": "https://ss123456-ap-southeast-2-aws.searchstax.com/solr/",
"status": "Running",
"provision_state": "Done",
"termination_lock": true,
"plan_type": "DedicatedDeployment",
"plan": "DN1",
"region_id": "ap-southeast-2",
"cloud_provider": "Amazon Web Services",
"cloud_provider_id": "aws",
"num_additional_app_nodes": 0,
"deployment_type": "Dedicated Node",
"num_nodes_default": 1,
"num_additional_zookeeper_nodes": 0,
"servers": [
"ss123456-1"
]
}
]
}
Deployment
Now for the main event. We request a new cluster, passing parameters specifying the cluster name, the Solr version, the cloud provider, the region, whether a termination lock is enabled, and some other parameters.
Request
curl -X POST \
https://app.searchstax.com/api/rest/v2/account/MyAccount/deployment/ \
-H 'Authorization: Token b09c' \
-d 'name=MyDeployment2&application_version=7.1.0&plan_type=DedicatedDeployment&cloud_provider_id=aws&plan=DN1®ion_id=us-west-1&termination_lock=true&application=Solr'
Response
For a response, we receive an HTTP 201 CREATED status code
Toggle Termination Lock
Notice that in our creation request, we set termination_lock to true. This protects against accidental deletion of the cluster. To enable deletion, we need to set the termination_lock to false.
Request
curl -X POST \
https://app.searchstax.com/api/rest/v2/account/MyAccount/deployment/ \
-H 'Authorization: Token b09c' \
-d 'name=MyDeployment2&application_version=7.1.0&plan_type=DedicatedDeployment&cloud_provider_id=aws&plan=DN1®ion_id=us-west-1&termination_lock=true&application=Solr'
Response
For the response, we receive an HTTP 200 OK status code.
Delete the Deployment
Once we are through using the cluster, we will want to tear it down using the Delete API. This can be done as follows:
Request
curl -X DELETE \
https://app.searchstax.com/api/rest/v2/account/VicRoads2/deployment/ss677906/ \
-H 'Authorization: Token b09c'
Response
For the response, we receive an HTTP 200 OK status code.
Summary
