Programmatic API Management of your MongoDB Atlas Database Clusters - Part II

In the blog post, "Programmatic API Management of your MongoDB Atlas Database Clusters", we introduced how to use curl to launch and edit a MongoDB Atlas cluster. The concepts in that post helped establish the groundwork for being able to launch MongoDB Atlas clusters with many of the popular DevOps automation tools available in the market.

We recommend reading the post linked above as a prerequisite to what we will cover here — some common use cases with the MongoDB Atlas API.

The GUI versus API

While point-and-click actions in the MongoDB Atlas GUI are easy, they do not necessarily scale well when rapid deployments or modifications are required or when you’re managing many clusters. You may also prefer to initiate new database build outs or changes using pre-existing automation/orchestration tools and processes. The MongoDB Atlas API permits you to implement your database infrastructure as code by enabling you to programmatically define and execute on new requirements.

This allows you to easily integrate Atlas as part of your DevOps automation process and grants better visibility into how your application (and its associated data layer) evolves over time by permitting git check-ins with a changelog.

What's available in the MongoDB Atlas API

Atlas exposes a REST API that include the following features:

JSON entities - All entities are expressed in JSON.

Digest authentication - To ensure that your API key is never sent over the network, API requests are authenticated using HTTP Digest Authentication.

Browsable interface - Using a consistent linking mechanism, you can browse the entire API by starting at the root resource and following links to related resources.

Security - In addition to Digest Authentication, the API is only accessible via HTTPS, and certain calls requiring even more security are protected by user-defined whitelists. Further, an API user’s capabilities are restricted by their assigned role(s). For example, a user with the Read Only role within a particular group is not allowed to modify any resources within that group.

The Atlas API provides the following resources:

Root	The starting point for the Atlas API.
Group IP Whitelist	Retrieves and edits the IP whitelist, which controls client access to your MongoDB clusters.
Clusters	Provides access to your MongoDB cluster configuration
Database Users	Retrieves and edits the MongoDB users who have access to your MongoDB clusters.
Alerts	Retrieves and acknowledges alerts.
Alert Configurations	Retrieves and edits alert configurations, which define the conditions that trigger alerts and the methods of notification.

As you can see, the API allows us to modify a number of different dimensions of our clusters.

Informational Queries.

First, click on your username on the upper right hand side of the browser; here you'll find the "Account" dropdown.

Once in the "Account" page, you'll find a section called "Public API Access". Follow the instructions on creating a key and whitelisting it from the aforementioned post. I have found if using bash that creating a variable for the token is helpful.

Example:

bash-3.2$ export TOKEN="exam-plest-ring-foryour-token"
bash-3.2$ echo $TOKEN
exam-plest-ring-foryour-token

I can now use the Clusters resource to get information about my MongoDB Atlas cluster. I will need my account's "group ID" in order to accomplish this. To get this ID, you can just review the URL in your browser and grab the appropriate string:

Here's mine:

https://cloud.mongodb.com/v2/588b776f96e82110b163ed93#clusters

Like the token we used before, let's go ahead and turn this into a variable as well:

bash-3.2$ export GROUPID="588b776f96e82110b163ed93"
bash-3.2$ echo $GROUPID
588b776f96e82110b163ed93

Now that the proper info we need has been configured, we can now start querying details about our cluster (I used python -m json.tool to pretty print our output):

bash-3.2$ curl -X GET -u "jay.gordon@mongodb.com:$TOKEN" --digest "https://cloud.mongodb.com/api/atlas/v1.0/groups/$GROUPID/clusters" |  python -m json.tool

{
    "links": [
        {
            "href": "https://cloud.mongodb.com/api/atlas/v1.0/groups/588b776f96e82110b163ed93/clusters?pageNum=1&itemsPerPage=100",
            "rel": "self"
        }
    ],
    "results": [
        {
            "backupEnabled": false,
            "diskSizeGB": 10.0,
            "groupId": "588b776f96e82110b163ed93",
            "links": [
                {
                    "href": "https://cloud.mongodb.com/api/atlas/v1.0/groups/588b776f96e82110b163ed93/clusters/records",
                    "rel": "self"
                }
            ],
            "mongoDBMajorVersion": "3.4",
            "mongoDBVersion": "3.4.6",
            "mongoURI": "mongodb://records-shard-00-00-x8fks.mongodb.net:27017,records-shard-00-01-x8fks.mongodb.net:27017,records-shard-00-02-x8fks.mongodb.net:27017",
            "mongoURIUpdated": "2017-04-12T18:33:00Z",
            "name": "records",
            "numShards": 1,
            "providerSettings": {
                "diskIOPS": 100,
                "encryptEBSVolume": false,
                "instanceSizeName": "M10",
                "providerName": "AWS",
                "regionName": "US_EAST_1"
            },
            "replicationFactor": 3,
            "stateName": "IDLE"
        },
        {
            "backupEnabled": false,
            "diskSizeGB": 20.0,
            "groupId": "588b776f96e82110b163ed93",
            "links": [
                {
                    "href": "https://cloud.mongodb.com/api/atlas/v1.0/groups/588b776f96e82110b163ed93/clusters/platespace-v2",
                    "rel": "self"
                }
            ],
            "mongoDBMajorVersion": "3.4",
            "mongoDBVersion": "3.4.6",
            "mongoURI": "mongodb://platespace-v2-shard-00-00-x8fks.mongodb.net:27017,platespace-v2-shard-00-01-x8fks.mongodb.net:27017,platespace-v2-shard-00-02-x8fks.mongodb.net:27017",
            "mongoURIUpdated": "2017-06-18T18:03:53Z",
            "name": "platespace-v2",
            "numShards": 1,
            "providerSettings": {
                "diskIOPS": 100,
                "encryptEBSVolume": false,
                "instanceSizeName": "M20",
                "providerName": "AWS",
                "regionName": "US_EAST_1"
            },
            "replicationFactor": 3,
            "stateName": "IDLE"
        }
    ],
    "totalCount": 2
}

We can see from above that I have two separate clusters currently running in this group. This exercise can be useful if you want information on all existing infrastructure because it's easy to run this query and export the data into a larger report. If you're running a number of different services with different providers, you can simplify the process of researching the specs by running an API call from something such as Jenkins.

Modifying the database cluster

Modifying the database cluster With the information I've received from the API, I'm able to get full specs on the records cluster in my group. In this example, we'll focus on a very common need — disk space. The disk space output from our API states the following: "diskSizeGB": 20.0

Let’s assume that we're running low on hard disk and want to utilize an API call to increase the size to 30.0 GB. The API resource for diskSizeGB can be found in the "Modify a Cluster" section of the documentation.

The syntax here requires that I POST to the API information that includes my user, my group id, my token and a payload that includes details of what modifications should be made.

I've put the following line together to execute with the curl binary that will modify my disk size to 32GB.

bash-3.2$ curl -i -u "jay.gordon@mongodb.com:$TOKEN" --digest -H "Content-Type: application/json" -X PATCH "https://cloud.mongodb.com/api/atlas/v1.0/groups/$GROUPID/clusters/records" --data '{"diskSizeGB" : 32}'

Once I execute this, I get a JSON document back indicating that the state is currently changing and our requested modification is underway:

{
    "backupEnabled": false,
    "diskSizeGB": 32.0,
    "groupId": "588b776f96e82110b163ed93",
    "links": [
        {
            "href": "https://cloud.mongodb.com/api/atlas/v1.0/groups/588b776f96e82110b163ed93/clusters/records",
            "rel": "self"
        }
    ],
    "mongoDBMajorVersion": "3.4",
    "mongoDBVersion": "3.4.6",
    "mongoURI": "mongodb://records-shard-00-00-x8fks.mongodb.net:27017,records-shard-00-01-x8fks.mongodb.net:27017,records-shard-00-02-x8fks.mongodb.net:27017",
    "mongoURIUpdated": "2017-04-12T18:33:00Z",
    "name": "records",
    "numShards": 1,
    "providerSettings": {
        "diskIOPS": 100,
        "encryptEBSVolume": false,
        "instanceSizeName": "M10",
        "providerName": "AWS",
        "regionName": "US_EAST_1"
    },
    "replicationFactor": 3,
    "stateName": "UPDATING"
}

Curl is easy? But what's next?

One of the keys to a fast moving DevOps environment is automating common tasks. Many of the common DevOps tools, like Chef, have functionality to implement an HTTP GET or POST to an API. By utilizing the Atlas API, you can quickly create, modify, and destroy clusters as part of your configuration management process.

Here are some HTTP functions embedded in each of the major configuration management tools which you can leverage with the Atlas API to work with your cluster:

Chef	http_request
Puppet	puppet-http or Exec
Ansible	uri Module
Terraform	HTTP Provider

By leveraging the information from this blog post and these tools, you can implement a fully codified version of your infrastructure, reduce the time it takes to build and modify your database, and seamlessly launch, scale, or turn down clusters based on your needs.

Get to work

Get started for free or click here to learn how you may be eligible for 3 free months if you migrate an existing workload.