This is a guest post from Brent Mills, Principal Infrastructure Engineer at Vyopta, an analytics company that enterprises use to monitor, support, and optimize their video and unified collaboration systems.
MongoDB has always been very attractive to developers in part due to the ease with which you can set up, manage, and interact with the database. Time-to-market is increasingly important, as businesses require increasing levels of speed and flexibility in how they deliver new products and features to their customers.
But as with any other back-end system, you have to manage monitoring and alerting, the clusters (and all of the idiosyncrasies of every distributed system), backups, encryption, provisioning for scale, etc.; These are all things operations teams have to consider in order to deliver production-ready applications, but they’re also things that can delay innovation velocity and can present operational overhead.
By moving our database to the MongoDB cloud service, we’ve been able to remove much of the operational load, saving us over 15 hours of work per month, enabling us to increase efficiency as we’ve moved apps from development to production.
Our Use Case
Below is a simplified version of our pre-MongoDB Atlas setup. I would imagine this looks familiar to many of you:
First off, there’s nothing wrong with this infrastructure, in fact it’s served us pretty well we’ve experienced very few issues. Even so, we were spending a non-trivial amount of time each month on operational overhead. Troubleshooting backup failures, relocating instances because of a network revamp, spinning up new environments for testing, configuring encryption for SOC2 compliance and additional requirements for customers storing data outside of the United States. All of this maintenance work adds up, slowing us down. We’d rather have our development team pushing out new features, and our DevOps team running our CI/CD process, or supporting new initiatives like our move to a microservices architecture.
MongoDB Atlas to the Rescue
Personally I’ve never been a big fan of managed cloud services. They always seem to be lacking important features, have frustrating usability issues, and/or monetarily punish you in creative ways. MongoDB Atlas, however, was very attractive to us for the following reasons:
- Fully functional MongoDB instance. With the ability to upgrade versions in a few clicks you can always have the newest feature set. It’s also close to identical to what you would deploy on-premise, not some specific “hosted” variation that can’t do x, y, or z.
- Easy backup and restore. Click to enable, edit the schedule in one pane with cost estimation and options for separate daily, weekly, and monthly retention periods. Straight-forward restore with option to download or query data in place, without having to restore the complete backup.
- Integrated metrics and alerts. No more setting up agents on every node. Built in alert integrations with services like PagerDuty.
- Easy scale up/down. Need more RAM? Edit the cluster configuration and select a bigger size. Need more disk space? Drag a slider.
- Easy to provision new clusters. A few mouse clicks or an API call and we can have a cluster with encryption, sharding, and backups. The GUI also shows you a direct cost estimation.
- No nickel and diming. The folks at MongoDB could have easily added extra charges for each cluster modification, encryption, restores, alerts, and various other things to muddy the water, but they did not. You pay for the managed instances and backups you use and that’s it.
- No lock-in. Migrating data in and out of Atlas is extremely simple.
After experimenting with the various features it became pretty obvious this would eliminate the vast majority of our overhead
With Atlas it now takes about 10 minutes to provision a full cluster with monitoring and any other features we need. We can even create a new group, which is sort of like a linked account, and give developers the ability to create their own clusters in an isolated environment without waiting on us.
This is Going to Hurt...Oh Wait...
So with all of this the only potential downside has to be the price, right? It turns out this was the most pleasant surprise of all.
Since our Atlas clusters are running on AWS and the cluster sizes are based on EC2 instances, we can make some rough comparisons on price:
- M30 cluster w/ 8gb RAM and 100gb storage - ~$430/mo
- 3 m4.large AWS instances w/ 8gb RAM and 100gb storage - ~265/mo
This does not include storage cost for backups or data transfer but gives us a good idea. If we’re running on an M30 instance in production, we don’t have to save many man hours to justify a ~$165/month difference. Another thing to note is that most people will over-provision their own clusters because it’s not easy to scale. With Atlas, we can eliminate this waste.
Going Forward
As we continue to build out our continuous deployment systems we keep looking for ways to dynamically provision environments and test for scale. Another thing Atlas offers in this respect is an API that encompasses most of their GUI features. This allows us to programmatically spin up a cluster as part of a test run:
We can also use a Docker container for this, but if we’re trying to replicate our production environment or test performance it can be quite difficult with containers.
The one thing missing from the API at the moment is access to metrics. It would be a killer feature if we could spin up a cluster, run some tests, then include production-like database performance stats in our test results. I know after speaking with the Atlas team, that this is coming in the next few months.
Conclusion
We have been using MongoDB Atlas in production for around 6 months and have been very happy with it. Our migration consisted of stopping MongoDB, running a mongodump, then simply restoring it with mongorestore using the new Atlas connection strings. We were able to schedule a maintenance window to accomplish this but there are also now options for doing a live migration if you require the uptime. Because of the simplicity and minimal effort involved in migrating the data, we were able to see cost savings in the first month. Clear, upfront cost estimates have been especially nice since I seem to spend a lot of my time in convoluted pricing calculators. Hopefully our experience inspires others to to give Atlas a try (which you can do without cost using the MongoDB Atlas free tier).