Atlas Stream Processing is Now Generally Available!

We're thrilled to announce that Atlas Stream Processing—the MongoDB-native way to process streaming data—is now generally available, empowering developers to quickly build responsive, event-driven applications!

Our team spent the last two years defining a vision and building a product that leans into MongoDB’s strengths to overcome the hard challenges in stream processing. After a decade of building stream processing products outside of MongoDB, we are using everything that makes MongoDB unique and differentiated—the Query API and powerful aggregation framework, as well as the document model and its schema flexibility—to create an awesome developer experience.

It’s a new approach to stream processing, and based on the feedback of so many of you in our community, it’s the best way for most developers using MongoDB to do it.

Let’s get into what’s new.

What's new in general availability?

1. Production Readiness
   Ready to support your production workloads, ensuring reliable and scalable stream processing for your mission-critical applications.

2. Time Series Collection Support
   Emit processor results into Time Series Collections. Pre-process data continuously while saving it for historical access later in a collection type available in MongoDB Atlas built to efficiently store and query time series data.

3. Development and Production Tiers
   Besides the SP30 cluster tier available during the public preview, we’re introducing an SP10 tier to provide flexibility and a cost-effective option for exploratory use cases and low-traffic stream processing workloads.

4. Improved Kafka Support
   Added support for Kafka headers allows applications to provide additional metadata alongside event data. They are helpful for various stream processing use cases (e.g., routing messages, conditional processing, and more).

5. Least Privilege Access
   Atlas Database Users can grant access to Stream Processing Instances and enable access to only those who need it. Read our tutorial for more information.

6. Stream Processor Alerting
   Gain insight and visibility into the health of your stream processors by creating alerts for when a failure occurs. Supported methods for alerting include email, SMS, monitoring platforms like Datadog, and more.

Why Atlas Stream Processing?

Atlas Stream Processing brings the power and flexibility of MongoDB's document model and Query API to the challenging stream processing space. With Atlas Stream Processing, developers can:

• Effortlessly handle complex and rapidly changing data structures

• Use the familiar MongoDB Query API for processing streaming data

• Seamlessly integrate with MongoDB Atlas

• Benefit from a fully managed service that eliminates operational overhead

Customer highlights

Read what developers are saying about Atlas Stream Processing:

What's ahead for Atlas Stream Processing?

We’re rapidly introducing new features and functionality to ensure MongoDB delivers a world-class stream processing experience for all development teams. Over the next few months, you can expect to see:

1. Advanced Networking Support
   Support for VPC Peering to Kafka Clusters for teams requiring additional networking capabilities

2. Expanded Cloud Region Support
   Support for all cloud regions available in Atlas Data Federation

3. Expanded Cloud Provider Support
   Support for Microsoft Azure

4. Expanded Data Source and Sink Support
   We have plans to expand beyond Kafka and Atlas databases in the coming months. Let us know which sources and sinks you need, and we will factor that into our planning

5. Richer Metrics & Observability
   Support for expanded visibility into your stream processors to help simplify monitoring and troubleshooting

6. Expanded Deployment Flexibility
   Support for deploying stream processors with Terraform. This integration will help to enable a seamless CI/CD pipeline, enhancing operational efficiency with infrastructure as code. Look out for a dedicated blog in the near future on how to get started with Atlas Stream Processing and Terraform.

So whether you're looking to process high-velocity sensor data, continuously analyze customer data to deliver personalized experiences, or perform predictive maintenance to increase yields and reduce costs, Atlas Stream Processing has you covered. Join the hundreds of development teams already building with Atlas Stream Processing. Stay tuned to hear more from us soon, and good luck building!