Streamline, Simplify, Accelerate: New MongoDB Features Reduce Complexity

At MongoDB World 2022, we announced several developer-centric features that provide more powerful analytics, streamline operations, and reduce complexity. In this post, we look at MongoDB Atlas Data Federation, MongoDB Atlas Search, MongoDB Atlas Device Sync and its Flexible Sync, and change streams.

As consumer expectations of the applications they use grow, developers must continue to create richer experiences. To do that, many are adding a variety of data systems and components to their architectures, including single-purpose NoSQL datastores, dedicated search engines, and analytics systems.

Piecing these disparate systems together adds complexity to workflows, schedules, and processes, however. For instance, one application could utilize a solution for database management, another solution for search functionality, and a third solution for mobile data sync. Even within an organization, teams often use different products to perform the same tasks, such as data analysis.

This way of building modern applications often causes significant problems, such as data silos and overly complex architectures. Additionally, developers are forced to spend extra time and effort to learn how each of these components functions, to ensure they work together, and to maintain them over the long term. It should not be the developer’s job to rationalize all these different technologies in order to build rich application experiences.

The developer data platform

For developers and their teams, cobbling together a data infrastructure from disparate components is inefficient and time-consuming. Providers have little incentive to ensure that their solutions can function alongside the products of their competitors. Further, internal documentation, which is key to demystifying the custom code and shortcuts in a bespoke architecture, might not be available or current, and organizational knowledge gets lost over time.

MongoDB Atlas, our developer data platform, was built to solve these issues. An ecosystem of intuitive, interlinked services, Atlas includes a full array of built-in data tools, all centered around the MongoDB Atlas database. Features are native to MongoDB, work with a common API, are designed for compatibility, and are intended to support any number of use cases or workloads, from transactional to operational, analytics to search, and anything in between.

Equally important, Atlas removes the hidden, manual work of running a sprawling architecture, from scaling infrastructure to building integrations between two or more products. With these rote tasks automated or cleared away, developers are free to focus on what they do best: build, iterate, and release new products.

MongoDB Atlas Data Federation

MongoDB Atlas Data Federation allows you to write a single query to work with data across multiple sources, such as your Amazon S3, Atlas Data Lake, and MongoDB Atlas clusters. Atlas Data Federation is not a separate repository of data, but a service to combine, enrich, and transform data across multiple sources, regardless of origin, and output to your preferred location.

With Atlas Data Federation, developers who want to aggregate data or federate queries do not need to use complex data pipelines or time-consuming transformations — a key advantage for those seeking to build real-time app features. Atlas Data Federation also makes it easier to quickly convert MongoDB data into columnar file formats, such as Parquet or CSV, so you can facilitate ingestion and processing by downstream teams that are using a variety of different analytics tools.

MongoDB Atlas Search

Rich, responsive search functionality has become table stakes for both consumer-facing and internal applications. But building high-quality search experiences isn’t always easy. Developers who use a third-party, bolt-on search engine to build search experiences have to deal with problems like the need to sync data between multiple systems; more operational overhead for scaling, securing, and provisioning; and using different query interfaces for database and search.

Built on the industry-leading Apache Lucene search library, MongoDB Atlas Search is the easiest way to build rich, fast, and relevant search directly into your applications. It compresses three systems — database, search engine, and sync mechanism — into one, so developers don’t have to deal with the problems that bolt-on search engines introduce. It can be enabled with a few API calls or clicks and uses the same query language as the rest of the MongoDB product family.

Atlas Search provides all of the features developers need for rich, personalized search experiences to users, like facets, now generally available, which offers users a way to quickly filter and navigate search results. With facets, developers can index data to map fields to categories like brand, size, or cost, and update query results based on relevance. This allows users to easily define multiple search criteria and see results updated in near real-time.

MongoDB Atlas Device Sync

With apps such as TikTok, Instagram, and Spotify, mobile users have come to expect features such as real-time updates, reactive UIs, and an always-on, always-available experience. While the user experience is effortless, building these abilities into a mobile app is anything but. Such features require lots of time and resources to develop, test, debug, and maintain.

MongoDB Atlas Device Sync is designed to help developers address mobile app data challenges, including limited connectivity, dead zones, and multiple collaborators (all with varying internet speeds and access) by gathering, syncing, and resolving any sync conflicts between the mobile database and MongoDB Atlas — without the burden of learning, deploying, and managing separate data technologies.

At World 2022, MongoDB announced Flexible Sync, a new way to sync data between devices and the cloud. Using Flexible Sync, developers can now define synced data using language-native queries and fine-grained permissioning, resulting in a faster, more seamless way of working — and one analogous to the way developers code and build.

Previously, developers had to sync full partitions of data; Flexible Sync enables synchronization of only the data that’s relevant. With support for filter logic, asymmetric sync, and hierarchical permissioning, Flexible Sync can reduce the amount of required code by 20% or more, and speed up build times from months to weeks.

Change Streams

Data changes quickly, and your applications need to react just as quickly. When a customer’s order is shipped, for instance, they expect an in-app or email notification — and they expect it immediately. Yet building applications that can respond to events in real time is difficult and often requires the use of polling infrastructure or third-party tools, both of which add to developer overhead. Latency and long reaction times result in data that is outdated, and poor experiences for users of that data.

Like Atlas’s Database Triggers, change streams enable developers to build event-driven applications and features that react to data changes as they happen. Along with reducing the complexity and cost of building this infrastructure from scratch, the new change stream enhancements (available in MongoDB 6.0) will enable you to determine the state of your database before and after an event occurs, so you can act on the changes and build business logic, analytics, and policies around it. That opens up new use cases, such as retrieving a copy of a document immediately after it is updated.

All of these updates and new capabilities focus on the critical need to eliminate complexity in order to build, deploy, and secure modern applications in any environment. Together, MongoDB helps solve what MongoDB president and CEO Dev Ittycheria called a key developer challenge in his MongoDB World 2022 keynote: reducing the friction and cost of working with data.