Free your data with the MongoDB Relational Migrator

Nothing is more frustrating than data that is just out of reach. Imagine wanting to combine customer behavior data from your CRM and usage data from your legacy product to trigger tailored promotions in your new mobile app, but not being able to locate the required data in the sea of tables in your relational database.

As MongoDB CTO Mark Porter explains in his MongoDB World keynote, the data that can make a difference might be locked up “somewhere that you can’t use.” Relying on his own hard-earned experience with data, Porter adds that this information can be trapped “in a schema with hundreds or thousands of tables that have built up over decades.”

“Schema is a huge part of this problem,” MongoDB product manager Tom Hollander explains during a presentation on MongoDB Relational Migrator at MongoDB World 2022. “So we’ve spent a lot of time building out the tools to enable you to map your tabular relational schema into a document schema and make use of the full power of the MongoDB document model.”

What is MongoDB Relational Migrator?

MongoDB Relational Migrator streamlines migrations from legacy data infrastructure to MongoDB by helping developers analyze relational database schemas, convert them into MongoDB schemas, and then migrate data from the source database to MongoDB. Currently, Relational Migrator is compatible with four of the most common relational databases: Oracle, SQL Server, MySQL, and PostgreSQL.

Migrator not only moves data from your relational database to MongoDB, but it also transforms it according to your new schema. As Hollander and MongoDB product marketing director Eric Holzhauer point out, developers often use a mix of software and tools (e.g., extract-transform-load pipelines, change data capture (CDC), message queues, and streaming) to execute migrations, which can be complicated, risky, and error-prone. Relational Migrator provides a single tool that can streamline the process while simultaneously ensuring that your data lands in an organized, logical manner.

By simplifying schema translations — one of the most complex, difficult parts of any relational migration — Relational Migrator grants developers and other technical teams a greater degree of control over (and increased visibility into) their new MongoDB schema. The result is to make data more accessible for analysis and decision making. “Now I can get at the data in my program without going through a translation layer,” Porter explains.

Migration mode: Snapshot or ongoing?

Migrator provides two modes of data transfer: a one-time snapshot or a continuous sync (which will be available later this year). To help decide which mode you should use, consider whether you can move over to MongoDB and immediately decommission your previous database or whether you need to keep your existing relational database up and running.

Organizations may wish to keep their relational database for various reasons, such as testing the effectiveness of your proposed document schema, running out the contract or licensing agreement to avoid expensive fees, or keeping old databases available for audits. In this situation, you can keep your relational database running so that Relational Migrator will continue to push data from your source to your new MongoDB clusters.

The case for modernization

At their core, relational databases are difficult to change — a liability in a world where more (and more varieties of) data is being generated than ever before. Relational databases were developed to meet a core set of conditions and assumptions, including limited RAM, the need for reliable storage, and the probability that such storage is slower than the network. Due to the rise of cloud computing, these circumstances no longer exist. Now, RAM is plentiful, failover is possible, and data can be retrieved faster than it can be moved over the network.

“Making schema changes to support modified business requirements can be extremely painful,” Hollander explains. “So anytime you have to change the schema, you probably have to look at some kind of in-place migration.” Teams also have to ensure that there are no duplicate columns, stored data, or IDs anywhere within the tables. This process requires significant work and results in long, winding, entity-relationship documents, all of which are products (and processes) that don’t scale easily.

For an older technology, relational databases are also more costly than one would expect. Because of their architecture, relational databases are nearly impossible to virtualize. This makes scaling very costly, because it requires the addition of bare metal servers rather than simply adding on virtual machines. Some relational products are also known for overly punitive licensing agreements (and the resulting audits). As Hollander points out, “Sometimes, someone will knock on the door with a clipboard when you least expect it, doing audits for your licenses.”

In contrast, non-relational, cloud-native databases were designed from the beginning to meet the challenges of modern application development — which results in more agility and velocity for developers. In his experience, Hollander and his team have seen costs that were up to 70% lower, as well as anywhere from a three- to five-fold increase in developer productivity and speed.

Additionally, the cloud is quickly gaining ground, especially among your competitors. Gartner predicts that by 2025, more than half of enterprise IT spending (in sectors like system infrastructure, business processes, and applications) will shift to the cloud — a total of almost $1.8 trillion.

The limits of Relational Migrator

As Hollander points out, Relational Migrator is only a tool — one intended to facilitate schema mapping, providing many abilities and options for effective schema design. “It’s not a silver bullet that will immediately modernize your application portfolio,” Hollander says. “It’s not going to do everything for you. You still have to do the planning.”

Furthermore, because database schema is a tricky topic even for seasoned experts, Hollander recommends that developers would benefit from working with architects, consultants, and partners — especially if they’re not familiar with MongoDB or schema design best practices.

Relational Migrator does not yet support continuous replication, which would enable your relational database and MongoDB clusters to coexist for an extended period of time. However, Hollander says that work on this feature is ongoing and it will be available in the future, along with additional capabilities like schema recommendations, an integration for the MongoDB Atlas developer data platform, and more.