Taking the stage at MongoDB World today, CTO Eliot Horowitz highlighted some of the features in the forthcoming MongoDB 4.2 that extend its lead in database technology. Distributed Transactions, Field Level Encryption, Wildcard Indexes and On-Demand Materialized Views are top of that list.
Distributed Transactions
The 4.2 release is due within weeks and will include Distributed Transactions that build upon 4.0's Multi-Document ACID Transactions to provide an even higher level of transaction capability. With Distributed Transactions, transactions will be able to work across MongoDB Atlas's Global Clusters and allow large sharded deployments, including those that span multiple regions, to handle these operations.
Distributed Transactions does all that while maintaining full atomicity of the multiple statement operations. If any the transaction fails to commit on a shard, it aborts on all the shards involved in that transaction. The best part is that using them is exactly the same as using 4.0's Multi-Document Transactions. That means you use the same familiar relational-style transactions syntax and your transaction aware code doesn't need to change when you decide to scale out.
Bringing Distributed Transactions to 4.2 also enabled a number of other new features. Mutable shard key values, larger transactions, driver-side helpers and logged metrics make for easier development too. Take mutable shard key values. With the ability to change the key on a record, that record can now migrate records between shards allowing global redistribution of data or tiered storage for aging out old documents, as long as that can take place during the transaction run time limit of 60 seconds.
With Distributed Transactions, MongoDB now covers most of the scenarios a developer might anticipate, from starting development only needing document level ACID guarantees, through to needing to work with multi-document operations in a cluster, to scaling transactions over a globally distributed cloud service.
Client Side Field Level Encryption
MongoDB has delivered a range of security options, from end to end encryption (E2E) with the encryption of data in flight and at rest, to fine-grained Role-Based Access Control (RBAC) and comprehensive auditing capabilities. With the rigor required by compliance to modern legal requirements, there comes the new need to ingest, query and store encrypted data securely in the database without the database having the ability to read it.
This is data that must always be encrypted within the database and only ever decrypted at a client. This challenge is being met with MongoDB's new Field Level Encryption option. With this option, the server has no knowledge of the fields being encrypted. All the encryption and decryption is done at the driver level.
It's the driver that, when it sees an encrypted field is involved in a write or query command, gets appropriate keys from the key manager, encrypts the data, and sends it to the server. The server then only sees ciphertext and has no knowledge of the keys. When results are returned from the server, that ciphertext is sent by the server to the driver and it's there that the driver, which already has the keys, decrypts it. From an application's point of view, this mechanism is completely transparent.
With the key management away from the database, there's the ability to have keys bound to individual records or users. This in turn makes deleting that users encrypted information easy. By deleting the relevant key in the key management system effectively deletes all data that has been encrypted with that key. It also means that you can safely use a managed service like MongoDB Atlas, knowing that the data is never visible unencrypted in logs, memory or any other part of the infrastructure.
If you've ever wrestled with the complexity of hand-coding encryption management of data, the clean design of Field Level Encryption in MongoDB 4.2 will really enhance your productivity.
Wildcard Indexes
Wildcard Indexes are an answer to the question that comes up with the flexible document model; what do I index? While some fields are easily identified by the workload as something to create an index for, there are situations where the answer is less clear cut. For example, you may have a variable set of fields in a sub-document which express product characteristics and as you sell everything from battleships to pencil sharpeners, that's a lot of different characteristics, and you want to search them quickly. With 4.2's Wildcard Indexes, you get to create an index for all the fields within that sub-document and a single index is created which covers all those fields and their values. You can read more about this new feature in "Coming in MongoDB 4.2 - 1: Wildcard Indexes", the first of our looks at 4.2's new features.
On-Demand Materialized Views
Creating a view of your dataset is a great way to present results of computations, pre-compute calculations or just filtered sections of your dataset to client applications. All that was needed was to put the $out operator at the end of your MongoDB aggregation pipeline to create a new collection. When you wanted to update it, you re-ran it and wrote out all the records again. Not any more: In MongoDB 4.2 you can use the new $merge operator to update your view collection. You get control of how new documents and updates are handled and can have indexes on your new views for faster access. And as added bonuses, these new materialized views can be sharded for those larger, persistent views, and you get complete control over when the views are updated.
These are just some of the highlights within the forthcoming MongoDB 4.2. Look out for previews and tutorials in the coming weeks which will show you how to put them to work for you.