Transactions Background Part 2: Logical Sessions in MongoDB

Recent features in MongoDB such as multi-document ACID transactions have required foundational enhancements in MongoDB to enable them. One of these foundational changes is the addition of Logical Sessions.

In the first part of this six part series, we looked at the handling of timestamps within the server and how it enabled multi-document transactions. In this part we move on to logical sessions and how they are used to bind together a client's actions as they pass through a MongoDB cluster.

Summary

By creating logical sessions, it is now possible for the resources being used in a single operation or a multiple-operation transaction to be tracked as they are consumed throughout the system. This enables simple, precise cancellation of operations and distributed garbage collection.

Background

Historically, there have been many operations in MongoDB that would have benefitted from being tracked as they progressed from the client, through the mongos query routers and out to the shards and replica sets that make up a cluster. There was, though, no such identifier to track these operations with, and so the system relied on a range of heuristics.

Logical Sessions

The solution for MongoDB was to create the Logical Session and the Logical Session Identifier. This is a small unique identifier, referred to as the lsid, that can be attached by a client to its communications with the MongoDB cluster which will, in turn, attach the lsid to any resources being used by that client.

The lsid is automatically generated at the client by the MongoDB drivers eliminating the need to call out to a central id service. It is composed of an id, which is a GUID (Globally Unique ID) generated by the client, and a uid, which is a SHA256 digest of the username.

From MongoDB 3.6 onwards, any client operation is associated with a Logical Session. The Logical Session Identifier, lsid, will then be associated with the operation of a command across the cluster.

Logical Sessions and Cancellation

Any operation consumes resources. For example, a find() operation will create cursors in all the relevant shards in a cluster. Each cursor will start acquiring results for its first batch to return. Before logical sessions existed, to cancel an operation like this would mean traversing all the shards with administration privileges, working out which activity was associated with your operation and then stopping it.

That process also introduced additional complexities. Where say the mongos process associated with the issuing of the command went down, that cancellation process would be even harder and you would have to wait for the cursors to build their first batch and then timeout waiting to return it.

With logical sessions, the entire process is simpler. It is possible to issue a kill command for a specific logical session to the cluster. As all resources, including the cursors, are tagged with a logical session identifier, it becomes a relatively simple operation to stop and free those resources with a particular lsid. It is also possible, because the user id is part of the lsid, to issue a command that removes all session resources for a particular user.

Logical Sessions and Distributed Garbage Collection

Timeouts on resources in MongoDB have previously been a local affair; the node where the resource resided would decide whether or not the resource had timed out and needed garbage collecting. Plans for future MongoDB features required that timeouts and garbage collection were aware of the cluster. To enable that, changes were made using lsids as a foundation.

In MongoDB 3.6 mongod and mongos processes began to do two things. First, run a controller process to manage sessions, and second, maintain a list of lsids that connected to the process in the controller. Every 5 minutes, controllers would then synchronize that list with a sessions collection, updating the time the session was last used. That last use time becomes the basis for a TTL index which triggers after 30 minutes. That triggering indicates that this is a session that has not been used by any controller for 30 minutes. This makes the session, and the resources it uses, eligible to be cleaned up.

Logical Sessions and Transactions

By tagging all the operations and resources with a logical session id, it is now easier to manage long-lived and widely distributed operations in MongoDB. The logical session id has immediate utility in cancellation and garbage collection scenarios and is a foundation for other MongoDB 4.0+ features. By ensuring a transaction takes place within a session, it enables the storing and cleaning up of that transaction whether it successfully commits or aborts.

In the next part of the series, we'll look at Local Snapshot Reads in MongoDB and how they enabled transactions.