Transactions Background Part 4: The Global Logical Clock

Recent features in MongoDB such as multi-document ACID transactions have required foundational enhancements in MongoDB to enable them. One of these foundational changes is the introduction of a Global Logical Clock.

Summary

Time synchronization within MongoDB has been enhanced by a move to a global logical clock. Implemented as a hybrid logical clock and protected by encryption, the global logical time now spans an entire sharded cluster. A consistent global view of time is essential for multi-document ACID transactions as it allows for global snapshots to be taken.

Background

MongoDB’s clock is tracked in the oplog of the primary in a MongoDB shard. Each operation, that is everything that creates a durable change in the data, is recorded in the oplog. As each operation is recorded, it ‘ticks’ the logical clock for the primary and the clock “time” is recorded with the operation in the oplog. This provides an ordering to the oplog which can be used to synchronize all nodes within the shard.

Hybrid Logical Clocks

While shards may have their own clock, that doesn’t work for the entire cluster; the clocks will have no common reference and increment on their own time. Enter the hybrid logical clock, as implemented in MongoDB 3.6.

This hybrid clock combines the system time and the counter for operations that happen at the same physical time to create a hybrid timestamp. This new timestamp is gossiped (or spread) by all connections that are established throughout the cluster. When a message with a timestamp is received by a server, if the timestamp is after that node’s current timestamp, then it updates their latest seen timestamp with the new value. The latest time seen is immediately incremented when a new modification occurs .

Tamper Prevention

There is always the possibility of an attacker injecting a bad time and count into the system such that it would “run out of time” to increment into for new operations, thus the shard would become unable to accept any new modifications. This issue is mitigated by creating a hash of the timestamp using the primary’s own private key. This hash is used to verify the source of the timestamp is within the zone of trust formed by the cluster; if an attacker can get their hands on the private key, then there’s a much broader compromise of the system at play.

Global Logical Clocks and Transactions

With a hybrid logical clock, it becomes simpler to coordinate and synchronize multiple nodes in a replica set and across multiple shards in a cluster. As the scope of transactions expands across multiple shards in MongoDB 4.2, having a reliable, synchronized clock that works with that scope is an essential foundation stone. Ensuring the clock cannot be tampered with makes that foundation more reliable.

In the next part of the series, we'll look at the safe secondary reads in MongoDB and what they mean for transactions.