RabbitMQ previously stored data using Mnesia, whose design meant that when a network partition happened, the cluster could end up running separate, independent versions of itself. Each partition would keep processing data on its own, without knowing what the other side was doing.

This led to a dangerous state called split-brain, where different sides of the partitions processed data independently and produced inconsistent results.

To avoid that, RabbitMQ introduced pause minority mode for Mnesia: whenever a partition happened, the smaller group of nodes (the minority) would be stopped. A stopped node can’t do anything at all: it doesn’t process messages, doesn’t accept connections, and essentially becomes inactive – ensuring that no inconsistent data can be created. Only the majority side of the cluster would continue running.

RabbitMQ now uses Khepri – a newer data storage built on the Raft consensus algorithm. It ensures that any data change is accepted only if approved by a majority of nodes, preventing split-brain by design. Thanks to Raft’s consensus mechanism, Khepri is highly safe and resilient to network failures, keeping the data consistent without needing to stop nodes.

How it works

Default Khepri behaviour

In summary

Lia Anh Nguyen

Discover more from SeventhState.io

Subscribe now to keep reading and get access to the full archive.

Continue reading