$ emrebener
home personal-projects consistent hashing demo

Consistent Hashing Demo

published: updated: type: study

Consistent hashing demo project ui

1. Animated consistent-hashing operations

A browser-only SPA that draws the Cassandra/DynamoDB-style consistent hashing ring and animates every operation on it, step by step. Each physical node owns sixteen virtual tokens on a 2322^{32} ring. PUTs and GETs hash the key, walk the ring clockwise, and animate the chosen replicas onto inspector cards. Add a node, remove a node, or slide the replication factor, and the ring rebalances on-screen with every migrated key flying from its old owner to its new one.

The point is teaching, not throughput. The whole stack runs client-side: pure logic in src/core/, a thin Zustand store, React with Framer Motion for the view. No backend, no fixtures, no fake data. The ring you see is the actual ring driving the math, and the bytes the calc panel walks you through are the bytes the lookup ran on.

2. Replica walk with duplicate skipping

Replica selection with virtual nodes has one easy way to be silently wrong. After hashing a key and binary-searching to the first token at or after that position, you walk the ring clockwise and collect successive vnodes. The trap is that neighbouring vnodes very often belong to the same physical node, so without deduplication an RF=3 lookup over sixteen vnodes per node will routinely land all three “replicas” on the same machine. The bug is invisible to most unit tests (writes still happen, reads still return), and the visualization is what makes it obvious.

lookupReplicas(positionOrHash: number): NodeId[] {
  if (this.tokens.length === 0) return [];
  const effectiveRf = Math.min(this.replicationFactor, this.nodeIds.length);
  // ...binary search for first token with position >= positionOrHash...
  const out: NodeId[] = [];
  const seen = new Set<NodeId>();
  for (let step = 0; step < this.tokens.length && out.length < effectiveRf; step++) {
    const t = this.tokens[(idx + step) % this.tokens.length];
    if (!seen.has(t.nodeId)) {
      seen.add(t.nodeId);
      out.push(t.nodeId);
    }
  }
  return out;
}

Two details carry the function. The dedupe via seen, and effectiveRf = min(rf, nodeCount). Without that clamp, a user sliding RF past the current node count puts the loop into an infinite walk. The stored RF stays at whatever the user picked, so adding nodes back later re-widens replication automatically without a second slide.

3. FNV-1a for keys, Murmur3 for vnodes

The first cut used FNV-1a everywhere. FNV-1a is fine for hashing keys, it’s small, deterministic, and the bytes of the calculation are short enough to render in the panel that walks the user through the math. It is not fine for placing virtual node tokens. FNV-1a propagates byte by byte without much mixing, so hashing N1#0, N1#1, …, N1#15 produces sixteen numbers crammed into a tight slice of the output space. On the ring, that meant every node’s sixteen vnodes piled up into two or three arcs instead of spreading around the circle, the exact opposite of what vnodes exist to demonstrate.

The fix splits responsibilities by purpose. Keys keep FNV-1a because that is the hash the calc panel walks the user through, byte by byte, on every PUT and GET. Vnode positions switched to MurmurHash3 (x86 32-bit), which has the avalanche to send nearby inputs to widely separated outputs. Now the ring looks like the textbook diagram, and the teaching artifact is honest about why vnodes work in the first place.