Cloudflare’s 6-Byte Struct Trick Frees 100TB of Consistent Hashing RAM

How a Six-Byte Struct Unlocked 100TB

Cloudflare’s Pingora Backend Router service has reclaimed more than 100TB of RAM worldwide by reworking the data structures behind its consistent hashing layer.

The savings came from two surgical moves: shrinking each hash point from eight bytes to six, then eliminating 90 percent of the hashes assigned to each server after a closed-form error analysis showed the extra points added almost no load-balancing value.

That reduction stacks on top of another 100TB memory win from Cloudflare’s DNS team, doubling a massive infrastructure efficiency gain at a company running thousands of servers and petabytes of RAM.

Why Consistent Hashing Became the Memory Hotspot

Consistent hashing lets Cloudflare route cacheable requests to servers by URL without relocating every file when a server joins or leaves.

Each server and each task maps onto a hash-based number line, and a task is assigned to the first server to its left, wrapping around at the boundary.

With a single hash per server, the distribution can be extremely uneven. In a 100-server example, the coefficient of variation is roughly 99 percent, meaning some nodes can handle nearly twice the ideal load while others sit almost idle.

Adding multiple hash points per server smooths the assignment. NGINX hardcodes 160 points per server, and Pingora used the same default, dropping the coefficient of variation from about 99 percent to roughly 8 percent in that same 100-server model.

Cloudflare also uses ketama-style weights so storage-heavy servers receive proportionally more traffic, which scales hash counts by disk capacity. Feature combinations multiply into dozens of separate rings, and those rings all reside in memory.

In some cases, the resulting structures consumed 6GB per process, which triggered the original ticket about excessive memory in pingora-ketama.

The Math That Justified a 90% Reduction

The first storage win targeted the Point struct. It originally paired a 32-bit hash with a 32-bit server index, consuming eight bytes in memory.

Since Pingora Backend Router will never coordinate more than roughly 65,000 servers at once, the index could safely drop to 16 bits.

Rust’s alignment rules prevented a simple six-byte struct, so the team stored the hash and index in a raw byte array and added getter methods to reconstruct the fields.

That change alone cut consistent hashing memory by 25 percent.

The deeper win came from questioning the hash count itself. The engineering team derived the exact coefficient of variation for k hashes per server instead of relying on approximations.

With a weight factor of 625, each server ended up with 100,000 hash points. The math showed that the final 90,000 points were buying only a 0.7 percent reduction in error margin.

Collisions in 32-bit hash space made the argument stronger. Simulated results for 2,048-server data centers showed error increasing as hash counts climbed between 10,000 and 100,000 per server.

That behavior reversed the usual assumption: fewer hashes could preserve distribution quality while freeing massive memory. The team cut hash counts by 90 percent.

A Staged Migration That Avoided Cache Collapse

Changing hash rings changes where cacheable requests land, so a global cutover would have invalidated nearly all cached content and triggered a surge in origin traffic.

Pingora Backend Router carried both the old ketama ring and the new smaller ring in memory during migration. Each request used the migration framework to choose a ring, keeping decisions stable per request hash and preserving a clean rollback path.

The rollout proceeded in layers, starting with small validation locations and expanding through progressively larger data center groups.

Traffic percentage and geographic scope were controlled independently, which kept cache churn from spreading everywhere at once.

Observability focused on backend-selection traces, ring-version counters, connection errors, process memory, startup time, cache behavior, and origin traffic. After reaching 100 percent migration, the team removed the legacy ring path and confirmed the 100TB memory drop.

The changes are now available in the pingora-ketama crate as an unadvertised cargo feature. The v2 ring includes compacted storage, faster sorting, and scalable base hash counts, while v1 remains identical so both can run side by side on a request-by-request basis.

What High-Scale Teams Should Recompute Next

The 100TB result shows how a small alignment-aware data change can compound across thousands of nodes when the underlying structure is repeated millions of times.

For performance teams, the larger lesson is the negative result: 90 percent of the hash points were not improving load distribution in any meaningful way.

That insight emerged only because the team replaced rules of thumb with exact probability formulas and collision-aware simulations.

The migration pattern adds another lesson. Running dual rings and controlling traffic geography independently turned a risky cache invalidation event into a reversible operational change.

The result is a 100TB memory reduction that did not sacrifice cache stability or traffic distribution.

For infrastructure teams turning algorithmic efficiency into production memory savings, Andres SEO Expert’s WordPress Speed Engineering service brings the same measurement-first approach — contact the team.