CLOUDFLARE says it reclaimed more than 100TB of RAM globally by optimising the consistent-hashing system used by its Pingora Backend Router (PBR), an internal load-balancing service. The issue was traced to pingora-ketama, Cloudflare’s open-source consistent-hashing library, which in some cases caused PBR to use as much as 6GB of memory.
The system uses hash rings to route cacheable requests consistently to back-end servers, with multiple rings required for different combinations of features and server capabilities. This created a very large number of stored hash points.
One change reduced each hash point from eight bytes to six by replacing a 32-bit server index with a 16-bit value. Cloudflare says this cut consistent-hashing memory use by 25%, after accounting for Rust’s alignment rules. Further analysis found that reducing the number of hashes generated for each server by 90% would cause no appreciable increase in distribution error. The company also considered collisions among 32-bit hash values, which become more likely as the number of hashes rises.
Cloudflare avoided a sudden global switch because changing the ring could redirect cacheable requests and cause widespread cache churn and increased origin traffic. It temporarily kept both rings, then migrated traffic in stages by data centre while monitoring backend selection, errors, memory use, cache behaviour and origin traffic. Once the migration reached 100%, the old ring was removed, producing the reported 100TB reduction. The updated v2 ring is available as an currently unadvertised feature in the pingora-ketama Rust crate, alongside the existing v1 implementation.