Eliminating Cold Starts 2: shard and conquer

Cloudflare’s Eliminating Cold Starts 2 introduces “Worker sharding,” a technique that uses a consistent hash ring to route requests to shard servers and their corresponding shard clients across Cloudflare’s network. The approach aims to reduce the total number of cold starts rather than merely masking them, by coalescing requests onto warm, existing Worker instances and minimising eviction-driven restarts.

The article explains how a lazy, Cap’n Proto RPC-based mechanism lets shard clients obtain a local, lazily loaded capability and avoid sending large request bytes back and forth, effectively shortening the path to invocation. In practice, this shift has yielded tangible gains: about 4% of total enterprise requests are sharded, enterprise eviction rates dropped by a factor of 10, and warm request rates rose to 99.99% from 99.9%, while cold-start rates fell to 0.01%.

The piece notes that TLS handshakes, especially TLS 1.3, still influence timing, but the new sharding strategy reduces reliance on handshakes to beat cold starts. Published on 26 September 2025, the post also discusses load shedding options and the need to scale to handle burst traffic without errors.