CLOUDFLARE has developed a prototype called Cache Transcoding to enhance cache efficiency by compressing eligible assets using the Zstandard (zstd) algorithm. This approach allows for significant storage savings and reduced bandwidth costs, with assets on average shrinking to a third of their original size. The encoding incurs a minor CPU increase, but the benefits are substantial, especially for compressible text data such as HTML and Javascript, which represent a large portion of requests.
The implementation operates by compressing uncompressed files upon cache entry, storing them in a compressed state, and decoding them only when served, thus optimizing both storage and data transfer between data centers. Key details include a compression ratio of approximately 2.8x and a processing cost that remains within acceptable limits. Moving forward, Cloudflare plans to refine the process further by exploring higher compression levels and diversifying eligible content.