CLOUDFLARE has integrated a new Transformer feature into Logpush that lets users filter, redact, and reshape log data as it is exported, rather than after delivery. Transformers are SQL queries tied to a specific Logpush dataset; they run in-process inside the delivery path (Bufstream) and apply to each batch before export.
This lets teams, for example, remove health checks or redact sensitive fields and reformat records to match downstream schemas, reducing data volume and cost since unwanted records never reach destinations. The system supports NDJSON output and can be managed via Transformer Studio in the Cloudflare dashboard or via the API, with new Transformer versions created automatically as changes are saved.
Under the hood, Cloudflare built Transformer support on Apache DataFusion (a Rust-based, Arrow-backed SQL engine) running inside Bufstream. This design keeps latency low and throughput high, avoiding external calls to remote workers that could slow delivery. Data is exchanged across a Go–Rust boundary using Arrow IPC, with a stable, dataset-defined schema guiding query validation.
The pipeline converts transformed Arrow batches back into NDJSON for export, while providing visibility into transformation success, data volumes, and duration. For customers, the feature is available on Free, Pro, Business, and Enterprise plans, with setup through Transformer Studio and the Logpush job to attach a Transformer to the dataset. Cloudflare notes that Transformers are designed to fail closed, retrying failed transforms and auto-disabling persistently failing jobs to protect healthy data flows.