CLOUDFLARE argues that the Internet now has a second audience: agents—software acting on behalf of people. This shift means more than half of Internet traffic is non-human, with AI-driven requests increasing traffic dramatically. Cloudflare cites that at the end of 2024 the service handled 63 million HTTP requests per second, rising to 115 million now, with peaks over 150 million, and a growth of over 1,700% in daily agent traffic over the past year.
For businesses, this means there are two audiences to serve and monetise: humans and agents. The policy implication is that simply blocking automated traffic is no longer viable; sites must differentiate between legitimate agent use and misuse while finding ways to monetise this new traffic.
The article outlines several products and approaches to manage and benefit from agent traffic. It introduces AI Crawl Control, Business Insights, and BotBase to identify what agents fetch and why, and Web Bot Auth to cryptographically verify agents so sites can distinguish legitimate agents from impersonators. In response to mixed-use crawlers and evolving business models, Cloudflare replaced a blanket “block AI bots” switch with separate controls for Search, Agent, and Training, plus a paid-use framework.
Pay Per Use and Monetization Gateway aim to let publishers monetize agents’ use of content, with buyers paying per actual use and publishers receiving reports of how content is used. The company emphasises open standards such as x402 and Web Bot Auth, arguing that licensing must be flexible and scalable, allowing domain owners to choose identities and payment methods while ensuring sites are compensated for agent-driven access.