An AI Index for all our customers

Cloudflare has announced the private beta of AI Index for domains on Cloudflare, a new type of web index that helps content creators make their data discoverable by AI and gives AI builders access to better data for fair compensation. According to Cloudflare, enabled AI Index will automatically create an AI-optimised search index for your domain, exposed through ready-to-use APIs and tools including an MCP server, LLMs.txt, a search API, and a bulk data API, with access controlled by AI Crawl Control.

You will be able to monetise access through Pay per crawl and the new x402 integrations, with mechanisms to opt out if you choose. The system supports pub/sub subscriptions so AI platforms can receive real-time, structured updates without indiscriminate crawling, and an aggregated Open Index will bundle participating sites for cross-site search. Participation involves choosing to enable the index on your domain and, if desired, enrolling your website via sign up today.