CLOUDFLARE has introduced a new feature called **Bot Preference Sync** to help website owners manage their bot traffic preferences more efficiently. This feature automatically updates a site's robots.txt file to reflect the configured preferences for AI bot categories—Search, Agent, and Training—ensuring alignment between stated and enforced rules. It aims to simplify the process of disallowing AI training on user content while allowing legitimate bots access for search indexing.
With Bot Preference Sync, users will have greater visibility and control over how their content is utilized by bots, promoting transparency regarding bot operations. The feature will be available to all customers soon, with default settings tailored to different user needs, particularly for publishers.