cyber.hendryadrian.com
Security News · blog.cloudflare.com

Cloudflare Introduces Ongoing Accountability Ratings for Verified Bots

· Original article ↗

Summary

Cloudflare says it is evaluating whether Verified Bots honor website preferences, including robots.txt rules and requests to slow down after HTTP 429 responses. The system is in internal use, with broader customer access planned.

Key points

  • Cloudflare’s system assesses repeated bot behavior over time and translates observations into an explainable trust band, rather than making a permanent judgment.
  • Initial signals include whether a bot follows site instructions and responds to direct feedback.
  • The system checks whether bots honor robots.txt preferences and adapt their request rate after HTTP 429 responses.
  • Cloudflare says it is using the system internally and plans to make it available to customers, who will be able to set a trust threshold for handling Verified Bot traffic.
  • Cloudflare cites removing Perplexity’s bot from its Verified Bot list in 2025 after it sought to access sites that opted out of scraping and attempted to circumvent controls.

Article Details

Event Type
Cloudflare announced a system to continuously evaluate and report how Verified bots follow website preferences.
Impact
The system assesses repeated bot behavior, including compliance with robots.txt and responses to HTTP 429 signals, to inform an explainable trust band. Cloudflare says it is using the system internally and plans to make it available to customers, who will be able to set trust thresholds for handling Verified bot traffic.

Vendors

CloudflareAs we described in a previous blog post on bot behavior, we strive to verify two questions: “Are you who you say you are?” and “Can we trust you?” To be a Cloudflare Verified Bot, you broadly need to meet three criteria:

Products

Bot ManagementThis system, alongside Bot Preference Sync and Bot Management, forms an accountability loop for automated traffic. Bot Preference Sync helps website owners publish their preferences, and observing how bots interact withBot Preference SyncThis system, alongside Bot Preference Sync and Bot Management, forms an accountability loop for automated traffic. Bot Preference Sync helps website owners publish their preferences, and observing how bots interact withBotBaseBots use Web Bot Auth to sign requests that establish their identity, which BotBase then catalogues in an open format. With identification addressed, our focus shifts to the second cardinal rule of Verified bots: thouRADARwho provide the information bots seek. In the future, we will be sharing these scores publicly on Radar, as well as expanding the ways we measure. This score gives responsible bot operators a way to demonstrate

Related Articles