Public scan
Public Content Value Scan
A public content value scan gives your team a fast, low-friction way to understand whether parts of your site may deserve AI crawler pricing review.
What the scan can inspect
The scan can inspect robots.txt, sitemap files, public URLs, page structure, structured data, tables, downloadable files, update dates, and apparent resource group size.
What the scan cannot know
Without Cloudflare traffic data, the scan cannot know actual crawler demand or willingness to pay. That is why ClassifyToday presents scenario economics instead of revenue forecasts.
What the scan produces
The report groups URLs, scores content value and discovery value, estimates resource count, and suggests whether each group is likely free, chargeable, restricted, or in need of further assessment.
Quick answers
Common questions
Do I need Cloudflare for the first scan?
No. The first scan uses public website signals and does not require access to your Cloudflare account.
Is the public scan enough to set prices?
It is enough to start a conversation and identify candidates. Actual pricing decisions should use Cloudflare crawler demand data.
What is a resource group?
A resource group is a URL pattern such as /research/*, /reports/*, /data/*, or /archive/* that can be assessed and priced consistently.
Next step