Sep 2, 2026 · 56 min
AI agents threaten the economics of the open web
Cloudflare CEO: We're Ready To Block Millions of Websites From AI — With Matthew Prince
As AI systems absorb and mediate online information, publishers face a choice between blocking access and surrendering the traffic that funds them.
- 1AI-generated traffic already exceeds human browsing and could multiply as agents handle more knowledge work and commerce.
- 2Cloudflare is building controls that let publishers restrict AI crawlers, but compensation systems and business models remain unsettled.
- 3Matthew Prince argues Google should pay for content access as traditional SEO and link-based discovery lose influence.
Don't miss
Matthew Prince predicts that millions of websites could disappear from Google’s radar unless AI companies and publishers reach a new economic arrangement.
The brief
Cloudflare CEO Matthew Prince says automated AI traffic has already overtaken human browsing, challenging the assumption that people remain the web’s primary users.
The growth comes from model training, new AI providers, and agents interacting with sites; Prince expects automated requests to expand dramatically as agents handle work and purchases.
That traffic threatens publishers because AI systems can extract reporting and information without sending readers back, weakening advertising, subscriptions, and the incentive to create.
Cloudflare plans tools and defaults that let ad-supported and subscription sites control AI access, while the industry still lacks a workable micropayment system.
Prince says Google should face the same access rules as other AI companies and predicts traditional SEO and ten-blue-link search will fade.
The episode’s sharpest warning is that millions of sites could vanish from Google’s radar unless AI access produces a sustainable market for content.
Featuring
Google
OpenAI
Anthropic
The New York TimesListen to the full episode and explore every guest, topic, and moment on PodLume.