Building an open Agentic Internet: readable, discoverable, callable, and payable (opens in new tab)
Agents are becoming a new kind of web visitor: they fetch information, use tools, and make purchases on behalf of people, without rendering pages like traditional browsers. The post argues that the web must evolve into an “Agentic Internet” that is readable, discoverable, callable, and payable. Cloudflare advocates open standards and interoperable infrastructure so agents and website owners can cooperate rather than rely on closed platforms. ## Why the Web Must Adapt - Well-behaved bots are generating billions of requests for unchanged pages, wasting bandwidth, compute, and money. - Agents differ from conventional scrapers: - They act on behalf of paying humans or businesses. - Blocking them can mean blocking potential customers. - Treating them like ordinary scrapers prevents publishers from understanding legitimate demand. - Existing web infrastructure, analytics, and business models were designed for human visitors and pageviews, not programmatic interactions with specific outcomes. - The future could be controlled by a few platforms, or remain open through publicly implementable standards. ## Open Infrastructure for Agents - Cloudflare proposes an open ecosystem based on standards including: - **x402** for payments - **MCP** for agent tools and interactions - **Web Bot Auth** for cryptographic agent identity - **PACT** for privacy-preserving trust signals - Website owners can choose their own identity providers, payment processors, and agent partners. - Cloudflare positions itself as one participant in the infrastructure rather than a mandatory intermediary. ## Identity and Trust - **Web Bot Auth** allows agents to cryptographically identify themselves to websites instead of relying on spoofable `User-Agent` headers. - **Private Access Control Tokens (PACT)** let trusted sites anonymously vouch for agents. - These mechanisms can reduce friction for legitimate agents while giving publishers more control over access. ## Readable: Content Designed for Agents - **Markdown for Agents** provides content in a format that requires fewer tokens and less bandwidth than full human-oriented HTML. - Rendering CSS, images, advertisements, and other interface elements is wasteful when an agent only needs the underlying information. - Cloudflare’s **Kitesurf** browser is designed for agents, running on Workers per request and omitting unnecessary human-browser features. ## Discoverable: Helping Agents Find Resources - Agents need interfaces designed for machine-oriented discovery rather than conventional keyword search. - **AI Search** allows public websites to become searchable by agents. - **Agent Engine Optimization (AEO)** measures how visible a brand is across relevant AI models and agents. - Poor visibility to the agents customers use could effectively make a business invisible to those customers. ## Callable: Exposing Actions Directly - Human-oriented websites force agents to parse HTML, identify controls, and simulate clicks. - **WebMCP** lets websites expose explicit tools through the browser, including structured schemas and executable actions. - This removes the need for DOM guessing and makes interactions more reliable. - Tools run within the page, allowing them to reuse the user’s session and existing state. - **Code Mode** lets agents invoke tools by writing code, which can be faster and more precise than natural-language tool calls. - Direct endpoint usage also gives content owners clearer insight into which content and actions agents actually use. ## Payable: Building a Sustainable Economic Model - Advertising depends on pageviews, rendered ads, and human attention—none of which necessarily occur during agent interactions. - Seat-based pricing is also poorly suited to software agents acting for users. - Micropayments could allow publishers to charge for individual fetches or articles: - Recipe sites might charge fractions of a cent per request. - Newspapers could license articles at read time without requiring a separate subscription. - Agents could use wallets and budgets configured by their users. - Payment standards such as **x402** are presented as a foundation for direct transactions between agents and merchants. ## Practical Conclusion The Agentic Internet will succeed if agents can reliably identify themselves, read content efficiently, discover useful resources, invoke structured actions, and pay directly. Open standards and interoperable systems are preferable to a future where a small number of platforms control discovery, identity, and commerce.