Introducing Markdown for Agents (opens in new tab)

AI agents increasingly need structured, efficient access to web content, making traditional HTML a costly format for machine consumption. Cloudflare’s Markdown for Agents lets enabled websites serve HTML pages as Markdown when clients request text/markdown, reducing token usage and parsing overhead. The post argues that websites should treat AI agents as first-class visitors alongside humans and search engines.

Why Markdown Matters for AI

• Markdown conveys document structure with far less surrounding markup than HTML.
• A Markdown heading such as ## About Us uses roughly 3 tokens, compared with 12–15 tokens for an equivalent HTML heading.
• The post’s HTML uses about 16,180 tokens, while its Markdown version uses approximately 3,150—a reduction of about 80%.
• Converting HTML to Markdown inside an AI pipeline adds computation, cost, and complexity, and may not preserve the publisher’s intended structure.

How Markdown for Agents Works

• Cloudflare-enabled zones can respond to content negotiation requests containing:

  Accept: text/markdown
  

• Cloudflare fetches the original HTML from the origin, converts it to Markdown at the network edge, and returns the converted response.
• Clients can request Markdown with curl, while Workers-based agents can use a fetch() request with Accept: "text/markdown, text/html".
• Responses use Content-Type: text/markdown and include Vary: accept.
• Existing coding agents, including Claude Code and OpenCode, already send compatible Accept headers.

Token Estimates and Agent Workflows

• Converted responses include an x-markdown-tokens header.
• Agents can use this estimate to:
  • Determine whether content fits within a context window
  • Plan chunking strategies
  • Manage processing costs and limits

Content Signals

• Markdown responses include:

  Content-Signal: ai-train=yes, search=yes, ai-input=yes
  

• These signals indicate that the content may be used for AI training, search results, and AI input, including agentic applications.
• Cloudflare says future versions will support custom Content Signal policies.

Availability

• Cloudflare enabled Markdown for Agents on its Developer Documentation and Blog.
• AI crawlers and agents can test the feature by requesting those pages with Accept: text/markdown.

Web publishers can make their content more accessible to AI systems by supporting Markdown negotiation, while agents should request text/markdown whenever available to reduce tokens, parsing work, and processing cost.