Slashing agent token costs by 98% with RFC 9457-compliant error responses

Cloudflare argues that HTML error pages are poorly suited to AI agents, wasting tokens while offering little actionable guidance. It now provides RFC 9457-compliant Markdown and JSON responses for Cloudflare-generated errors, including instructions on whether to retry, wait, stop, or escalate. The structured formats reduce payload size and token usage by more than 98% compared with HTML.

Problems with HTML Error Pages

• Cloudflare errors often result from customer policies or edge conditions, including:
  • DNS or host configuration problems
  • WAF, geographic, ASN, or bot restrictions
  • Rate limits
• Traditional responses contain large amounts of HTML, CSS, and human-oriented text.
• Agents may struggle to identify:
  • What went wrong
  • Whether retrying is appropriate
  • How long to wait
  • Whether human intervention is required
• Custom Error Rules are configuration-dependent and therefore cannot provide a consistent contract across websites.

RFC 9457 Structured Responses

• Cloudflare now supports structured responses for all 1xxx-class errors.
• Clients can request:
  • Accept: text/markdown
  • Accept: application/json
  • Accept: application/problem+json
• Markdown responses include:
  • YAML frontmatter with machine-readable fields
  • “What happened” and “What you should do” guidance
• JSON responses expose the same information as a flat object.
• Support for Cloudflare-generated 4xx and 5xx errors is planned next.

Machine-Readable Error Guidance

Responses include stable fields such as:

• error_code, error_name, and error_category for classification
• retryable and retry_after for automated backoff
• owner_action_required to indicate when the agent should stop or escalate
• ray_id, timestamp, and zone for debugging and support

This allows agents to implement durable control flow without scraping HTML or interpreting changing prose.

Compatibility with RFC 9457

Cloudflare’s JSON format follows the standard Problem Details structure:

• type: Documentation URL for the error
• status: Actual HTTP status code
• title: Short problem description
• detail: Explanation of the specific occurrence
• instance: Identifier for the individual error, corresponding to the Ray ID

Cloudflare-specific operational fields are included as RFC 9457 extension members. Clients that do not recognize those fields can safely ignore them.

Example: Rate Limiting

A Cloudflare 1015 response identifies the request as rate-limited and includes:

• HTTP status 429
• Error code 1015
• Category rate_limit
• A description stating that the website owner’s configuration caused the limit
• A Ray ID and timestamp
• Retry-related guidance for implementing backoff

The change is automatic across Cloudflare’s network. Browsers continue receiving HTML, while agents receive structured instructions when they explicitly request Markdown or JSON.