Slashing agent token costs by 98% with RFC 9457-compliant error responses
Cloudflare argues that HTML error pages are poorly suited to AI agents, wasting tokens while offering little actionable guidance. It now provides RFC 9457-compliant Markdown and JSON responses for Cloudflare-generated errors, including instructions on whether to retry, wait, stop, or escalate. The structured formats reduce payload size and token usage by more than 98% compared with HTML. ## Problems with HTML Error Pages - Cloudflare errors often result from customer policies or edge conditions, including: - DNS or host configuration problems - WAF, geographic, ASN, or bot restrictions - Rate limits - Traditional responses contain large amounts of HTML, CSS, and human-oriented text. - Agents may struggle to identify: - What went wrong - Whether retrying is appropriate - How long to wait - Whether human intervention is required - Custom Error Rules are configuration-dependent and therefore cannot provide a consistent contract across websites. ## RFC 9457 Structured Responses - Cloudflare now supports structured responses for all 1xxx-class errors. - Clients can request: - `Accept: text/markdown` - `Accept: application/json` - `Accept: application/problem+json` - Markdown responses include: - YAML frontmatter with machine-readable fields - “What happened” and “What you should do” guidance - JSON responses expose the same information as a flat object. - Support for Cloudflare-generated 4xx and 5xx errors is planned next. ## Machine-Readable Error Guidance Responses include stable fields such as: - `error_code`, `error_name`, and `error_category` for classification - `retryable` and `retry_after` for automated backoff - `owner_action_required` to indicate when the agent should stop or escalate - `ray_id`, `timestamp`, and `zone` for debugging and support This allows agents to implement durable control flow without scraping HTML or interpreting changing prose. ## Compatibility with RFC 9457 Cloudflare’s JSON format follows the standard Problem Details structure: - `type`: Documentation URL for the error - `status`: Actual HTTP status code - `title`: Short problem description - `detail`: Explanation of the specific occurrence - `instance`: Identifier for the individual error, corresponding to the Ray ID Cloudflare-specific operational fields are included as RFC 9457 extension members. Clients that do not recognize those fields can safely ignore them. ## Example: Rate Limiting A Cloudflare 1015 response identifies the request as rate-limited and includes: - HTTP status `429` - Error code `1015` - Category `rate_limit` - A description stating that the website owner’s configuration caused the limit - A Ray ID and timestamp - Retry-related guidance for implementing backoff The change is automatic across Cloudflare’s network. Browsers continue receiving HTML, while agents receive structured instructions when they explicitly request Markdown or JSON.
Read original(opens in new tab)