techlist.io
KREN
cloudflare

The AI engineering stack we built internally — on the platform we ship (opens in new tab)

database-designci-cdopen-sourcepostgresqlzero-trustai-gatewaycloudflare-accessworkers-ai

Cloudflare built an internal AI engineering stack that now supports AI coding tools for 93% of its R&D organization. The system combines centralized authentication and model routing with internal knowledge, automated review, and sandboxed agent execution. Cloudflare argues that building these capabilities on its own platform improved security, visibility, cost control, and developer velocity, while also serving as a proving ground for products it ships publicly.

Adoption and Impact

  • Over the previous 30 days:
    • 3,683 employees used AI coding tools, representing 60% of the company and 93% of R&D.
    • AI tools generated 47.95 million requests.
    • 295 teams used agentic AI tools or coding assistants.
    • AI Gateway handled 20.18 million requests and routed 241.37 billion tokens.
    • Workers AI processed 51.83 billion tokens.
  • The four-week rolling average of merge requests increased from roughly 5,600 per week to more than 8,700.
  • The week of March 23 reached 10,952 merge requests, nearly twice the Q4 baseline.
  • Cloudflare’s initial focus was MCP servers, but the effort expanded to standards, code review, onboarding, and propagating changes across thousands of repositories.

Architecture at a Glance

Cloudflare organized the stack into three layers:

  • Platform layer: Authentication, model routing, inference, MCP access, and code execution.
  • Knowledge layer: System context and repository guidance through Backstage and AGENTS.md.
  • Enforcement layer: Automated quality controls using AI Code Reviewer and the Engineering Codex.

The stack uses Cloudflare products including:

  • Cloudflare Access for zero-trust authentication.
  • AI Gateway for centralized LLM routing, cost tracking, bring-your-own-key support, and zero-data-retention controls.
  • Workers AI for open-weight model inference.
  • Workers and Access for an MCP Server Portal with single OAuth.
  • Dynamic Workers for sandboxed agent-generated code execution.
  • Agents SDK and Durable Objects for stateful, long-running agent sessions.
  • Sandbox SDK for isolated cloning, building, and testing environments.
  • Workflows for durable, multi-step processes.
  • Backstage for a 16,000-plus-entity knowledge graph.

Centralized Authentication and AI Routing

  • Cloudflare Access authenticates users and enforces zero-trust policies.
  • Every LLM request passes through AI Gateway, providing a single control point for:
    • Provider credentials
    • Usage and cost attribution
    • Model selection
    • Data-retention policies
    • Provider permissions
  • In the past month, frontier providers handled 91.16% of requests, while Workers AI handled 8.84%.
  • Cloudflare routes requests through a proxy Worker rather than connecting clients directly to AI Gateway.
  • The proxy enables later additions such as per-user attribution, model catalogs, permission enforcement, and support for new coding tools without changing client configurations.

Workers AI and Open-Weight Models

  • Workers AI runs open-source models on GPUs distributed across Cloudflare’s global network.
  • Keeping inference on the same network as Workers, Durable Objects, and storage reduces latency, network failures, and cross-cloud configuration.
  • Kimi K2.5, with a 256,000-token context window, tool calling, and structured outputs, processes more than 7 billion tokens per day for a Cloudflare security agent.
  • Cloudflare estimates that running this workload on Workers AI is 77% cheaper than using a mid-tier proprietary model.
  • Workers AI is also used for:
    • Documentation review in CI
    • Generating AGENTS.md files
    • Lightweight inference where latency matters more than maximum model capability
  • Cloudflare expects open-source models to handle an increasing proportion of its internal workloads.

One-Command Client Configuration

  • Engineers begin setup with:

    opencode auth login https://opencode.internal.domain

  • The command uses an OpenCode discovery endpoint at:

    https://opencode.internal.domain/.well-known/opencode

  • The Worker-hosted endpoint provides authentication and configuration information.

  • This mechanism is designed to configure providers, models, MCP servers, agents, commands, and permissions without requiring engineers to edit configuration files manually.

Overall Recommendation

Cloudflare’s experience suggests that organizations adopting AI coding tools should build a centralized control plane early: authenticate users consistently, route model traffic through one managed gateway, maintain shared system knowledge, and enforce quality through automated review and isolated execution. Using the same production platform for internal tooling can also expose product gaps and accelerate improvements to the platform itself.