cloudflare-containers

2 posts

cloudflare

Cloudflare Workers and Containers now support inbound TCP connections and gRPC (opens in new tab)

Cloudflare is expanding Workers to support low-latency, TCP-based applications such as real-time voice AI and gRPC services. New inbound socket handling lets Workers route connections through Durable Objects and Containers, while Cloudflare also enables full-duplex gRPC servers running in containers. Together, these capabilities allow developers to deploy language-agnostic TCP and gRPC services closer to users across Cloudflare’s global network. ## Inbound TCP with `connect(socket)` - Workers can now accept inbound TCP sockets through a new `connect()` handler. - The socket exposes readable and writable streams, allowing Workers to send, receive, and proxy raw bytes. - Connections can be routed: - Between Workers - From Workers to Durable Objects - From Durable Objects to Cloudflare Containers - Developers can pipe data in both directions to preserve full-duplex communication. - Containers can run arbitrary TCP servers written in any language, such as Python services listening on port `8080`. - Cloudflare Spectrum provides the TCP ingress layer and routes incoming connections to the selected Worker. ## Full-Duplex gRPC in Containers - Developers can deploy gRPC servers written in languages such as Go inside Cloudflare Containers. - Bidirectional streaming allows clients and servers to exchange messages over one persistent connection. - This is particularly useful for: - Real-time voice AI - Low-latency inference - Mobile and distributed applications - Streaming RPC workflows - A sample Go server sends an initial connection message, echoes incoming messages, and sends a closing message when the client disconnects. - Cloudflare’s network of more than 330 locations can bring gRPC workloads closer to users, reducing latency. ## gRPC APIs from Workers - Workers can serve unary and server-streaming gRPC APIs. - Workers can also call external gRPC servers. - Developers write the application using gRPC-Web, while Cloudflare automatically converts incoming and outgoing requests to standard gRPC. - This provides a simpler integration path for applications that need gRPC without managing raw protocol translation themselves. ## Availability - The features are being introduced through a private beta. - Interested developers must sign up to gain access. Cloudflare recommends these capabilities for applications requiring persistent, low-latency, bidirectional communication. The combination of Spectrum, Workers, Durable Objects, and Containers provides a flexible path for running raw TCP protocols and gRPC services close to end users.

cloudflare

Browser Run: now running on Cloudflare Containers, it’s faster and more scalable (opens in new tab)

Browser Run was rebuilt on Cloudflare Containers to improve speed, reliability, and scale. The migration increased capacity to 60 browser launches per minute and 120 concurrent browsers—four times the previous limit—while cutting Quick Action response times by more than 50%. The main architectural changes were global Container deployment, regional pools of pre-warmed browsers, and replacing eventually consistent KV state with transactional D1 and batched Queue updates. ## Browser Run’s Role - Provides programmatic access to headless browsers on Cloudflare’s global network. - Supports: - End-to-end testing - Suspicious URL investigation - PDF rendering - Screenshots and content extraction - Web interaction for AI agents - The goal is to offer secure, responsible browser automation at massive scale. ## Why the Previous Infrastructure Was Limiting - Browser Run originally shared infrastructure with Browser Isolation (BISO). - BISO’s larger container images caused slower startup and development cycles. - Browser Run lacked optimal global distribution, affecting latency and resilience. - BISO’s long-running sessions conflicted with Browser Run’s short, bursty workloads. - These differences created scaling and availability bottlenecks. ## Gradual Migration to Containers - A Worker initially routed a small number of requests to Container-based browsers while others continued using BISO. - This dual-running setup allowed the team to: - Compare performance - Find implementation bugs - Validate stability - Rollout stages included: - Quick Actions - Free-account Workers browser binding connections - Pay-as-you-go accounts - Contract customers - Customers did not need to change code or redeploy Workers. ## Regional Pools for Lower Latency - Durable Object-enabled Containers can create the Durable Object near the request while starting the Container elsewhere. - This is acceptable for one-off commands but inefficient for WebSocket workflows involving many messages. - The team introduced regional pools of pre-warmed, Durable Object-backed browsers. - Requests are assigned to a nearby Durable Object–Container pair, reducing latency between: - The user and Durable Object - The Durable Object and browser Container - The design requires global browser-state observability so capacity can be allocated and reassigned as demand changes. ## Replacing KV with D1 and Queues - Workers KV was initially used to track browser availability. - Its eventual consistency and cache TTL—around 30 seconds or longer—caused race conditions: - A browser could appear available when another request had already claimed it. - Delayed state updates led to over-allocation and limited responsiveness to traffic spikes. - Browser state was moved to D1, whose SQLite transactions provide atomic assignment. - A browser is exclusively assigned to one user, preventing simultaneous claims through transactional updates. Example acquisition logic updates selected candidates to `picked` and returns their data in one operation: ```sql WITH candidate_pool AS (...) UPDATE containers SET status = 'picked' WHERE sessionId IN ( SELECT sessionId FROM candidate_pool ORDER BY RANDOM() LIMIT ?5 ) RETURNING data; ``` ## Batching State Updates - D1 shards are maintained by location. - Thousands of containers report their state every five seconds, which could overload the database if each update were written individually. - Queue-based batching groups 100 updates into a single write. - This increases theoretical capacity from roughly 5,000 containers per location to as many as 500,000. - The team reports a P95 batch-write latency of 0.1 ms. - Queue consumers use: - Maximum batch size: 100 - Maximum batch timeout: 1 second - Maximum retries: 1 The migration is live, requires no customer changes, and gives Browser Run more room to handle demand from AI agents and other high-volume browser automation workloads.