tcp

4 posts

cloudflare

Cloudflare Workers and Containers now support inbound TCP connections and gRPC (opens in new tab)

Cloudflare is expanding Workers to support low-latency, TCP-based applications such as real-time voice AI and gRPC services. New inbound socket handling lets Workers route connections through Durable Objects and Containers, while Cloudflare also enables full-duplex gRPC servers running in containers. Together, these capabilities allow developers to deploy language-agnostic TCP and gRPC services closer to users across Cloudflare’s global network. ## Inbound TCP with `connect(socket)` - Workers can now accept inbound TCP sockets through a new `connect()` handler. - The socket exposes readable and writable streams, allowing Workers to send, receive, and proxy raw bytes. - Connections can be routed: - Between Workers - From Workers to Durable Objects - From Durable Objects to Cloudflare Containers - Developers can pipe data in both directions to preserve full-duplex communication. - Containers can run arbitrary TCP servers written in any language, such as Python services listening on port `8080`. - Cloudflare Spectrum provides the TCP ingress layer and routes incoming connections to the selected Worker. ## Full-Duplex gRPC in Containers - Developers can deploy gRPC servers written in languages such as Go inside Cloudflare Containers. - Bidirectional streaming allows clients and servers to exchange messages over one persistent connection. - This is particularly useful for: - Real-time voice AI - Low-latency inference - Mobile and distributed applications - Streaming RPC workflows - A sample Go server sends an initial connection message, echoes incoming messages, and sends a closing message when the client disconnects. - Cloudflare’s network of more than 330 locations can bring gRPC workloads closer to users, reducing latency. ## gRPC APIs from Workers - Workers can serve unary and server-streaming gRPC APIs. - Workers can also call external gRPC servers. - Developers write the application using gRPC-Web, while Cloudflare automatically converts incoming and outgoing requests to standard gRPC. - This provides a simpler integration path for applications that need gRPC without managing raw protocol translation themselves. ## Availability - The features are being introduced through a private beta. - Interested developers must sign up to gain access. Cloudflare recommends these capabilities for applications requiring persistent, low-latency, bidirectional communication. The combination of Spectrum, Workers, Durable Objects, and Containers provides a flexible path for running raw TCP protocols and gRPC services close to end users.

cloudflare

When "idle" isn't idle: how a Linux kernel optimization became a QUIC bug (opens in new tab)

CUBIC, the default congestion controller in Linux and quiche, can become permanently stuck at its minimum congestion window after an early congestion collapse. Cloudflare found the bug in a QUIC test where packet loss stopped completely, yet CUBIC continued oscillating between recovery and congestion avoidance instead of increasing its sending rate. The problem was traced to a Linux TCP optimization for idle or app-limited connections, and ultimately fixed with an elegant near-one-line change. ## How CUBIC manages traffic - CUBIC controls the sender’s congestion window (`cwnd`), limiting how many bytes can be in flight. - It increases `cwnd` when acknowledgments arrive without loss and reduces it when loss suggests the network is overloaded. - As quiche’s default congestion controller, CUBIC affects a substantial amount of QUIC traffic. - Recovery from the minimum congestion window is an important but relatively under-tested part of congestion control. ## The failing test - The test downloaded a 10 MB file over HTTP/3 between local quiche client and server. - Network conditions included: - 10 ms RTT - 30% random packet loss during the first two seconds - No packet loss after two seconds - A 10-second timeout - The expected result was for CUBIC to reduce its window during loss, then steadily recover once the network became reliable. - Instead, approximately 60% of repeated 100-run test batches failed to finish in time. ## CUBIC becomes stuck at its minimum - After packet loss stopped at two seconds, bytes in flight remained flat rather than increasing. - CUBIC’s congestion window stayed at its minimum of 2,700 bytes—roughly two full-sized packets. - The controller repeatedly switched between recovery and congestion avoidance: - 999 transitions over about 6.7 seconds - One transition approximately every 14 ms - The oscillation closely matched the connection’s RTT, indicating that each ACK round was triggering the behavior. - Because the test was a download, client ACKs caused the server’s bytes in flight to fall to zero; the server then sent another two-packet burst, repeatedly provoking the faulty state transition. - Reno passed the same test 100% of the time, confirming that the issue was specific to CUBIC rather than the test setup. ## The connection to Linux TCP - The investigation focused on behavior when `bytes_in_flight == 0`, effectively an idle or app-limited condition. - A 2017 Linux kernel change addressed a TCP CUBIC issue after application idle periods. - Before the change, CUBIC’s epoch could remain unchanged for a long time while the application was idle. - When sending resumed, the elapsed time used by CUBIC could be extremely large, producing an excessively aggressive growth slope and dangerous congestion-window inflation. - The kernel optimization was intended to align CUBIC with the app-limited exclusion described in RFC 9438 §4.2-12. - Porting this logic to QUIC exposed an unintended interaction: repeated short periods with no bytes in flight could be interpreted incorrectly, causing CUBIC to cycle between states and remain at its minimum window. The practical lesson is that congestion-control implementations must test not only steady-state throughput and ordinary loss recovery, but also recovery from the minimum window and repeated app-limited or idle periods. In this case, a small adjustment to the idle-state handling broke the cycle and allowed CUBIC to recover normally.

cloudflare

Introducing Programmable Flow Protection: custom DDoS mitigation logic for Magic Transit customers (opens in new tab)

Programmable Flow Protection lets Magic Transit Enterprise customers define custom DDoS mitigation logic for proprietary UDP protocols. Customers write eBPF programs that identify valid and malicious packets, then deploy them across Cloudflare’s global network to pass, drop, or challenge traffic. Currently in beta for an additional cost, the system addresses the limitations of generic blocking and rate limiting. ## Custom Protection for Proprietary UDP - Existing Cloudflare protections understand established protocols such as TCP, DNS, NTP, RDP, and SIP. - Proprietary UDP protocols are harder to protect because Cloudflare cannot interpret their application-level payloads. - Customers can now define what constitutes “good” and “bad” traffic using their own protocol knowledge. - Programs can drop or challenge invalid packets before they reach the customer’s origin. ## Why Generic UDP Mitigation Falls Short - UDP is connectionless and optimized for speed, making it useful for gaming, VoIP, and streaming. - When traffic does not match a known protocol, mitigation typically falls back to: - Blocking a destination IP and port - Applying a generic rate limit - These approaches cannot distinguish legitimate packets from attack traffic, potentially causing lag or connection loss for real users. - Fixed rate limits may also be inappropriate: - A network expecting 1 Gbps may need stricter limits. - A network expecting 25 Gbps may require more permissive thresholds. ## How Programmable Flow Protection Works - Customers upload custom eBPF programs that run on every packet destined for their network. - Programs execute in userspace rather than kernel space, providing isolation and flexibility across different customers and use cases. - Execution occurs after Cloudflare’s existing DDoS protections, preserving baseline security coverage. - Like kernel-based XDP eBPF programs, these programs: - Compile to BPF bytecode - Pass safety and termination verification - Run inside a lightweight, isolated virtual machine - Cloudflare provides specialized helpers for: - Maintaining client state between packet executions - Performing cryptographic validation - Sending challenge packets to clients ## Example: Protecting a Proprietary Game Protocol - A gaming provider running on UDP port 207 could inspect its proprietary application header. - If the header contains a protocol-specific token, the customer’s eBPF program can: - Parse the packet - Extract part of the token, such as its final byte - Pass packets with the expected value - Drop packets that fail validation - This allows legitimate players’ traffic through even when attacks use randomized source addresses, ports, and payloads. Programmable Flow Protection is best suited to Magic Transit customers whose custom UDP protocols cannot be protected effectively by standard protocol-aware controls. By combining customer-specific packet logic with Cloudflare’s global network and stateful challenge mechanisms, it enables more precise mitigation than blanket blocking or generic rate limiting.

cloudflare

A QUICker SASE client: re-building Proxy Mode (opens in new tab)

Cloudflare rebuilt the Cloudflare One Client’s proxy mode to address performance problems caused by translating TCP traffic into IP packets through WireGuard. The new design uses HTTP/3 and QUIC streams for direct Layer 4 proxying, eliminating the smoltcp translation layer. Internal tests showed download and upload speeds doubling while latency decreased significantly. ## Limitations of the Original Proxy Architecture - Proxy mode exposed a local SOCKS5 or HTTP proxy for broad application compatibility. - WireGuard operates at Layer 3, while proxy traffic arrives as Layer 4 TCP streams. - The Client used the Rust-based `smoltcp` stack to convert TCP streams into IP packets. - Cloudflare’s edge then converted those packets back into TCP streams. - This added overhead, limited access to modern TCP features, and caused sluggish performance for media-heavy websites, large transfers, and video calls. ## Direct Layer 4 Proxying with QUIC - Cloudflare deprecated WireGuard for proxy mode and adopted QUIC-based transport. - HTTP/3’s `CONNECT` method encapsulates proxy traffic directly in QUIC streams rather than breaking it into Layer 3 packets. - The new architecture: - Removes the smoltcp translation layer. - Uses QUIC’s built-in congestion and flow control. - Allows the Client and Cloudflare edge to tune transport parameters for performance. - Testing showed approximately doubled upload and download speeds and substantially reduced latency. ## Use Cases That Benefit - **Third-party VPN coexistence:** Users can combine legacy VPNs for on-premises resources with zero trust web security without imposing as much performance loss. - **Application partitioning:** Specific browser traffic can be routed through Cloudflare Gateway while other operating-system traffic remains on the local network. - **High-bandwidth workloads:** Streaming, large dataset transfers, and other data-intensive applications receive faster proxy connections. - **Developer and CLI workflows:** Tools using the SOCKS5 listener benefit from lower-latency API calls and data transfers. ## Availability and Configuration - The improvement requires Cloudflare One Client version `2025.8.779.0` or later on Windows, macOS, or Linux. - In the Cloudflare One dashboard: - Go to **Teams & Resources > Devices > Device profiles > General profiles**. - Set **Service mode** to **Local proxy mode**. - Set **Device tunnel protocol** to **MASQUE**. - Verify the active protocol with: ```bash warp-cli settings | grep protocol ``` Organizations using proxy mode should upgrade the client and switch to MASQUE to gain the new QUIC-based performance improvements.