Adopting AV1 for Real-Time Communication (RTC) at Scale
Meta’s adoption of AV1 for real-time communication has been a multi-year effort focused on codec efficiency, device compatibility, latency, power consumption, and resilience to poor networks. AV1 can reduce bitrate by at least 20% versus H.264/AVC while preserving quality, especially below 100 kbps and for screen content. However, making it practical for mobile video calls required low-complexity encoding, careful decoder selection, and systems designed to avoid freezes caused by bitrate spikes and packet loss. ## Why Meta Adopted AV1 for RTC - AV1 delivers comparable visual quality at substantially lower bitrates than H.264/AVC. - Offline testing showed at least a 20% bitrate reduction on low-end and mid-range devices under Meta’s product settings. - Lower bitrate is especially valuable in real-time calls, where network rates may fluctuate between 10 and 400 kbps. - At a 100 kbps limit, AV1 video remained much clearer than H.264/AVC, which appeared blurry. - AV1 is also well suited to screen sharing: - **Palette mode** efficiently represents frames with a limited number of colors. - **Intra-block copy** predicts repeated patterns within the same frame. - Both tools improve the readability of text and other computer-generated content. ## Challenges of Real-Time Video - RTC requires end-to-end latency below roughly 300 milliseconds; techniques common in video-on-demand, such as multi-pass encoding and extensive buffering, can introduce unacceptable delay. - Sudden bitrate increases can cause freezes. - Network bandwidth changes may require adjustments to resolution or frame rate. - Resolution changes generally require a key frame, producing a temporary bitrate spike. - Packet loss can trigger retransmissions or additional key frames, also increasing the risk of freezes. - Mobile devices must encode and decode video simultaneously, making power efficiency essential. ## Encoder Selection and Complexity - AV1’s advanced coding tools improve compression but can significantly increase encoding complexity. - An open-source AV1 encoder consumed 14% more power than H.264/AVC on a Pixel 8 during testing. - AV1 also used more memory, contributing to application crash regressions. - Meta therefore adopted an internal low-complexity AV1 encoder with power consumption comparable to H.264 baseline. - The encoder supports multiple presets: - Higher-complexity presets prioritize quality. - An ultra-low-complexity preset offers complexity comparable to H.264/AVC. - Meta selects the encoder preset according to device capabilities, allowing AV1 deployment beyond high-end phones. ## Decoder Selection - Although decoding is generally less demanding than encoding, AV1 decoding was still challenging for low-end mobile devices. - Initial tests found real-time decoding failures, video freezes, and audio/video synchronization problems on some devices. - Meta evaluated multiple open-source decoders and selected **dav1d** based on its power efficiency and reliability. Meta’s experience shows that AV1 adoption in RTC requires more than simply enabling a new codec. A practical deployment depends on device-specific complexity controls, efficient decoding, strict latency management, and mechanisms that prevent bandwidth changes or packet loss from interrupting calls.
Read original(opens in new tab)