Image Content Moderation in Large-Scale Service Environments (feat. Multimodal LLM)
Image content moderation has evolved from simple rule-based filtering into an AI-powered decision system capable of handling visual context, text, and policy complexity. At large platforms, the challenge is not only accuracy but also latency, cost, scalability, and adaptability to changing policies. LY Corporation addresses these demands through optimized traditional ML models, a hybrid ML–multimodal LLM pipeline, and modular decision-making that combines OCR, visual analysis, and contextual reasoning. ## The Evolution of Content Moderation - Early systems relied on keyword matching, rule-based filters, and predefined patterns. - Machine learning enabled broader pattern recognition and detection of modified or less explicit violations. - Modern systems combine: - Deep learning for text and image classification - Multimodal models for joint image–text understanding - LLMs for context-sensitive judgments - Separate prediction and policy layers for operational flexibility - Despite these advances, image moderation remains difficult because images lack explicit structure and their meaning often depends on context. ## Why Image Moderation Is Difficult - **Visual complexity:** Backgrounds, objects, people, colors, and composition interact in ways that simple object detection cannot fully interpret. - **Context dependency:** Symbols, gestures, and imagery may have different meanings across cultures; embedded text can also determine whether an image is harmful. - **Evasion and variation:** Memes, composites, partially obscured images, and AI-generated edits continually challenge existing detectors. - **Scale requirements:** Platforms may receive millions or tens of millions of images daily, requiring high accuracy alongside low latency, reliability, and cost efficiency. ## LY Corporation’s Moderation API - LY Corporation operates a monitoring platform designed to process large-scale traffic and enforce diverse content policies. - Its image moderation API detects: - Adult content - Violent or graphic scenes - Offensive or disturbing imagery - Identity documents containing personal information - Social media screenshots and other policy-sensitive images - The system is designed to apply service-specific policies consistently while maintaining high throughput. ## Improving Accuracy, Speed, and Cost ### Traditional ML Model Optimization - A PyTorch-based image classification model was selected with latency, cost, and throughput in mind. - The model was converted to ONNX and optimized with FP16 precision. - ONNX Runtime improved execution efficiency, while FP16 reduced memory usage and inference time. - These changes increased throughput by up to **4.3 times**. ### Hybrid ML and Multimodal LLM Architecture - The traditional classifier acts as a fast first-stage filter. - Clear cases are resolved immediately by the image model. - Ambiguous cases are sent to a multimodal LLM for deeper analysis. - More than 90% of production data could be classified by the traditional model alone. - Since multimodal LLM throughput was over 100 times lower than that of the traditional model, routing every image to the LLM would have significantly increased GPU usage and cost. - The hybrid approach preserves high-quality reasoning where necessary while avoiding unnecessary LLM calls. ### vLLM-Based LLM Optimization The team optimized multimodal LLM serving with vLLM, using characteristics such as repeated prompts, predictable token lengths, and prefill-heavy workloads. - **`enable_prefix_caching`:** Reuses KV-cache blocks for repeated system prompts and templates, reducing prefill computation. - **`max_model_len`:** Limits the maximum input-plus-output length to avoid excessive KV-cache allocation. - **`max_num_seqs`:** Controls concurrent requests, balancing throughput against per-request latency and resource contention. - **`max_num_batched_tokens`:** Sets the token budget per scheduling step; larger values can improve throughput for prefill-heavy workloads. - Regularly updating vLLM is recommended because new releases add improvements such as asynchronous scheduling, CUDA graph support, and broader quantization options. ## Moving Beyond Single-Model Policy Prediction - Earlier end-to-end vision models directly predicted final policy categories from images. - This worked for visually obvious violations, such as detecting smoking, but struggled with complex behaviors such as tobacco sales. - Sales-related judgments may require combining: - Product presence - Prices - Sales language - Contact information - Encouragement to purchase - Directly learning every combination of national regulations, service policies, and exceptions created overly complex output classes. - It also made the model harder to extend and maintain, while limiting the use of text embedded in images. ## Hybrid Decision-Making with OCR and Multimodal Reasoning - The redesigned system separates visual and textual information rather than forcing one model to learn every policy combination. - OCR extracts text from images when relevant. - Extracted text helps identify policy-violating behavior or intent. - Visual signals and textual evidence are then combined with a multimodal LLM. - This allows the system to reason about context and intent beyond simple object detection, while making policy logic more modular and adaptable. The practical recommendation is to avoid routing all traffic through expensive general-purpose models. Use fast specialized models for clear cases, reserve multimodal LLMs for ambiguity, optimize serving according to workload characteristics, and separate content understanding from policy decisions so the system can evolve as requirements change.
Read original(opens in new tab)
