Finding Real Threats Among Hundreds of Millions of Security Signals — Transforming the Security Monitoring Paradigm with AI
Kakao argues that monitoring hundreds of millions of daily security events cannot scale through human analysts and increasingly complex rules alone. Its solution is a hybrid AI pipeline that filters noise early, analyzes only high-value events with multiple models, and continuously improves through verified feedback. The goal is not to generate more alerts, but to understand context and identify threats worth investigating. ## The Scale Problem: Finding Threats in a Haystack - Endpoint activity such as process execution, network connections, file changes, and privilege escalation produces hundreds of millions of events. - The volume grows rapidly as services expand, while the proportion of genuine attacks remains very small. - Increasing the number of analysts alongside event volume is economically and operationally unsustainable. - AI is needed to correlate events, interpret behavior statistically and contextually, and dynamically distinguish normal activity from anomalies. ## Limitations of Rule-Based Monitoring - Rules can identify what happened, but not why, who initiated it, or whether it fits the environment. - Legitimate deployment commands can resemble backdoor installation, causing high false-positive rates. - Analysis quality varies by analyst experience, shift, and time of day. - Analysts must manually assemble host information, network sessions, process histories, and related logs into an incident narrative. - Expanding detection categories—behavior sequences, statistical anomalies, multi-source correlations, and rare events—makes manual rule maintenance impractical. - SIEM correlation improves on single-event rules but remains limited to predefined scenarios and struggles with unknown attack patterns. - As rule sets and event volumes grow, both maintenance costs and matching performance become problematic. ## A Funnel-Based Hybrid Architecture - Kakao filters events through multiple stages before using AI: - Rule-based filters remove obvious noise. - Learned normal patterns are automatically excluded. - AI performs detailed analysis only on the small remainder requiring judgment. - Rules handle clear, deterministic patterns quickly, while AI evaluates complex contextual situations. - The framework is designed to accommodate new threat types and detection categories without creating a separate system for each scenario. ## Multi-Model Verification and Operational Resilience - Multiple AI models independently analyze the same event and cross-check one another. - Disagreement is treated as an uncertainty signal that can trigger deeper analyst review. - Model diversity helps reduce bias, false positives, and missed detections. - It also provides resilience against model failures, API outages, and quality changes after model updates. - The design balances cost, processing speed, and accuracy rather than optimizing only for detection precision. ## Teaching AI the Environment’s Context - Generic LLMs initially misclassified legitimate activity because they lacked knowledge of Kakao’s infrastructure. - The system supplies structured context, including: - Host roles - Services running on each host - Accounts used for automation - Normal communication and operational patterns - This context allows the model to act more like an analyst familiar with the organization than a generic security classifier. ## Analyzing Complete Behavior Flows - Individual commands such as `curl`, `chmod`, and script execution can occur in both normal deployments and attacks. - Kakao therefore reconstructs activity at the host level, linking: - Process execution history - Network sessions - File changes - Temporal ordering - The same command can have different meanings depending on when, where, and in what sequence it occurred. - AI evaluates the complete sequence to distinguish routine operations from intrusion behavior. ## Translating Events into AI-Usable Data - Sending raw events directly to an LLM wastes tokens on irrelevant information and reduces accuracy. - Different detection tasks require different signals; statistical anomaly detection and sequence analysis cannot rely on one fixed format. - Kakao introduced: - A standardized event schema - Dynamic feature construction tailored to each detection type - This reduces token usage while improving the relevance and precision of AI analysis. ## WALT: A Self-Learning Detection Loop - Initially, analysts had to manually convert AI conclusions into new detection policies. - Kakao developed WALT, or **Whitelist-Assisted Learning and Tuning**, to automate this feedback process. - Repeatedly verified normal patterns are converted into exception policies. - Those policies filter future matching events before they reach the AI engine. - Thousands of detection policies are reportedly being generated and operated this way, allowing accuracy to improve over time. ## Cost and Performance Constraints - Sending every event to an AI model caused unsustainable costs and processing delays. - The funnel architecture addresses this by reserving expensive AI analysis for events that survive earlier filtering. - The overall system must continuously balance economic cost, response speed, detection accuracy, and reliability. Kakao’s practical recommendation is to treat AI as part of a carefully designed security pipeline—not as a replacement for rules or analysts. Effective large-scale monitoring combines deterministic filtering, contextual multi-model analysis, structured data, and a controlled feedback loop that learns from verified outcomes.
Read original(opens in new tab)