Siem | Techlist.io

figmaJul 29, 20262 min readCurated summary

How We Secure Figma’s Internal Systems With Agents | Figma Blog

Figma built an AI-powered security agent to reduce the manual work involved in investigating SIEM alerts. What began as a retrieval system for finding similar past incidents evolved into an agent that investigates alerts, queries security data, writes fixes, opens pull requests, and retains useful knowledge. The system reportedly reduced alert time-to-resolution by 71% and changed how security engineers handle on-call work. ## The Challenge of Internal Security Operations - Figma’s infrastructure, SaaS tools, identity systems, and employee devices change constantly. - Panther, Figma’s SIEM, monitors these systems and sends alerts to Slack while creating Asana tickets. - On-call engineers previously spent significant time gathering context: - Comparing alerts with incidents from the previous week - Checking whether an existing pull request addressed the issue - Searching Slack discussions for related investigations - Determining whether an alert was new, recurring, or already understood - Existing agent work focused on securing Figma’s codebase, but the team needed a broader system for the many issues detected by its SIEM. ## The RAG Layer: Giving Alerts Historical Context - Figma first created a retrieval-augmented generation system using: - AWS Bedrock Knowledge Bases - Amazon Kendra - Lambda handlers connected to Panther - Each Panther alert is converted into a standardized searchable document. - The system extracts structured information such as: - IP addresses - Usernames and actors - AWS account IDs from ARNs - Alert type, severity, tags, status, and timestamps - Similar alerts are retrieved semantically using the alert title, typically containing the detection name and actor username. - Searches prioritize: - Recent alerts, since investigation procedures evolve - Alerts containing actual investigation context - Comments from engineers rather than merely closed alerts ## Turning Engineer Comments into Institutional Memory - When an on-call engineer comments in a Slack alert thread, Figma captures that text and attaches it to the original alert. - Asana tickets follow a similar process. - The updated alert document is reindexed with: - The accumulated investigation comments - A `has_investigation_context` flag - Future alerts can retrieve these previous explanations and recommendations. - Engineers do not need to adopt a separate annotation workflow; their normal Slack and Asana comments become reusable knowledge. - Each useful investigation effectively makes subsequent similar alerts faster and less expensive to triage. Figma’s approach demonstrates how security agents can build on existing workflows rather than requiring entirely new ones. Starting with searchable historical context allowed the team to progressively develop a broader agentic system while turning everyday investigative work into persistent operational knowledge.

Read original

Siem AI Agents TypeScript RAG+4

gitlabApr 30, 20263 min readCurated summary

Automate detection testing with GitLab CI/CD and Duo

GitLab’s WATCH framework continuously tests whether security detections still work in real conditions, rather than only verifying that detection rules deploy successfully. It runs simulated attacks in staging, checks alert propagation through logging, SIEM, and SOAR systems, and reports failures automatically. The framework uses GitLab CI/CD to schedule randomized tests, correlate expected alerts, and publish detection-health results. ## The Detection-Validation Gap - Security detections can silently fail because of: - Log schema changes - SIEM updates - Ingestion or pipeline misconfigurations - Other changes between the log source and alerting systems - Reinjecting synthetic logs into a SIEM can test rule logic, but it does not validate real-world behavior or the log-ingestion path. - GitLab’s detections-as-code pipelines confirm that rules can be created and deployed, but not that they fire when the targeted activity occurs. - WATCH fills this gap by validating detections end to end. ## WATCH’s Testing Lifecycle - **Scheduling:** A weekly GitLab CI/CD pipeline discovers active tests and assigns them randomized execution times. - **Heads-up notification:** WATCH creates a dedicated “WATCH Heads Up” SOAR record containing the detections expected to fire. - **Execution:** Scripts perform simulated malicious actions in staging, such as resetting an administrator password or making suspicious API calls. - **Detection:** Activity logs flow through ingestion into the SIEM, where detection rules process them. - **Correlation:** SOAR matches alerts to registered WATCH tests using: - The time window between execution and alerting - Actor identity, such as an IP address or username - The detection rule ID - **Verification:** A follow-up job confirms that all expected detections fired, updates detection metadata, and publishes results to a GitLab Pages dashboard. - Failed tests generate notifications in the team’s Slack channel. - Correlation prevents test alerts from being escalated as genuine incidents while still validating the complete alerting pipeline. ## GitLab CI/CD Implementation WATCH is organized into three pipeline stages: - **`schedule_pipelines`:** - Runs weekly. - Finds active tests and groups them into scheduled pipelines. - Passes the selected tests through the `TESTS_TO_RUN` variable. - **`run_tests`:** - Executes the assigned attack simulations. - Saves execution results in `detection_status.json`. - Records SOAR identifiers needed for later alert correlation. - **`pages`:** - Queries the SOAR to verify alert generation and routing. - Updates `detection_status.json` with test results. - Deploys the latest status data and dashboard assets to GitLab Pages. The example configuration uses Python 3.12, pipeline inputs to enable weekly scheduling or dashboard updates, conditional `rules`, and GitLab Pages artifacts. Scheduled execution is randomized to avoid predictable test patterns and to expose timing-related problems. ## Practical Recommendation Organizations with critical security detections should add continuous behavioral testing alongside detections-as-code validation. A framework like WATCH can provide earlier warning of broken ingestion, rules, or routing while reducing the cost and generic limitations of commercial breach-and-attack simulation tools.

Read original

Siem Database Design Python Security Monitoring+4

kakaoMar 15, 20264 min readCurated summary

Finding Real Threats Among Hundreds of Millions of Security Signals — Transforming the Security Monitoring Paradigm with AI

Kakao argues that monitoring hundreds of millions of daily security events cannot scale through human analysts and increasingly complex rules alone. Its solution is a hybrid AI pipeline that filters noise early, analyzes only high-value events with multiple models, and continuously improves through verified feedback. The goal is not to generate more alerts, but to understand context and identify threats worth investigating. ## The Scale Problem: Finding Threats in a Haystack - Endpoint activity such as process execution, network connections, file changes, and privilege escalation produces hundreds of millions of events. - The volume grows rapidly as services expand, while the proportion of genuine attacks remains very small. - Increasing the number of analysts alongside event volume is economically and operationally unsustainable. - AI is needed to correlate events, interpret behavior statistically and contextually, and dynamically distinguish normal activity from anomalies. ## Limitations of Rule-Based Monitoring - Rules can identify what happened, but not why, who initiated it, or whether it fits the environment. - Legitimate deployment commands can resemble backdoor installation, causing high false-positive rates. - Analysis quality varies by analyst experience, shift, and time of day. - Analysts must manually assemble host information, network sessions, process histories, and related logs into an incident narrative. - Expanding detection categories—behavior sequences, statistical anomalies, multi-source correlations, and rare events—makes manual rule maintenance impractical. - SIEM correlation improves on single-event rules but remains limited to predefined scenarios and struggles with unknown attack patterns. - As rule sets and event volumes grow, both maintenance costs and matching performance become problematic. ## A Funnel-Based Hybrid Architecture - Kakao filters events through multiple stages before using AI: - Rule-based filters remove obvious noise. - Learned normal patterns are automatically excluded. - AI performs detailed analysis only on the small remainder requiring judgment. - Rules handle clear, deterministic patterns quickly, while AI evaluates complex contextual situations. - The framework is designed to accommodate new threat types and detection categories without creating a separate system for each scenario. ## Multi-Model Verification and Operational Resilience - Multiple AI models independently analyze the same event and cross-check one another. - Disagreement is treated as an uncertainty signal that can trigger deeper analyst review. - Model diversity helps reduce bias, false positives, and missed detections. - It also provides resilience against model failures, API outages, and quality changes after model updates. - The design balances cost, processing speed, and accuracy rather than optimizing only for detection precision. ## Teaching AI the Environment’s Context - Generic LLMs initially misclassified legitimate activity because they lacked knowledge of Kakao’s infrastructure. - The system supplies structured context, including: - Host roles - Services running on each host - Accounts used for automation - Normal communication and operational patterns - This context allows the model to act more like an analyst familiar with the organization than a generic security classifier. ## Analyzing Complete Behavior Flows - Individual commands such as `curl`, `chmod`, and script execution can occur in both normal deployments and attacks. - Kakao therefore reconstructs activity at the host level, linking: - Process execution history - Network sessions - File changes - Temporal ordering - The same command can have different meanings depending on when, where, and in what sequence it occurred. - AI evaluates the complete sequence to distinguish routine operations from intrusion behavior. ## Translating Events into AI-Usable Data - Sending raw events directly to an LLM wastes tokens on irrelevant information and reduces accuracy. - Different detection tasks require different signals; statistical anomaly detection and sequence analysis cannot rely on one fixed format. - Kakao introduced: - A standardized event schema - Dynamic feature construction tailored to each detection type - This reduces token usage while improving the relevance and precision of AI analysis. ## WALT: A Self-Learning Detection Loop - Initially, analysts had to manually convert AI conclusions into new detection policies. - Kakao developed WALT, or **Whitelist-Assisted Learning and Tuning**, to automate this feedback process. - Repeatedly verified normal patterns are converted into exception policies. - Those policies filter future matching events before they reach the AI engine. - Thousands of detection policies are reportedly being generated and operated this way, allowing accuracy to improve over time. ## Cost and Performance Constraints - Sending every event to an AI model caused unsustainable costs and processing delays. - The funnel architecture addresses this by reserving expensive AI analysis for events that survive earlier filtering. - The overall system must continuously balance economic cost, response speed, detection accuracy, and reliability. Kakao’s practical recommendation is to treat AI as part of a carefully designed security pipeline—not as a replacement for rules or analysts. Effective large-scale monitoring combines deterministic filtering, contextual multi-model analysis, structured data, and a controlled feedback loop that learns from verified outcomes.

Read original

Siem AI Large Language Models Anomaly Detection+4

datadogAug 15, 20172 min readCurated summary

Improving cloud security visibility with ChatOps | Datadog

Datadog announces that it was named a Leader in the 2026 Gartner® Magic Quadrant™ for Observability Platforms. The provided content does not include the blog post’s body or Gartner’s evaluation details; it mainly contains Datadog’s navigation menu and product links. ## Announcement - Datadog highlights its recognition as a Leader in Gartner’s 2026 Observability Platforms Magic Quadrant. - The linked resource appears to be a Gartner-related announcement rather than a technical deep dive. ## Datadog’s Product Scope The navigation reflects a broad observability and operations platform covering: - Infrastructure monitoring, metrics, containers, Kubernetes, networks, serverless systems, and cloud costs - Application performance monitoring, profiling, dynamic instrumentation, and agent observability - Database, data-stream, jobs, and quality monitoring - Log management, sensitive-data scanning, audit trails, and observability pipelines - Security capabilities including cloud security, SIEM, workload protection, code security, and vulnerability management - Digital experience tools such as real-user monitoring, session replay, synthetic monitoring, and error tracking - Software delivery, CI visibility, testing, feature flags, and code coverage - Incident response, service catalogs, SLOs, workflow automation, and case management - AI agents, GPU monitoring, AI integrations, and investigation tools The supplied excerpt does not provide enough information to summarize Gartner’s criteria, Datadog’s strengths or weaknesses, or the report’s comparative findings.

Read original

Siem Kubernetes Datadog Observability+4