How Figma Stays Ahead of Vulnerabilities With Agents | Figma Blog
Figma uses AI agents to prevent, detect, and fix vulnerabilities during code generation, pull-request review, and historical code audits. Its central strategy is a shared security policy combined with continuous measurement of precision and recall. The company began with PR review because developer feedback and replaying known bugs created the fastest improvement loop. ## A Shared Policy Across the Development Lifecycle - The same policy guides agents during: - Code generation - Pull-request review - Full-repository auditing - The policy records: - Trust boundaries - Accepted risks - Security precedents - Improving PR review first helped Figma refine the policy before applying it to other stages. ## Measuring Precision and Recall - **Precision** measures how many reported findings are genuine vulnerabilities; low precision creates false positives. - **Recall** measures how many real vulnerabilities the system detects; low recall creates false negatives. - Figma measures: - Precision through author thumbs-up or thumbs-down feedback on findings. - Recall by replaying the reviewer against commits containing known bugs. - These separate signals allow the team to improve both dimensions rather than optimizing for only one. ## AI-Assisted Pull-Request Review - Every pull request receives an automated review. - Findings are posted directly to the PR, allowing developers to respond and fix issues in context. - Figma currently runs: - Claude Code with Opus 4.8 at extra-high effort - Codex with GPT-5.6 Sol at high effort - The models identify different classes of bugs, so Figma reports a finding if either model detects one. - Reviews cost approximately $0.50 per pull request at the median. The agents have identified both sophisticated and conventional vulnerabilities, including: - A multi-step desktop-client exploit in which an injected sandbox object exposed the host realm’s `Function` constructor and enabled possible code execution. - An insecure direct object reference where an authenticated user could retrieve another organization’s invoice by supplying its ID. ## Building Trust Before Exposing Findings - Figma launched Anthropic’s Claude Code Security Reviewer in August 2025 in shadow mode. - Initial results were strong at reproducing known vulnerability root causes, but only 4 of 27 findings—about 15%—were valid. - The team prioritized precision first because developers quickly lose trust in tools that generate excessive false positives. - Figma set a practical target of at least 70% precision. - Developer-facing comments were withheld until precision exceeded that level over a two-week period without severe false positives. - Security engineers replayed the reviewer across eight weeks of historical pull requests and manually labeled incorrect findings. - Those examples were used to create and refine the shared security policy. ## Continuous Improvement Through Precedents - A **precedent** documents why a finding is valid or invalid in a particular context. - Human feedback and historical vulnerability replays feed back into the policy. - This process lets Figma improve the agent automatically rather than relying only on model changes or one-time prompt tuning. Figma’s approach treats agentic security as an engineering and measurement problem, not simply a matter of asking an AI to scan code. Organizations adopting similar systems should establish feedback loops, measure precision and recall independently, and build developer trust before making automated findings part of everyday development.
Read original(opens in new tab)
