visual-search

2 posts

figma

How We Built AI-Powered Search in Figma | Figma Blog (opens in new tab)

Figma’s AI search emerged from a practical problem: designers often struggled to find existing work, sometimes relying on Slack to locate files from screenshots or vague descriptions. The team initially pursued design autocomplete, but research showed that designers more often reuse and adapt prior work than create from scratch. This led Figma toward visual and semantic search, launched in 2024, using AI to help users find and reuse relevant designs. ## The Problem of Finding Existing Designs - Designers frequently knew what they wanted visually but not where the source file was located. - At Figma, hundreds of Slack messages showed designers asking colleagues to identify files from screenshots or descriptions. - Traditional keyword search was insufficient when users did not know a component’s exact name or file location. ## From Design Autocomplete to AI Search - Figma began with a three-day AI hackathon in June 2023. - One prototype, design autocomplete, suggested likely next components—for example, a “Get started” button in an onboarding flow. - The team believed AI should handle repetitive tasks so designers could focus on higher-level thinking and user needs. - Internal testing and user interviews revealed that designers commonly revisit old explorations, reuse existing work, and build on prior designs rather than starting from nothing. ## Using Search to Improve AI Suggestions - Figma built search infrastructure alongside autocomplete. - Retrieval-Augmented Generation (RAG) could improve AI responses by supplying relevant examples from existing designs. - Finding designs similar to the user’s current work could make autocomplete recommendations more useful and context-aware. - As testing continued, the team recognized that locating and reusing existing work was a more fundamental need than predicting the next component. ## Visual and Semantic Search - **Visual search** allows users to search with: - A screenshot - A selected frame - A quick sketch - **Semantic search** interprets the meaning and context of text-based queries, even when users do not know the precise component name or description. - Search results can help users discover designs and components and then open, preview, or insert them into their projects. Figma’s experience shows that successful AI features often emerge through iteration rather than from an initial prototype. The practical recommendation is to start with real user behavior, test ambitious ideas, and use AI where it removes friction—in this case, helping designers quickly find and reuse relevant existing work.

figma

Meet Figma AI: Empowering Designers with Intelligent Tools | Figma Blog (opens in new tab)

Figma AI is a suite of tools designed to help designers overcome creative blocks, work faster, and explore ideas more easily. Rather than treating AI as hype, Figma presents it as a practical way to solve common workflow problems, including finding existing designs and components. The features are initially available in a limited beta and are free during 2024, though usage limits and future pricing may change. ## Figma’s Practical Approach to AI - Figma AI builds on the company’s earlier AI features for FigJam. - The tools target several stages of design work: - Finding inspiration and existing assets - Exploring different design directions - Automating repetitive tasks - Generating interfaces from text prompts - Figma emphasizes helping designers remain efficient and creative rather than replacing their judgment. ## Visual Search - Visual Search allows users to find similar designs by: - Uploading an image - Selecting part of a canvas - Entering a text query - Results are drawn from team files the user can access. - Relevant frames can be inserted directly into the current working file. - Figma plans to expand search to Community files, with attribution, links to source files, and access to creators’ other work. ## AI-Enhanced Asset Search - Asset Search now uses semantic understanding rather than relying only on exact keyword matches. - A search such as “primary button” can find a component named `btn_large`. - The system considers the meaning and typical use of design elements, making components in large or complex design systems easier to discover. - The goal is to make finding assets feel more natural and reduce time spent searching through files and libraries. ## Beta Availability and Pricing - Figma AI and UI3 are being rolled out through a limited beta. - Users can join through Figma’s help menu by selecting **“Join UI3 + AI waitlist.”** - Features are free during the beta period, which runs through 2024. - Figma may introduce beta usage limits as it evaluates demand and infrastructure costs. - Pricing for general availability will be announced later. Figma AI is positioned as an assistive layer within the existing design workflow: it helps users locate useful starting points and reduce friction while leaving creative decisions with the designer.