RAG | Techlist.io

lineMay 23, 2025Original article

Implementing a RAG-based (opens in new tab)

To address the operational burden of handling repetitive user inquiries for the AWX automation platform, LY Corporation developed a support bot utilizing Retrieval-Augmented Generation (RAG). By combining internal documentation with historical Slack thread data, the system provides automated, context-aware answers that significantly reduce manual SRE intervention. This approach enhances service reliability by ensuring users receive immediate assistance while allowing engineers to focus on high-priority development tasks. ### Technical Infrastructure and Stack * **Slack Integration**: The bot is built using the **Bolt for Python** framework to handle real-time interactions within the company’s communication channels. * **LLM Orchestration**: **LangChain** is used to manage the RAG pipeline; the developers suggest transitioning to LangGraph for teams requiring more complex multi-agent workflows. * **Embedding Model**: The **paraphrase-multilingual-mpnet-base-v2** (SBERT) model was selected to support multi-language inquiries from LY Corporation’s global workforce. * **Vector Database**: **OpenSearch** serves as the vector store, chosen for its availability as an internal PaaS and its efficiency in handling high-dimensional data. * **Large Language Model**: The system utilizes **OpenAI (ChatGPT) Enterprise**, which ensures business data privacy by preventing the model from training on internal inputs. ### Enhancing LLM Accuracy through RAG and Vector Search * **Overcoming LLM Limits**: Traditional LLMs suffer from "hallucinations," lack of up-to-date info, and opaque sourcing; RAG fixes this by providing the model with specific, trusted context during the prompt phase. * **Embedding and Vectorization**: Textual data from wikis and chats are converted into high-dimensional vectors, where semantically similar phrases (e.g., "Buy" and "Purchase") are stored in close proximity. * **k-NN Retrieval**: When a user asks a question, the bot uses **k-Nearest Neighbors (k-NN)** algorithms to retrieve the top *k* most relevant snippets of information from the vector database. * **Contextual Generation**: Rather than relying on its internal training data, the LLM generates a response based specifically on the retrieved snippets, leading to higher accuracy and domain-specific relevance. ### AWX Support Bot Workflow and Data Sources * **Multi-Source Indexing**: The bot references two main data streams: the official internal AWX guide wiki and historical Slack inquiry threads where previous solutions were discussed. * **Automated First Response**: The workflow begins when a user submits a query via a Slack workflow; the bot immediately processes the request and provides an initial AI-generated answer. * **Human-in-the-Loop Validation**: After receiving an answer, users can click "Issue Resolved" to close the ticket or "Call AWX Admin" if the AI's response was insufficient. * **Efficiency Gains**: This tiered approach filters out "RTFM" (Read The F***ing Manual) style questions, ensuring that human administrators only spend time on unique or complex technical issues. Implementing a RAG-based support bot is a highly effective strategy for SRE teams looking to scale their internal support without increasing headcount. For the best results, organizations should focus on maintaining clean internal documentation and selecting embedding models that reflect the linguistic diversity of their specific workforce.

RAG AI Python Vector Db+5

googleMay 13, 2025Original article

Deeper insights into retrieval augmented generation: The role of sufficient context (opens in new tab)

Google Research has introduced "sufficient context" as a critical new metric for evaluating Retrieval Augmented Generation (RAG) systems, arguing that simple relevance is an inadequate measure of performance. By focusing on whether a retrieved context contains all the necessary information to definitively answer a query, researchers developed an LLM-based autorater that classifies context sufficiency with 93% accuracy. This framework reveals that many RAG failures, specifically hallucinations, occur because models fail to abstain from answering when information is incomplete or contradictory. ## Defining and Measuring Sufficient Context * Sufficient context is defined as containing all information necessary to provide a definitive answer, while insufficient context is relevant but incomplete, inconclusive, or contradictory. * The researchers developed an "autorater" using Gemini 1.5 Pro, utilizing chain-of-thought prompting and 1-shot examples to evaluate query-context pairs. * In benchmarks against human expert "gold standard" labels, the autorater achieved 93% accuracy, outperforming specialized models like FLAMe (fine-tuned PaLM 24B) and NLI-based methods. * Unlike traditional metrics, this approach does not require ground-truth answers to evaluate the quality of the retrieved information. ## RAG Failure Modes and Abstention Challenges * State-of-the-art models (Gemini, GPT, Claude) perform exceptionally well when provided with sufficient context but struggle when context is lacking. * The primary driver of hallucinations in RAG systems is the "abstention" problem, where a model attempts to answer a query based on insufficient context rather than stating "I don't know." * Analyzing model responses through the lens of sufficiency allows developers to distinguish between "knowledge" (the model knows the answer internally) and "grounding" (the model correctly uses the provided context). ## Implementation in Vertex AI * The insights from this research have been integrated into the Vertex AI RAG Engine via a new LLM Re-Ranker feature. * The re-ranker prioritizes retrieved snippets based on their likelihood of providing a sufficient answer, significantly improving retrieval metrics such as normalized Discounted Cumulative Gain (nDCG). * By filtering for sufficiency during the retrieval phase, the system reduces the likelihood that the LLM will be forced to process misleading or incomplete data. To minimize hallucinations and improve the reliability of RAG applications, developers should move beyond keyword-based relevance and implement re-ranking stages that specifically evaluate context sufficiency. Ensuring that an LLM has the "right" to answer based on the provided data—and training it to abstain when that data is missing—is essential for building production-grade generative AI tools.

RAG AI Large Language Models Gemini+5

microsoftMar 3, 20252 min readCurated summary

How Microsoft Engineers Build AI: Learn about scalable RAG-enabled AI Apps

Microsoft’s new *How Microsoft Engineers Build AI* video series explains how its teams develop AI applications at scale. The first episode focuses on retrieval-augmented generation (RAG), using Copilot for Azure’s Ask Learn plugin as a practical example. It shows how RAG can combine proprietary data with large language models to deliver accurate, contextually relevant answers. ## Building AI Applications with RAG - RAG is presented as a practical way to improve AI applications without relying solely on model fine-tuning. - It retrieves relevant information from a knowledge base and provides that context to an LLM when generating responses. - The approach is useful for applications that need current, domain-specific, or proprietary information. ## The Ask Learn Plugin - Microsoft engineers explain how they built the Ask Learn RAG plugin for Copilot for Azure. - The plugin helps Azure developers find answers quickly within their existing workflow. - The project involved product managers and engineering leaders sharing development challenges, design decisions, and best practices. ## Challenges in Developing Reliable RAG - Selecting the right source content is essential for producing useful answers. - Data must be preprocessed effectively before it can be retrieved. - RAG systems require careful performance evaluation to measure accuracy and relevance. - Keeping responses accurate and up to date requires ongoing improvements to content and retrieval methods. ## Broader Microsoft Applications - The episode discusses RAG implementations across: - Copilot in Azure - Microsoft Security Copilot - Dynamics 365 Business Central - These examples demonstrate how RAG can support different products and business scenarios. The episode is intended as a practical introduction for developers building RAG-based applications, covering prototyping, data management, evaluation, and common pitfalls. Developers can explore the series alongside Microsoft Learn resources and Azure AI development tools such as Visual Studio and GitHub Copilot.

Read original

RAG AI Large Language Models Machine Learning+4

figmaOct 24, 20242 min readCurated summary

How We Built AI-Powered Search in Figma | Figma Blog

Figma’s AI search emerged from a practical problem: designers often struggled to find existing work, sometimes relying on Slack to locate files from screenshots or vague descriptions. The team initially pursued design autocomplete, but research showed that designers more often reuse and adapt prior work than create from scratch. This led Figma toward visual and semantic search, launched in 2024, using AI to help users find and reuse relevant designs. ## The Problem of Finding Existing Designs - Designers frequently knew what they wanted visually but not where the source file was located. - At Figma, hundreds of Slack messages showed designers asking colleagues to identify files from screenshots or descriptions. - Traditional keyword search was insufficient when users did not know a component’s exact name or file location. ## From Design Autocomplete to AI Search - Figma began with a three-day AI hackathon in June 2023. - One prototype, design autocomplete, suggested likely next components—for example, a “Get started” button in an onboarding flow. - The team believed AI should handle repetitive tasks so designers could focus on higher-level thinking and user needs. - Internal testing and user interviews revealed that designers commonly revisit old explorations, reuse existing work, and build on prior designs rather than starting from nothing. ## Using Search to Improve AI Suggestions - Figma built search infrastructure alongside autocomplete. - Retrieval-Augmented Generation (RAG) could improve AI responses by supplying relevant examples from existing designs. - Finding designs similar to the user’s current work could make autocomplete recommendations more useful and context-aware. - As testing continued, the team recognized that locating and reusing existing work was a more fundamental need than predicting the next component. ## Visual and Semantic Search - **Visual search** allows users to search with: - A screenshot - A selected frame - A quick sketch - **Semantic search** interprets the meaning and context of text-based queries, even when users do not know the precise component name or description. - Search results can help users discover designs and components and then open, preview, or insert them into their projects. Figma’s experience shows that successful AI features often emerge through iteration rather than from an initial prototype. The practical recommendation is to start with real user behavior, test ambitious ideas, and use AI where it removes friction—in this case, helping designers quickly find and reuse relevant existing work.

Read original

RAG Large Language Models Semantic Search Ai Search+2