microsoft·Feb 11, 2026·4 min readCurated summary

How we built the Microsoft Learn MCP Server

Microsoft Learn MCP Server gives AI agents direct, standardized access to current Microsoft documentation through the Model Context Protocol (MCP). Rather than requiring custom APIs, scraping, or embeddings, agents can dynamically discover and use tools for searching documentation, fetching full articles, and finding code samples. Microsoft’s experience shows that successful MCP systems depend not only on retrieval quality, but also on agent-oriented tool design, operational resilience, clear descriptions, and defensive compatibility practices. ## Purpose of Learn MCP Server - Provides trusted, up-to-date Microsoft Learn content to GitHub Copilot and other AI agents. - Uses Streamable HTTP Transport so MCP-compatible clients can connect to a remote server. - Supports three tools: - `microsoft_docs_search` for titles, relevant content sections, and source URLs. - `microsoft_docs_fetch` for retrieving complete article content. - `microsoft_code_sample_search` for locating language-specific code examples. - Grounds agent responses in official Microsoft documentation rather than relying solely on model memory. ## Why MCP Instead of a Traditional API - Conventional APIs require each client to implement: - Authentication and request formatting. - Documentation and integration logic. - Error handling and compatibility maintenance. - MCP allows clients to discover available tools and schemas at runtime. - The same server can support many agents without custom integrations. - Runtime discovery helps clients adapt to evolving tool contracts and reduces hardcoded assumptions. ## Architecture - The remote MCP server sits in front of the Microsoft Learn knowledge service. - It uses the official C# MCP SDK and runs on Azure App Service. - Clients communicate through Streamable HTTP Transport. - The server uses the same content vector store as Ask Learn, providing shared: - Freshness guarantees. - Relevance ranking. - Index coverage. - Ask Learn delivers retrieval directly to users, while Learn MCP Server exposes that capability through a protocol usable by external agents. ## Designing Tools Around Agent Workflows - Internal retrieval APIs expose many low-level options, such as `topK`, index selection, thresholds, filters, and search modes. - Learn MCP Server hides that complexity behind intuitive search-and-fetch operations. - Tool contracts should reflect how agents work rather than mirror backend APIs. - Keeping retrieval details internal prevents implementation choices from leaking into the agent-facing interface. ## Operating a Remote MCP Service - A public MCP server has distributed-systems concerns despite using JSON-RPC: - Cross-region deployment. - Dynamic scaling. - CORS. - Session affinity. - Statelessness. - Data protection. - Operational design and SDK collaboration are as important as implementing the tools themselves. ## Tool Descriptions Shape Agent Behavior - Tool and parameter descriptions act as instructions for language models. - Small wording changes can significantly affect whether agents select a tool and how successfully they use it. - Microsoft created automated evaluation tooling to test descriptions against observed agent behavior and success metrics. - Updated descriptions can be delivered when clients refresh their MCP sessions. ## Combining Search and Fetch - Search and fetch are more effective together than independently. - A typical workflow is: - Search for the most relevant Learn article or section. - Fetch the full Markdown page for additional context. - Use that content to produce a better-grounded answer with stronger citations. - Explicitly describing this follow-up pattern improved downstream results. ## Handling Hardcoded Clients - Some MCP clients treat discovered tools like fixed APIs and hardcode schemas. - Renaming the `question` parameter to `query` caused 2–5% of requests to fail. - Supporting both names during a deprecation period reduced disruption. - Public MCP services must evolve defensively, even though the protocol supports dynamic discovery. - Tools such as MCP Interviewer can help identify schema and behavioral problems before deployment. ## Using Data to Guide Improvements - Usage data showed that most requests involve: - Coding tasks. - Explanations. - Troubleshooting. - The team prioritized retrieval and description changes around these intents. - Documentation-level agent instructions also encourage use of Learn tools when Microsoft technologies are involved. Microsoft Learn MCP Server replaces the manual process of searching, opening, and copying documentation into a development environment. The practical recommendation is to connect compatible agents to the server so they can retrieve official Learn content directly, while MCP tool authors should design simple contracts, measure real agent behavior, and preserve compatibility as their services evolve.

Read original(opens in new tab)

RAGAI AgentsModel Context ProtocolGithub Copilot+4

google·Feb 5, 2026·3 min readCurated summary

How AI tools can redefine universal design to increase accessibility

Google Research proposes Natively Adaptive Interfaces (NAI), a framework that uses multimodal and agentic AI to make interfaces adapt to individual users rather than forcing everyone into a fixed design. Developed through co-design with disability communities, NAI aims to reduce the accessibility gap by embedding assistive capabilities directly into products. Early prototypes suggest that personalized, context-aware interfaces can improve experiences for disabled users while also benefiting the broader population. ## Community-led co-design - Google follows the principle “Nothing About Us, Without Us,” involving people with disabilities as co-designers from the beginning. - Partnerships include RIT/NTID, The Arc of the United States, RNID, and Team Gleason. - These collaborations focus on real-world barriers and recognize the expertise of disability communities. - The approach also aims to create employment and economic opportunities for people who help shape the technology. ## Moving from reactive accessibility to adaptive interfaces - Google identifies an “accessibility gap” between the release of new features and the development of compatible assistive tools. - NAI addresses this by making accessibility native to the interface instead of adding it afterward. - Static navigation is replaced with dynamic, agent-driven modules that can interpret context and adjust the experience. ## Multi-system agents - An Orchestrator maintains shared context and delegates tasks to specialized sub-agents. - A Summarization Agent breaks down complex documents and assigns subtasks to expert agents. - A Settings Agent dynamically adjusts interface elements such as text size. - This structure lets users accomplish tasks without navigating complicated menus or searching for the right control. ## Multimodal interaction - Gemini-based prototypes combine voice, vision, and text rather than limiting accessibility to text-to-speech. - Live video can be converted into interactive audio descriptions. - Users can ask follow-up questions about specific visual details as events unfold. - Conversational interaction provides situational awareness and may reduce cognitive load. ## Proven prototypes - **StreetReaderAI** - Supports blind and low-vision users navigating physical spaces. - Combines an AI Describer that analyzes visual and geographic information with an AI Chat system for questions. - Maintains context so users can ask about previously encountered locations, such as the position of a bus stop. - **Multimodal Agent Video Player (MAVP)** - Makes audio description interactive rather than static. - Users can change the level of detail or ask questions during playback. - Uses an offline “dense index” of visual descriptions and retrieval-augmented generation (RAG) for fast responses. - **Grammar Laboratory** - Developed by RIT/NTID with Google.org support for American Sign Language and English learners. - Provides grammar instruction through ASL videos, English captions, spoken narration, and written transcripts. - Uses adaptive AI to customize lessons according to each student’s language preferences and interactions. ## The curb-cut effect - Accessibility features designed for people with significant constraints can benefit many other users. - Voice interfaces created for blind users may help sighted people who are multitasking. - AI synthesis and learning tools designed for people with learning disabilities can also support users who want information presented more clearly or flexibly. - NAI therefore treats accessibility as a source of better universal design, not as a specialized add-on. NAI’s central recommendation is to build accessibility into interfaces from the start, using multimodal AI, persistent context, and community-led design. The most effective systems will adapt to users while remaining accountable to the people whose needs they are intended to serve.

Read original(opens in new tab)

RAGAI AgentsGenerative AIAccessibility+4

dropbox·Jan 28, 2026·3 min readCurated summary

Engineering VP Josh Clemm on how we use knowledge graphs, MCP, and DSPy in Dash

Dropbox VP Josh Clemm argues that useful workplace AI requires a unified context engine capable of securely understanding and retrieving information across many SaaS applications. Dropbox Dash combines custom connectors, multimodal content processing, knowledge graphs, hybrid search, and personalized access-control-aware ranking to make that possible. Clemm favors indexed retrieval over purely federated approaches because preprocessing enables richer context, faster search, and company-wide access, though it requires substantial engineering and infrastructure. ## Building Dash’s Context Engine - Custom connectors crawl third-party applications while handling: - Rate limits - API differences - Application-specific permissions and ACLs - Incoming content is normalized, often into Markdown, and enriched with: - Titles and metadata - Extracted links - Embeddings - Other key information - Different media types require different processing: - Documents can be text-extracted and indexed. - Images may require CLIP or multimodal models. - PDFs combine text, figures, and other elements. - Audio is transcribed. - Videos may require scene-by-scene multimodal analysis when dialogue is insufficient. - Dash models relationships between content as a knowledge graph: - Meetings can connect to documents, participants, transcripts, and previous notes. - Cross-application relationships provide richer context for search and agents. - Data is stored in secure systems using: - BM25 lexical search - Dense-vector storage - Hybrid retrieval - Multiple ranking stages personalize results and enforce user-specific permissions. ## Indexed Retrieval Versus Federated Retrieval - Federated retrieval queries external systems at runtime. - Its advantages include: - Fast initial implementation - Minimal storage requirements - Relatively fresh data - Easy addition of MCP servers and connectors - Its drawbacks include: - Inconsistent API speed, quality, and ranking - Limited access to company-wide content - Expensive post-processing and reranking - Large token usage when agents reason over returned results - Indexed retrieval preprocesses content during ingestion. - Its advantages include: - Access to shared company connectors - Enriched datasets created offline - Faster queries - More opportunities for recall and ranking experiments - Its costs include: - Significant custom connector development - Freshness challenges - High hosting costs - Difficult storage and architecture choices involving vector search, BM25, hybrid retrieval, or graph RAG. ## Making MCP Practical at Scale - MCP can simplify tool integration, but tool definitions consume substantial context-window space. - Large tool descriptions and retrieval results contribute to context rot and reduce agent effectiveness. - Dash aims to limit context usage to roughly 100,000 tokens. - MCP-based agents can also be slow: simple queries may take up to 45 seconds, while direct index retrieval returns results within seconds. - Dropbox’s approach is to wrap its index in a consolidated “super tool,” reducing the need to expose many separate tools to the agent. ## Broader AI Engineering Practices - The talk also covers Dropbox’s use of: - LLMs as evaluators or judges - Prompt optimization with DSPy - Tool calling and MCP design - These techniques complement the underlying context engine rather than replacing the indexing, enrichment, graph modeling, and permission systems required for reliable workplace AI. A practical takeaway is that organizations building AI over proprietary data should treat retrieval as a full data-platform problem. Start with robust connectors and permissions, enrich content before retrieval, model relationships across sources, and use MCP selectively where indexed retrieval can provide faster and more controlled results.

Read original(opens in new tab)

RAGModel Context ProtocolMultimodal AIDspy+4

netflix·Jan 27, 2026·3 min readCurated summary

The AI Evolution of Graph Search at Netflix

Netflix is evolving Graph Search from structured DSL queries toward natural-language search using large language models (LLMs). The goal is to let users describe what they want in everyday language while preserving the accuracy and trustworthiness required by Netflix’s complex, federated GraphQL data. Rather than replacing existing applications, Netflix plans to augment them with AI-generated filters and future retrieval-augmented generation capabilities. ## Why Natural-Language Search Is Needed - Graph Search currently relies on a Filter DSL, with applications translating UI interactions into structured queries. - Netflix has hundreds of applications with inconsistent query-building experiences, forcing users to learn different interfaces. - Some indexes contain hundreds of filterable fields, making large forms slow and cumbersome even for subject-matter experts. - Users naturally express goals in language such as “show all movies from the 90s about robots from the US,” not through query builders or DSL syntax. - Natural-language input could reduce friction while allowing each application to retain its own domain-specific presentation. ## Converting Text into Graph Search Filters - The core task is translating an ambiguous natural-language request into a valid Graph Search Filter DSL statement. - Graph Search indexes are defined through GraphQL queries containing typed fields, including booleans, strings, enums, and controlled vocabularies. - Generated filters can combine: - Comparisons such as `>` and `==` - Inclusion or exclusion operators such as `IN` - Logical operators such as `AND` - Netflix evaluates generated queries at three levels: - **Syntactic correctness:** The statement follows the DSL grammar and can be parsed. - **Semantic correctness:** The query uses existing fields, respects field types, and selects valid controlled-vocabulary values. - **Pragmatic correctness:** The filter accurately reflects the user’s intended meaning. ## Context Engineering for the LLM - The LLM needs index metadata to generate semantically valid filters. - Netflix derives much of this context from GraphQL schemas, including: - Field paths - Field descriptions from schema comments - Field types - Valid enum or controlled-vocabulary values - Controlled vocabularies define finite, governed sets of values, such as countries, and prevent generated queries from using invalid alternatives. - Supplying all metadata works for simple examples but does not scale: - Some indexes contain hundreds of fields. - Some vocabularies contain thousands of values. - Larger prompts increase latency and can reduce generation accuracy. - Netflix therefore needs ways to provide the LLM with relevant metadata without overwhelming its context, while still grounding generated queries in the actual schema and allowed values. Netflix’s approach combines schema-aware context, LLM-based query generation, and validation to make natural-language Graph Search practical. The key recommendation is to use AI as an augmentation layer over existing Graph Search applications, with strong grounding and correctness checks rather than treating generated queries as inherently reliable.

Read original(opens in new tab)

RAGLarge Language ModelsNatural Language ProcessingGraphQL+4

toss·Jan 21, 2026Original article

Will developers be replaced by AI? (opens in new tab)

The current AI hype cycle is a significant economic bubble where massive infrastructure investments of $560 billion far outweigh the modest $35 billion in generated revenue. However, drawing parallels to the 1995 dot-com era, the author argues that while short-term expectations are overblown, the long-term transformation of the developer role is inevitable. The conclusion is that developers won't be replaced but will instead evolve into "Code Creative Directors" who manage AI through the lens of technical abstraction and delegation. ### The Economic Bubble and Amara’s Law * The industry is experiencing a 16:1 imbalance between AI investment and revenue, with 95% of generative AI implementations reportedly failing to deliver clear efficiency improvements. * Amara’s Law suggests that we are overestimating AI's short-term impact while potentially underestimating its long-term necessity. * Much of the current "AI-driven" job market contraction is actually a result of companies cutting personnel costs to fund expensive GPU infrastructure and AI research. ### Jevons Paradox and the Evolution of Roles * Jevons Paradox indicates that as the "cost" of producing code drops due to AI efficiency, the total demand for software and the complexity of systems will paradoxically increase. * The developer’s identity is shifting from "code producer" to "system architect," focusing on agent orchestration, result verification, and high-level design. * AI functions as a "power tool" similar to game engines, allowing small teams to achieve professional-grade output while amplifying the capabilities of senior engineers. ### Delegation as a Form of Abstraction * Delegating a task to AI is an act of "work abstraction," which involves choosing which low-level details a developer can afford to ignore. * The technical boundary of what is "hard to delegate" is constantly shifting; for example, a complex RAG (Retrieval-Augmented Generation) pipeline built for GPT-4 might become obsolete with the release of a more capable model like GPT-5. * The focus for developers must shift from "what is easy to delegate" to "what *should* be delegated," distinguishing between routine boilerplate and critical human judgment. ### The Risks of Premature Abstraction * Abstraction does not eliminate complexity; it simply moves it into the future. If the underlying assumptions of an AI-generated system change, the abstraction "leaks" or breaks. * Sudden shifts in scaling (traffic surges), regulation (GDPR updates), or security (zero-day vulnerabilities) expose the limitations of AI-delegated work, requiring senior intervention. * Poorly managed AI delegation can lead to "abstraction debt," where the cost of fixing a broken AI-generated system exceeds the cost of having written it manually from the start. To thrive in this environment, developers should embrace AI not as a replacement, but as a layer of abstraction. Success requires mastering the ability to define clear boundaries for AI—delegating routine CRUD operations and boilerplate while retaining human control over architecture, security, and complex business logic.

RAGAILarge Language ModelsGenerative AI+3

line·Jan 14, 2026Original article

Building an Enterprise LLM (opens in new tab)

LY Corporation’s engineering team developed an AI assistant for their private cloud platform, Flava, by prioritizing "context engineering" over traditional prompt engineering. To manage a complex environment of 260 APIs and hundreds of technical documents, they implemented a strategy of progressive disclosure to ensure the LLM receives only the most relevant information for any given query. This approach allows the assistant to move beyond simple RAG-based document summarization to perform active diagnostics and resource management based on real-time API data. ### Performance Limitations of Long Contexts * Research indicates that LLM performance can drop by 13.9% to 85% as context length increases, even if the model technically supports a large token window. * The phenomenon of "context rot" occurs when low-quality or irrelevant information is mixed into the input, causing the model to generate confident but incorrect answers. * Because LLMs are stateless, maintaining conversation history and processing dense JSON responses from multiple APIs quickly exhausts context windows and degrades reasoning quality. ### Progressive Disclosure and Tool Selection * The system avoids loading all 260+ API definitions at once; instead, it analyzes the user's intent to select only the necessary tools, such as loading only Redis-related APIs when a user asks about a cluster. * Specific product usage hints, such as the distinction between private and CDN settings for Object Storage, are injected only when those specific services are invoked. * This phased approach significantly reduces token consumption and prevents the model from being overwhelmed by irrelevant technical specifications. ### Response Guidelines and the "Mock Tool Message" Strategy * The team distinguished between "System Prompts" (global rules) and "Response Guidelines" (situational instructions), such as directing users to a console UI before suggesting CLI commands. * Injecting specific guidelines into the system prompt often caused "instruction conflict," where the LLM might hallucinate information to satisfy a guideline while ignoring core requirements like using search tools. * To resolve these conflicts, the team utilized "ToolMessages" to inject guidelines; by formatting instructions as if they were results from a tool execution, the LLM treats the information as factual context rather than a command that might override the system prompt. To build a robust enterprise LLM service, developers should focus on dynamic context management rather than static prompt optimization. Treating operational guidelines as external data via mock tool messages, rather than system instructions, provides a scalable way to reduce hallucinations and maintain high performance across hundreds of integrated services.

RAGAILarge Language ModelsPrompt Engineering+4

google·Dec 17, 2025Original article

Google Research 2025: Bolder breakthroughs, bigger impact (opens in new tab)

Google Research in 2025 has shifted toward an accelerated "Magic Cycle" that rapidly translates foundational breakthroughs into real-world applications across science, society, and consumer products. By prioritizing model efficiency, factuality, and agentic capabilities, the organization is moving beyond static text generation toward interactive, multi-modal systems that solve complex global challenges. This evolution is underpinned by a commitment to responsible AI development, ensuring that new technologies like quantum computing and generative UI are both safe and culturally inclusive. ## Enhancing Model Efficiency and Factuality * Google introduced new efficiency-focused techniques like block verification (an evolution of speculative decoding) and the LAVA scheduling algorithm, which optimizes resource allocation in large cloud data centers. * The Gemini 3 model achieved state-of-the-art results on factuality benchmarks, including SimpleQA Verified and the newly released FACTS benchmark suite, by emphasizing grounded world knowledge. * Research into Retrieval Augmented Generation (RAG) led to the development of the LLM Re-Ranker in Vertex AI, which helps models determine if they possess sufficient context to provide accurate answers. * The Gemma open model expanded to support over 140 languages, supported by the TUNA taxonomy and the Amplify initiative to improve socio-cultural intelligence and data representation. ## Interactive Experiences through Generative UI * A novel implementation of generative UI allows Gemini 3 to dynamically create visual interfaces, web pages, and tools in response to user prompts rather than providing static text. * This technology is powered by specialized models like "Gemini 3-interactive," which are trained to output structured code and design elements. * These capabilities have been integrated into AI Mode within Google Search, allowing for more immersive and customizable user journeys. ## Advanced Architectures and Agentic AI * Google is exploring hybrid model architectures, such as Jamba-style models that combine State Space Models (SSMs) with traditional attention mechanisms to handle long contexts more efficiently. * The development of agentic AI focuses on models that can reason, plan, and use tools, exemplified by Project Astra, a prototype for a universal AI agent. * Specialized models like Gemini 3-code have been optimized to act as autonomous collaborators for software developers, assisting in complex coding tasks and system design. ## AI for Science and Planetary Health * In biology, research teams utilized AI to map human heart and brain structures and employed RoseTTAFold-Diffusion to design new proteins for therapeutic use. * The NeuralGCM model has revolutionized Earth sciences by combining traditional physics with machine learning for faster, more accurate weather and climate forecasting. * Environmental initiatives include the FireSat satellite constellation for global wildfire detection and the expansion of AI-driven flood forecasting and contrail mitigation. ## Quantum Computing and Responsible AI * Google achieved significant milestones in quantum error correction, developing low-overhead codes that bring the industry closer to a reliable, large-scale quantum computer. * Security and safety remain central, with the expansion of SynthID—a watermarking tool for AI-generated text, audio, and video—to help users identify synthetic content. * The team continues to refine the Secure AI Framework (SAIF) to defend against emerging threats while promoting the safe deployment of generative media models like Veo and Imagen. To maximize the impact of these advancements, organizations should focus on integrating agentic workflows and RAG-based architectures to ensure their AI implementations are both factual and capable of performing multi-step tasks. Developers can leverage the Gemma open models to build culturally aware applications that scale across diverse global markets.

RAGAILarge Language ModelsGenerative AI+5

kakao·Dec 11, 2025Original article

Korean and Images at Once (opens in new tab)

Kakao has developed Kanana-v-embedding, a specialized multimodal embedding model designed to bridge the gap between Korean text and visual data within a unified semantic space. By leveraging a Vision-Language Model (VLM) framework, the model enables seamless search and recommendation across various combinations of text and images, offering a significant performance boost over existing English-centric models like CLIP. This development provides a robust technical foundation for enhancing Kakao’s services, including RAG-based systems and localized content discovery. ### Unified Multimodal Meaning Space * The model maps text and images into a single vector space where semantic similarity is measured via cosine similarity. * Unlike traditional CLIP models that use independent encoders, this architecture treats text and images as a single sequence, allowing for "text + image" combined queries. * It supports four primary interaction modes: Text-to-Text, Text-to-Image, Image-to-Image, and (Text+Image)-to-(Text+Image). ### VLM-Based Architecture and Instruction Tuning * The system utilizes a VLM consisting of an LLM and an image encoder, extracting embeddings from the final hidden state of the [EOS] token. * It employs instruction-based query embedding, where specific prompts (e.g., "Find an image matching this caption") guide the model to generate embeddings tailored to the specific task, such as retrieval or classification. * The model is optimized for the Korean language and cultural context, addressing the limitations of previous models that struggled with non-English data. ### Advanced Training for Scalability and Precision * **Gradient Caching:** To overcome GPU memory limitations, this technique allows the model to train with effectively large batch sizes, which is critical for the InfoNCE loss used in contrastive learning. * **Matryoshka Representation Learning (MRL):** The model supports flexible embedding sizes ranging from 64 to 2,048 dimensions. This allows services to choose between low-latency (smaller dimensions) or high-precision (larger dimensions) without retraining. * **Hard Negative Mining:** The training process incorporates "hard negatives"—items that are similar but incorrect—to sharpen the model’s ability to distinguish between subtle differences in data. ### Performance Benchmarks and Efficiency * Kanana-v-embedding significantly outperforms CLIP and VLM2Vec on the KoEmbed benchmark, particularly in Korean Text-to-Image and Image-to-Text retrieval tasks. * In the M-BEIR (Multimodal Benchmark for Retrieval), the model demonstrated superior performance in multimodal document retrieval and image-to-text tasks compared to established open-source models. * Evaluation of MRL showed that the model retains high accuracy even when dimensions are reduced to 256 or 512, providing a 4x to 8x improvement in storage and search efficiency with minimal loss in quality. For organizations looking to implement multimodal RAG or advanced recommendation systems in Korean-language environments, Kanana-v-embedding offers a highly adaptable solution. Its ability to balance computational cost and retrieval quality through Matryoshka learning makes it particularly suitable for large-scale production environments where latency is a primary concern.

RAGLarge Language ModelsContrastive LearningMultimodal Embedding+4

google·Dec 11, 2025Original article

Spotlight on innovation: Google-sponsored Data Science for Health Ideathon across Africa (opens in new tab)

Google Research, in partnership with several pan-African machine learning communities, recently concluded the Africa-wide Data Science for Health Ideathon to address regional medical challenges. By providing access to specialized open-source health models and technical mentorship, the initiative empowered local researchers to develop tailored solutions for issues ranging from maternal health to oncology. The event demonstrated that localized innovation, supported by high-performance AI foundations, can effectively bridge healthcare gaps in resource-constrained environments. ## Collaborative Framework and Objectives * The Ideathon was launched at the 2025 Deep Learning Indaba in Kigali, Rwanda, in collaboration with SisonkeBiotik, Ro’ya, and DS-I Africa. * The primary goal was to foster capacity building within the African AI community, moving beyond theoretical research toward the execution of practical healthcare tools. * Participants received hands-on training on Google’s specialized health models and were supported with Google Cloud Vertex AI compute credits and mentorship from global experts. * Submissions were evaluated based on their innovation, technical feasibility, and contextual relevance to African health systems. ## Technical Foundations and Google Health Models * Developers focused on a suite of open health AI models, including MedGemma for clinical reasoning, TxGemma for therapeutics, and MedSigLIP for medical vision-language tasks. * The competition utilized a two-phase journey: an initial "Idea Development" stage where teams defined clinical problems and outlined AI approaches, followed by a "Prototype & Pitch" phase. * Technical implementations frequently involved advanced techniques such as Retrieval-Augmented Generation (RAG) to ensure alignment with local medical protocols and WHO guidelines. * Fine-tuning methods, specifically Low-Rank Adaptation (LoRA), were utilized by teams to specialize large-scale models like MedGemma-27B-IT for niche datasets. ## Innovative Solutions for Regional Health * **Dawa Health:** This first-place winner developed an AI-powered cervical cancer screening tool that uses MedSigLIP to identify abnormalities in colposcopy images uploaded via WhatsApp, combined with Gemini RAG for clinical guidance. * **Solver (CerviScreen AI):** This team built a web application for automated cervical-cytology screening by fine-tuning MedGemma-27B-IT on the CRIC dataset to assist cytopathologists with annotated images. * **Mkunga:** A maternal health call center that adapts MedGemma and Gemini to provide advice in Swahili using Speech-to-Text (STT) and Text-to-Speech (TTS) technologies. * **HexAI (DermaDetect):** Recognized for the best proof-of-concept, this offline-first mobile app allows community health workers to triage skin conditions using on-device versions of MedSigLIP, specifically designed for low-connectivity areas. The success of the Ideathon underscores the importance of "local solutions for local priorities." By making sophisticated models like MedGemma and MedSigLIP openly available, the technical barrier to entry is lowered, allowing African developers to build high-impact, culturally and linguistically relevant medical tools. For organizations looking to implement AI in global health, this model of providing foundational tools and cloud resources to local experts remains a highly effective strategy for sustainable innovation.

RAGAIMachine LearningGenerative AI+5

aws·Dec 2, 2025Original article

Amazon S3 Vectors now generally available with increased scale and performance (opens in new tab)

Amazon S3 Vectors has reached general availability, establishing the first cloud object storage service with native support for storing and querying vector data. This serverless solution allows organizations to reduce total ownership costs by up to 90% compared to specialized vector database solutions while providing the performance required for production-grade AI applications. By integrating vector capabilities directly into S3, AWS enables a simplified architecture for retrieval-augmented generation (RAG), semantic search, and multi-agent workflows. ### Massive Scale and Index Consolidation The move to general availability introduces a significant increase in data capacity, allowing users to manage massive datasets without complex infrastructure workarounds. * **Increased Index Limits:** Each index can now store and search across up to 2 billion vectors, representing a 40x increase from the 50 million limit during the preview phase. * **Bucket Capacity:** A single vector bucket can now scale to house up to 20 trillion vectors. * **Simplified Architecture:** The increased scale per index removes the need for developers to shard data across multiple indexes or implement custom query federation logic. ### Performance and Latency Optimizations The service has been tuned to meet the low-latency requirements of interactive applications like conversational AI and real-time inference. * **Query Response Times:** Frequent queries now achieve latencies of approximately 100ms or less, while infrequent queries consistently return results in under one second. * **Enhanced Retrieval:** Users can now retrieve up to 100 search results per query (increased from 30), providing broader context for RAG applications. * **Write Throughput:** The system supports up to 1,000 PUT transactions per second for streaming single-vector updates, ensuring new data is immediately searchable. ### Serverless Efficiency and Ecosystem Integration S3 Vectors functions as a fully serverless offering, eliminating the need to provision or manage underlying instances while paying only for active storage and queries. * **Amazon Bedrock Integration:** It is now generally available as a vector storage engine for Bedrock Knowledge Bases, facilitating the building of RAG applications. * **OpenSearch Support:** Integration with Amazon OpenSearch allows users to utilize S3 Vectors for storage while leveraging OpenSearch for advanced analytics and search features. * **Expanded Footprint:** The service is now available in 14 AWS Regions, up from five during the preview period. With its massive scale and 90% cost reduction, S3 Vectors is a primary candidate for organizations looking to move AI prototypes into production. Developers should consider migrating high-volume vector workloads to S3 Vectors to benefit from the serverless operational model and the native integration with the broader AWS AI stack.

RAGAIAmazon BedrockServerless+4

dropbox·Nov 17, 2025·3 min readCurated summary

How Dash uses context engineering for smarter AI

Dash evolved from a traditional RAG search system into an agentic AI that can interpret information, plan tasks, and act on users’ behalf. Dropbox’s experience shows that better agent performance comes not from adding more tools and data, but from carefully engineering context: limiting choices, filtering for relevance, and delegating complex work to specialized agents. The central conclusion is that precise, timely context improves reasoning speed, accuracy, and efficiency. ## From Search to Agentic AI - Dash initially combined semantic and keyword search to retrieve documents and generate concise answers. - Users began asking it to interpret, summarize, and act on retrieved information. - This required Dash to plan and execute multi-step tasks rather than simply search and summarize. - The resulting challenge was determining which information and tools the model actually needed at each stage. ## The Cost of Too Many Tools - Every tool adds descriptions and parameters to the model’s context window. - More tools expand the model’s decision space, potentially causing slower or less reliable choices. - Tool definitions also consume tokens, increasing cost and reducing room for reasoning. - Longer-running tasks suffered from “context rot,” where accumulated tool-call information degraded accuracy. - Model Context Protocol (MCP) standardizes tool descriptions, but does not eliminate the problem of excessive context. ## Limiting Tool Definitions - Dash found that exposing retrieval tools from many services—such as Confluence, Google Docs, and Jira—created confusion. - Instead of requiring the model to choose among numerous APIs, Dash consolidated retrieval into one purpose-built tool backed by its universal search index. - A single retrieval interface: - Simplifies planning - Reduces tool-selection errors - Keeps the context window focused - Provides consistent access across connected services - The same principle shaped Dash’s MCP server, which exposes retrieval through one lean tool to applications such as Claude, Cursor, and Goose. ## Filtering Context for Relevance - Retrieved information is not automatically useful for the task at hand. - Dash combines data from multiple sources in a unified index and uses a knowledge graph to connect people, activity, and content. - These relationships help rank results according to the query and the user’s context. - By filtering results before presenting them to the model, Dash ensures that each piece of supplied context is relevant. - Precomputing the index and graph allows runtime retrieval to remain fast and focused. ## Using Specialized Agents for Complex Tasks - Some tools require substantial instructions and examples to use correctly. - Dash Search became complex because query construction involves: - Understanding user intent - Mapping intent to index fields - Rewriting queries for semantic matching - Handling typos, synonyms, and implicit context - Adding these instructions directly to the main planning agent consumed context that could otherwise support broader reasoning. - Dash therefore moved search into a specialized agent: - The main agent decides when searching is necessary. - The search agent independently constructs the query using its dedicated prompt. - This division lets the main agent focus on the overall task while the specialist handles search details. Dash’s approach recommends treating context as a limited engineering resource. Use a small number of well-designed tools, pre-filter information for relevance, and delegate technically demanding subtasks to specialized agents rather than overwhelming one general-purpose model.

Read original(opens in new tab)

RAGAI AgentsModel Context ProtocolContext Engineering+3

dropbox·Oct 2, 2025·3 min readCurated summary

A practical blueprint for evaluating conversational AI at scale

Conversational AI systems depend on many probabilistic stages, so even small changes can cause unexpected regressions. Dropbox Dash’s experience shows that evaluation should be treated like production engineering: systematic, repeatable, and required before changes are shipped. The approach combines curated datasets, actionable metrics, LLM-based judging, and human review. ## Evaluation as a Development Discipline - AI pipelines include intent classification, retrieval, ranking, prompt construction, inference, and safety filtering. - Changes to any stage can affect final answer quality in unpredictable ways. - Dropbox initially used ad-hoc testing, but shifted to a standardized process in which every model, prompt, or retrieval change had to pass evaluation before merging. - The evaluation framework covers datasets, metrics, tools, and workflows. - Future-proof evaluation must extend beyond text to images, video, and audio. ## Curating Public and Internal Datasets - Public datasets established baseline retrieval and question-answering performance: - **Natural Questions** tested retrieval from very large documents. - **MS MARCO** emphasized queries requiring multiple document hits. - **MuSiQue** tested multi-hop reasoning. - Internal datasets captured real-world usage from anonymized Dropbox employee queries and content. - Representative query sets reflected actual user behavior, using proxy labels or internal annotators. - Representative content sets focused on shared files, documentation, and connected data sources. - LLM-generated synthetic questions and answers covered tables, images, tutorials, and factual lookups. - These datasets became the foundation for automated pass/fail checks and experiment monitoring. ## Why Traditional Metrics Fall Short - Metrics such as BLEU, ROUGE, METEOR, BERTScore, and embedding similarity are fast and useful for detecting major regressions. - They often fail to measure production requirements, including: - Correct source citations - Factual accuracy - Valid file names and references - Reliable table parsing - Clear formatting - High ROUGE or BERTScore can coexist with hallucinations, missing citations, or buried factual errors. - Production evaluation therefore requires metrics tied directly to task requirements. ## Using LLMs as Evaluation Judges - LLM judges can assess dimensions traditional metrics miss, including: - Whether an answer addresses the query - Whether claims are supported by context - Citation correctness - Clarity, formatting, and tone - Judges receive the query, model answer, source context, and sometimes a hidden reference answer. - They return structured scores—scalar or categorical—alongside explanations. - Rubrics and judge models must themselves be tested, calibrated, versioned, and improved. - Specialized judges may be needed for particular languages or technical domains. ## Calibration and Human Review - Periodic manual labeling of sampled outputs created calibration sets for comparing human and judge-model decisions. - The team tracked agreement and judge drift over time. - Engineers manually reviewed 5–10% of each release’s regression suite. - Disagreements were investigated as either judge-prompt defects or model errors. - Recurring problems led to prompt revisions or more detailed scoring criteria. Dropbox’s evaluation-first approach treats AI changes like code changes: test them against realistic datasets, score them with task-specific rubrics, validate the evaluators, and retain human oversight. This makes conversational AI development more reliable as systems and modalities grow more complex.

Read original(opens in new tab)

RAGLarge Language ModelsNatural Language ProcessingPrompt Engineering+4

google·Sep 18, 2025Original article

Deep researcher with test-time diffusion (opens in new tab)

Google Cloud researchers have introduced Test-Time Diffusion Deep Researcher (TTD-DR), a framework that treats long-form research report writing as an iterative diffusion process. By mimicking human research patterns, the system treats initial drafts as "noisy" versions that are gradually polished through retrieval-augmented denoising and self-evolutionary algorithms. This approach achieves state-of-the-art results in generating comprehensive academic-style reports and solving complex multi-hop reasoning tasks. ### The Backbone DR Architecture The system operates through a three-stage pipeline designed to transition from a broad query to a detailed final document: * **Research Plan Generation:** Upon receiving a query, the agent produces a structured outline of key areas to guide the subsequent information-gathering process. * **Iterative Search Agents:** Two sub-agents work in tandem; one formulates specific search questions based on the plan, while the other performs Retrieval-Augmented Generation (RAG) to synthesize precise answers from available sources. * **Final Report Synthesis:** The agent combines the initial research plan with the accumulated question-answer pairs to produce a coherent, evidence-based final report. ### Component-wise Self-Evolution To ensure high-quality inputs at every stage, the framework employs a self-evolutionary algorithm that optimizes the performance of individual agents: * **Diverse Variant Generation:** The system explores multiple diverse answer variants to cover a larger search space and identify the most valuable information. * **Environmental Feedback:** An "LLM-as-a-judge" assesses these variants using auto-raters for metrics like helpfulness and comprehensiveness, providing specific textual feedback for improvement. * **Revision and Cross-over:** Variants undergo iterative revisions based on feedback before being merged into a single, high-quality output that consolidates the best information from all evolutionary paths. ### Report-level Refinement via Diffusion The core innovation of TTD-DR is modeling the writing process as a denoising diffusion mechanism: * **Messy-to-Polished Transformation:** The framework treats the initial rough draft as a noisy input that requires cleaning through factual verification. * **Denoising with Retrieval:** The agent identifies missing information or weak arguments in the draft and uses search tools as a "denoising step" to inject new facts and strengthen the content. * **Continuous Improvement Loop:** This process repeats in cycles, where each iteration uses newly retrieved information to refine the draft into a more accurate and high-quality final version. TTD-DR demonstrates that shifting AI development from linear generation to iterative, diffusion-based refinement significantly improves the depth and rigor of long-form content. This methodology serves as a powerful blueprint for building autonomous agents capable of handling complex, multi-step knowledge tasks.

RAGAILarge Language ModelsAI Agents+5

line·Aug 12, 2025Original article

The Current State of LY Corporation (opens in new tab)

Tech-Verse 2025 showcased LY Corporation’s strategic shift toward an AI-integrated ecosystem following the merger of LINE and Yahoo Japan. The event focused on the practical hurdles of deploying generative AI, concluding that the transition from experimental models to production-ready services requires sophisticated evaluation frameworks and deep contextual integration into developer workflows. ## AI-Driven Engineering with Ark Developer LY Corporation’s internal "Ark Developer" solution demonstrates how AI can be embedded directly into the software development life cycle. * The system utilizes a Retrieval-Augmented Generation (RAG) based code assistant to handle tasks such as code completion, security reviews, and automated test generation. * Rather than treating codebases as simple text documents, the tool performs graph analysis on directory structures to maintain structural context during code synthesis. * Real-world application includes a seamless integration with GitHub for automated Pull Request (PR) creation, with internal users reporting higher satisfaction compared to off-the-shelf tools like GitHub Copilot. ## Quantifying Quality in Generative AI A significant portion of the technical discussion centered on moving away from subjective "vibes-based" assessments toward rigorous, multi-faceted evaluation of AI outputs. * To measure the quality of generated images, developers utilized traditional metrics like Fréchet Inception Distance (FID) and Inception Score (IS) alongside LAION’s Aesthetic Score. * Advanced evaluation techniques were introduced, including CLIP-IQA, Q-Align, and Visual Question Answering (VQA) based on video-language models to analyze image accuracy. * Technical challenges in image translation and inpainting were highlighted, specifically the difficulty of restoring layout and text structures naturally after optical character recognition (OCR) and translation. ## Global Technical Exchange and Implementation The conference served as a collaborative hub for engineers across Japan, Taiwan, and Korea to discuss the implementation of emerging standards like the Model Context Protocol (MCP). * Sessions emphasized the "how-to" of overcoming deployment hurdles rather than just following technical trends. * Poster sessions (Product Street) and interactive Q&A segments allowed developers to share localized insights on LLM agent performance and agentic workflows. * The recurring theme across diverse teams was that the "evaluation and verification" stage is now the primary driver of quality in generative AI services. For organizations looking to scale AI, the key recommendation is to move beyond simple implementation and invest in "evaluation-driven development." By building internal tools that leverage graph-based context and quantitative metrics like Aesthetic Scores and VQA, teams can ensure that generative outputs meet professional service standards.

RAGAILarge Language ModelsModel Context Protocol+5

line·Jul 16, 2025Original article

LY's Tech Conference, ' (opens in new tab)

LY Corporation’s Tech-Verse 2025 conference highlighted the company's strategic pivot toward becoming an AI-centric organization through the "Catalyst One Platform" initiative. By integrating the disparate infrastructures of LINE and Yahoo! JAPAN into a unified private cloud, the company aims to achieve massive cost efficiencies while accelerating the deployment of AI agents across its entire service ecosystem. This transformation focuses on empowering engineers with AI-driven development tools to foster rapid innovation and deliver a seamless, "WOW" experience for global users. ### Infrastructure Integration and the Catalyst One Platform To address the redundancies following the merger of LINE and Yahoo! JAPAN, LY Corporation is consolidating its technical foundations into a single internal ecosystem known as the Catalyst One Platform. * **Private Cloud Advantage:** The company maintains its own private cloud to achieve a four-fold cost reduction compared to public cloud alternatives, managed by a lean team of 700 people supporting 500,000 servers. * **Unified Architecture:** The integration spans several layers, including Infrastructure (Project "DC-Hub"), Cloud (Project "Flava"), and specialized Data and AI platforms. * **Next-Generation Cloud "Flava":** This platform integrates existing services to enhance VM specifications, VPC networking, and high-performance object storage (Ceph and Dragon). * **Information Security:** A dedicated "SafeOps" framework is being implemented to provide governance and security across all integrated services, ensuring a safer environment for user data. ### AI Strategy and Service Agentization A core pillar of LY’s strategy is the "AI Agentization" of all its services, moving beyond simple features to proactive, personalized assistance. * **Scaling GenAI:** Generative AI has already been integrated into 44 different services within the group. * **Personalized Agents:** The company is developing the capacity to generate millions of specialized agents that can be linked together to support the unique needs of individual users. * **Agent Ecosystem:** The goal is to move from a standard platform model to one where every user interaction is mediated by an intelligent agent. ### AI-Driven Development Transformation Beyond user-facing services, LY is fundamentally changing how its engineers work by deploying internal AI development solutions to all staff starting in July. * **Code and Test Automation:** Proof of Concept (PoC) results showed a 96% accuracy rate for "Code Assist" and a 97% reduction in time for "Auto Test" procedures. * **RAG Integration:** The system utilizes Retrieval-Augmented Generation (RAG) to leverage internal company knowledge and guidelines, ensuring high-quality, context-aware development support. * **Efficiency Gains:** By automating repetitive tasks, the company intends for engineers to shift their focus from maintenance to creative service improvement and innovation. The successful integration of these platforms and the aggressive adoption of AI-driven development tools suggest that LY Corporation is positioning itself to be a leader in the "AI-agent" era. For technical organizations, LY's model serves as a case study in how large-scale mergers can leverage private cloud infrastructure to fund and accelerate a company-wide AI transition.

RAGAIAI AgentsGenerative AI+4