Microsoft/github-copilot

4 posts

microsoft

How we built the Microsoft Learn MCP Server (opens in new tab)

Microsoft Learn MCP Server gives AI agents direct, standardized access to current Microsoft documentation through the Model Context Protocol (MCP). Rather than requiring custom APIs, scraping, or embeddings, agents can dynamically discover and use tools for searching documentation, fetching full articles, and finding code samples. Microsoft’s experience shows that successful MCP systems depend not only on retrieval quality, but also on agent-oriented tool design, operational resilience, clear descriptions, and defensive compatibility practices. ## Purpose of Learn MCP Server - Provides trusted, up-to-date Microsoft Learn content to GitHub Copilot and other AI agents. - Uses Streamable HTTP Transport so MCP-compatible clients can connect to a remote server. - Supports three tools: - `microsoft_docs_search` for titles, relevant content sections, and source URLs. - `microsoft_docs_fetch` for retrieving complete article content. - `microsoft_code_sample_search` for locating language-specific code examples. - Grounds agent responses in official Microsoft documentation rather than relying solely on model memory. ## Why MCP Instead of a Traditional API - Conventional APIs require each client to implement: - Authentication and request formatting. - Documentation and integration logic. - Error handling and compatibility maintenance. - MCP allows clients to discover available tools and schemas at runtime. - The same server can support many agents without custom integrations. - Runtime discovery helps clients adapt to evolving tool contracts and reduces hardcoded assumptions. ## Architecture - The remote MCP server sits in front of the Microsoft Learn knowledge service. - It uses the official C# MCP SDK and runs on Azure App Service. - Clients communicate through Streamable HTTP Transport. - The server uses the same content vector store as Ask Learn, providing shared: - Freshness guarantees. - Relevance ranking. - Index coverage. - Ask Learn delivers retrieval directly to users, while Learn MCP Server exposes that capability through a protocol usable by external agents. ## Designing Tools Around Agent Workflows - Internal retrieval APIs expose many low-level options, such as `topK`, index selection, thresholds, filters, and search modes. - Learn MCP Server hides that complexity behind intuitive search-and-fetch operations. - Tool contracts should reflect how agents work rather than mirror backend APIs. - Keeping retrieval details internal prevents implementation choices from leaking into the agent-facing interface. ## Operating a Remote MCP Service - A public MCP server has distributed-systems concerns despite using JSON-RPC: - Cross-region deployment. - Dynamic scaling. - CORS. - Session affinity. - Statelessness. - Data protection. - Operational design and SDK collaboration are as important as implementing the tools themselves. ## Tool Descriptions Shape Agent Behavior - Tool and parameter descriptions act as instructions for language models. - Small wording changes can significantly affect whether agents select a tool and how successfully they use it. - Microsoft created automated evaluation tooling to test descriptions against observed agent behavior and success metrics. - Updated descriptions can be delivered when clients refresh their MCP sessions. ## Combining Search and Fetch - Search and fetch are more effective together than independently. - A typical workflow is: - Search for the most relevant Learn article or section. - Fetch the full Markdown page for additional context. - Use that content to produce a better-grounded answer with stronger citations. - Explicitly describing this follow-up pattern improved downstream results. ## Handling Hardcoded Clients - Some MCP clients treat discovered tools like fixed APIs and hardcode schemas. - Renaming the `question` parameter to `query` caused 2–5% of requests to fail. - Supporting both names during a deprecation period reduced disruption. - Public MCP services must evolve defensively, even though the protocol supports dynamic discovery. - Tools such as MCP Interviewer can help identify schema and behavioral problems before deployment. ## Using Data to Guide Improvements - Usage data showed that most requests involve: - Coding tasks. - Explanations. - Troubleshooting. - The team prioritized retrieval and description changes around these intents. - Documentation-level agent instructions also encourage use of Learn tools when Microsoft technologies are involved. Microsoft Learn MCP Server replaces the manual process of searching, opening, and copying documentation into a development environment. The practical recommendation is to connect compatible agents to the server so they can retrieve official Learn content directly, while MCP tool authors should design simple contracts, measure real agent behavior, and preserve compatibility as their services evolve.

microsoft

The Interaction Changes Everything: Treating AI Agents as Collaborators, Not Automation (opens in new tab)

The article argues that effective AI agents should be treated as engineering collaborators, not automation scripts. Microsoft applied this approach to migrate hundreds of repositories from Entra SDK v1 to v2, reducing work from 4–6 weeks per repository to under two hours with 80–90% accuracy. The key improvement came from giving the agent a role, mission, priorities, and permission to exercise judgment. ## The Entra SDK Migration Challenge - The migration involved hundreds of repositories and sensitive authentication security boundaries. - Traditional migrations required extensive human review and took 4–6 weeks per repository. - The AI agent completed comparable work in under two hours while achieving 80–90% accuracy. - The goal was not merely speed, but reliable handling of custom configurations, edge cases, and security concerns. ## Problems with Automation Thinking - Initial attempts treated the agent like a script executor: - Detailed transformation instructions were provided. - Every anticipated edge case was documented. - The agent was expected to follow a checklist. - This approach repeatedly failed because complex migrations require: - Context-dependent decisions - Handling of undocumented patterns - Security-boundary evaluation - Trade-offs between correctness, speed, and preservation of custom logic - The central lesson is that judgment cannot be fully automated, but it can be supported through collaboration with an intelligent agent. ## Identity Instead of Instructions - The team reframed the agent as a member of the migration team rather than a tool. - The prompt described the agent as a “co-creative engineer” expected to: - Use judgment - Stay curious - Act carefully - Ask for help when uncertain - This change improved accuracy and edge-case handling. - The agent was more likely to surface uncertainty instead of guessing or failing silently. ## The Co-Creative Partnership Framework ### Identity and Mission - Establish the agent’s team, mission, and the importance of the work. - Explicitly state that the agent is not a script executor. - Explain why the task matters so the agent can prioritize appropriately. - Encourage judgment, curiosity, and care. ### Purpose and Intent - Describe the guide as supporting both human and AI team members. - Make priorities explicit, such as security over speed or correctness over completion. - Allow autonomy when repository contexts differ. - Frame uncertainty as a reason to collaborate rather than as failure. ### Prioritized Goals - List primary, secondary, quality, and human-in-the-loop objectives in order. - Explicit priorities help the agent resolve conflicts. - Including quality and collaboration prevents optimizing for speed alone. ### Step-by-Step Guidance with Judgment - Provide concrete actions, conditional logic, edge-case handling, and before-and-after examples. - Specify what must remain unchanged, including custom logic. - Define situations requiring escalation, such as unusual patterns, ambiguity, or possible security violations. - The framework combines procedural guidance with room for context-sensitive decisions. ## Practical Recommendation For complex migrations, security reviews, or architectural work, write prompts that define a collaborative role and decision-making framework—not just a list of commands. Give the agent context, priorities, preservation rules, examples, and clear escalation points so it can act autonomously while knowing when human judgment is required.

microsoft

Enhancing Code Quality at Scale with AI-Powered Code Reviews (opens in new tab)

Microsoft developed an AI-powered pull request reviewer to reduce routine review work, catch defects earlier, and help developers merge code faster. What began as an internal experiment now supports more than 90% of Microsoft’s PRs—over 600,000 per month—and has influenced GitHub’s Copilot for Pull Request Reviews. The central lesson is that AI works best as a human-in-the-loop assistant embedded directly into existing workflows. ## Addressing PR Review Bottlenecks - Human reviewers often spend time on style issues and minor bugs while overlooking architectural or security concerns. - Large, multi-file PRs can lack sufficient context and may wait days or weeks for review. - The AI reviewer automatically joins new PRs and handles repetitive or easily missed checks, allowing humans to focus on higher-level decisions. ## AI-Powered Review Features - **Automated comments:** Flags issues such as missing null checks, error-handling problems, sensitive-data risks, inefficient algorithms, and style inconsistencies. - **Suggested fixes:** Provides corrected snippets or alternative implementations, but authors must explicitly review and apply changes. AI does not commit changes automatically. - **PR summaries:** Generates descriptions of the change and highlights key modifications across the diff. - **Interactive Q&A:** Reviewers can ask questions about parameters, code behavior, or the impact on other modules directly in the PR discussion. - **Workflow integration:** The assistant behaves like a normal reviewer, requiring no separate tools or interfaces and optionally engaging as soon as a PR is opened. ## Effects on Quality and Development Speed - AI-assisted reviews reduced median PR completion times by 10–20% in early studies across 5,000 repositories. - Early feedback reduces waiting time, back-and-forth cycles, and the chance that minor issues delay approval. - The system has identified bugs such as missing null checks and incorrectly ordered API calls before they reached production. - Developers, particularly new hires, can use the explanations as continuous guidance on coding standards and best practices. ## Team-Specific Customization - Teams can configure repository-specific review guidelines. - Custom prompts support specialized checks, including regression detection based on historical crash patterns and validation of deployment or change gates. - This extensibility allows the reviewer to address concerns beyond generic code quality rules. ## Feedback Between Internal and External Products - Microsoft’s internal deployment provided early feedback on review quality, usability, and developer trust. - Internal experiments helped shape features such as inline suggestions and human-controlled change application. - These lessons contributed to GitHub Copilot for Pull Request Reviews, which reached general availability in April 2025. - Microsoft also uses learnings from GitHub’s broader external adoption to improve its internal development practices, creating an ongoing feedback loop between first-party and third-party products. Overall, the post recommends treating AI review as an always-available first pass—not a replacement for human judgment. Its greatest value comes from seamless integration, strong customization, and keeping authors and reviewers accountable for final decisions.

microsoft

How Microsoft Engineers Build AI: Learn about scalable RAG-enabled AI Apps (opens in new tab)

Microsoft’s new *How Microsoft Engineers Build AI* video series explains how its teams develop AI applications at scale. The first episode focuses on retrieval-augmented generation (RAG), using Copilot for Azure’s Ask Learn plugin as a practical example. It shows how RAG can combine proprietary data with large language models to deliver accurate, contextually relevant answers. ## Building AI Applications with RAG - RAG is presented as a practical way to improve AI applications without relying solely on model fine-tuning. - It retrieves relevant information from a knowledge base and provides that context to an LLM when generating responses. - The approach is useful for applications that need current, domain-specific, or proprietary information. ## The Ask Learn Plugin - Microsoft engineers explain how they built the Ask Learn RAG plugin for Copilot for Azure. - The plugin helps Azure developers find answers quickly within their existing workflow. - The project involved product managers and engineering leaders sharing development challenges, design decisions, and best practices. ## Challenges in Developing Reliable RAG - Selecting the right source content is essential for producing useful answers. - Data must be preprocessed effectively before it can be retrieved. - RAG systems require careful performance evaluation to measure accuracy and relevance. - Keeping responses accurate and up to date requires ongoing improvements to content and retrieval methods. ## Broader Microsoft Applications - The episode discusses RAG implementations across: - Copilot in Azure - Microsoft Security Copilot - Dynamics 365 Business Central - These examples demonstrate how RAG can support different products and business scenarios. The episode is intended as a practical introduction for developers building RAG-based applications, covering prototyping, data management, evaluation, and common pitfalls. Developers can explore the series alongside Microsoft Learn resources and Azure AI development tools such as Visual Studio and GitHub Copilot.