microsoftFeb 11, 2026

How we built the Microsoft Learn MCP Server (opens in new tab)

Microsoft Learn MCP Server gives AI agents direct, standardized access to current Microsoft documentation through the Model Context Protocol (MCP). Rather than requiring custom APIs, scraping, or embeddings, agents can dynamically discover and use tools for searching documentation, fetching full articles, and finding code samples. Microsoft’s experience shows that successful MCP systems depend not only on retrieval quality, but also on agent-oriented tool design, operational resilience, clear descriptions, and defensive compatibility practices. ## Purpose of Learn MCP Server - Provides trusted, up-to-date Microsoft Learn content to GitHub Copilot and other AI agents. - Uses Streamable HTTP Transport so MCP-compatible clients can connect to a remote server. - Supports three tools: - `microsoft_docs_search` for titles, relevant content sections, and source URLs. - `microsoft_docs_fetch` for retrieving complete article content. - `microsoft_code_sample_search` for locating language-specific code examples. - Grounds agent responses in official Microsoft documentation rather than relying solely on model memory. ## Why MCP Instead of a Traditional API - Conventional APIs require each client to implement: - Authentication and request formatting. - Documentation and integration logic. - Error handling and compatibility maintenance. - MCP allows clients to discover available tools and schemas at runtime. - The same server can support many agents without custom integrations. - Runtime discovery helps clients adapt to evolving tool contracts and reduces hardcoded assumptions. ## Architecture - The remote MCP server sits in front of the Microsoft Learn knowledge service. - It uses the official C# MCP SDK and runs on Azure App Service. - Clients communicate through Streamable HTTP Transport. - The server uses the same content vector store as Ask Learn, providing shared: - Freshness guarantees. - Relevance ranking. - Index coverage. - Ask Learn delivers retrieval directly to users, while Learn MCP Server exposes that capability through a protocol usable by external agents. ## Designing Tools Around Agent Workflows - Internal retrieval APIs expose many low-level options, such as `topK`, index selection, thresholds, filters, and search modes. - Learn MCP Server hides that complexity behind intuitive search-and-fetch operations. - Tool contracts should reflect how agents work rather than mirror backend APIs. - Keeping retrieval details internal prevents implementation choices from leaking into the agent-facing interface. ## Operating a Remote MCP Service - A public MCP server has distributed-systems concerns despite using JSON-RPC: - Cross-region deployment. - Dynamic scaling. - CORS. - Session affinity. - Statelessness. - Data protection. - Operational design and SDK collaboration are as important as implementing the tools themselves. ## Tool Descriptions Shape Agent Behavior - Tool and parameter descriptions act as instructions for language models. - Small wording changes can significantly affect whether agents select a tool and how successfully they use it. - Microsoft created automated evaluation tooling to test descriptions against observed agent behavior and success metrics. - Updated descriptions can be delivered when clients refresh their MCP sessions. ## Combining Search and Fetch - Search and fetch are more effective together than independently. - A typical workflow is: - Search for the most relevant Learn article or section. - Fetch the full Markdown page for additional context. - Use that content to produce a better-grounded answer with stronger citations. - Explicitly describing this follow-up pattern improved downstream results. ## Handling Hardcoded Clients - Some MCP clients treat discovered tools like fixed APIs and hardcode schemas. - Renaming the `question` parameter to `query` caused 2–5% of requests to fail. - Supporting both names during a deprecation period reduced disruption. - Public MCP services must evolve defensively, even though the protocol supports dynamic discovery. - Tools such as MCP Interviewer can help identify schema and behavioral problems before deployment. ## Using Data to Guide Improvements - Usage data showed that most requests involve: - Coding tasks. - Explanations. - Troubleshooting. - The team prioritized retrieval and description changes around these intents. - Documentation-level agent instructions also encourage use of Learn tools when Microsoft technologies are involved. Microsoft Learn MCP Server replaces the manual process of searching, opening, and copying documentation into a development environment. The practical recommendation is to connect compatible agents to the server so they can retrieve official Learn content directly, while MCP tool authors should design simple contracts, measure real agent behavior, and preserve compatibility as their services evolve.

ai-agentmodel-context-protocolgithub-copilotrag+4

microsoftJan 28, 2026

Diagnosing instability in production-scale agent reinforcement learning (opens in new tab)

Hugging Face has integrated its Post-Training Toolkit into TRL, bringing production-ready diagnostics to reinforcement learning and agent post-training pipelines. The work identifies a late-phase instability specific to tool-using, on-policy agents: variance can grow in post-tool contexts even while loss, reward, entropy, and global KL remain stable. Targeted tail, distributional, and effective-sample-size diagnostics can expose this failure before it becomes divergence. ## Production Monitoring for Long-Running Agents - Modern agent training runs over long horizons, uses external tools, and adapts continuously. - Failures often develop gradually rather than appearing as a single catastrophic event. - Standard aggregate metrics can hide rare but increasingly severe updates. - The proposed monitoring approach: - Computes diagnostics in-stream. - Separates text-only and post-tool interactions. - Aggregates statistics across workers. - Uses lightweight rolling windows and percentile tracking at fixed intervals. ## Tool-Conditioned Variance Amplification - Tool calls expand the state space through external transitions, exposing the policy to contexts it may rarely encounter in the reference distribution. - Training states can be modeled as: `d(s) = (1−α)·d_text(s) + α·d_tool(s)` - As the proportion of tool-conditioned states, `α`, increases, more updates occur where the reference policy assigns low probability to sampled actions. - This causes importance-weighted updates to develop increasingly large tails. - The mechanism is distinct from global entropy collapse or optimizer instability, though those factors may interact with it. ## Minimal Reproduction and Tail Diagnostics - A small on-policy experiment with an instruction-tuned open-weight model reproduced the pattern. - The 95th percentile of absolute per-token log-ratios, `|r|`, was tracked separately for text-only and post-tool contexts. - Findings included: - Text-only tail magnitudes remained stable or declined. - Post-tool tails grew steadily under fixed-policy baselines. - Drift-aware training substantially reduced tail growth. - Constraining tool outputs also suppressed the effect. - Aggregate loss, reward, and entropy remained stable while the tail was expanding. ## Distributional Shift in the Right Tail - Empirical CDFs across early, middle, and late training showed a change in distribution shape rather than a simple threshold crossing. - In tool-conditioned contexts: - The right tail flattened and stretched. - More probability mass moved toward high-magnitude updates. - Drift-aware baselines muted or reversed the shift. - This supports a distributional explanation rather than an artifact of choosing a particular percentile. ## Importance Ratios and Effective Sample Size - For ratio-based on-policy objectives, gradient variance is related to: `Var[ĝ] ∝ E[(π_θ(a|s) / π_ref(a|s))²]` - When `π_ref(a|s)` is small in tool-conditioned states, a small number of updates can dominate the estimator. - Larger batches and improved baselines may reduce noise but do not fix poor support overlap. - Effective sample size (ESS) provides a supporting signal: - ESS declines as importance weights become concentrated. - It is sensitive to window size and batch structure. - Its trends align with post-tool tail growth, but absolute values should not be over-interpreted. ## Delayed Failure and Misdiagnosis - Instability appears first in tool-conditioned contexts and may remain invisible in global metrics for a long time. - By the time aggregate metrics change, substantial variance amplification may already have accumulated. - The problem is often incorrectly attributed solely to optimizer behavior or inadequate global variance reduction. - Such interventions may delay failure without addressing the underlying support mismatch. - The mechanism is less pronounced when tool outputs are tightly constrained, policies are effectively frozen after tool calls, or interaction diversity plateaus early. The practical recommendation is to add slice-aware, tail-focused diagnostics to production TRL pipelines. Monitoring post-tool log-ratio percentiles, distributional changes, and supporting ESS trends can provide an early warning system for instability that global loss, reward, entropy, and KL metrics miss.

ai-agentreinforcement-learningpost-trainingtrl+3

microsoftDec 3, 2025

The Interaction Changes Everything: Treating AI Agents as Collaborators, Not Automation (opens in new tab)

The article argues that effective AI agents should be treated as engineering collaborators, not automation scripts. Microsoft applied this approach to migrate hundreds of repositories from Entra SDK v1 to v2, reducing work from 4–6 weeks per repository to under two hours with 80–90% accuracy. The key improvement came from giving the agent a role, mission, priorities, and permission to exercise judgment. ## The Entra SDK Migration Challenge - The migration involved hundreds of repositories and sensitive authentication security boundaries. - Traditional migrations required extensive human review and took 4–6 weeks per repository. - The AI agent completed comparable work in under two hours while achieving 80–90% accuracy. - The goal was not merely speed, but reliable handling of custom configurations, edge cases, and security concerns. ## Problems with Automation Thinking - Initial attempts treated the agent like a script executor: - Detailed transformation instructions were provided. - Every anticipated edge case was documented. - The agent was expected to follow a checklist. - This approach repeatedly failed because complex migrations require: - Context-dependent decisions - Handling of undocumented patterns - Security-boundary evaluation - Trade-offs between correctness, speed, and preservation of custom logic - The central lesson is that judgment cannot be fully automated, but it can be supported through collaboration with an intelligent agent. ## Identity Instead of Instructions - The team reframed the agent as a member of the migration team rather than a tool. - The prompt described the agent as a “co-creative engineer” expected to: - Use judgment - Stay curious - Act carefully - Ask for help when uncertain - This change improved accuracy and edge-case handling. - The agent was more likely to surface uncertainty instead of guessing or failing silently. ## The Co-Creative Partnership Framework ### Identity and Mission - Establish the agent’s team, mission, and the importance of the work. - Explicitly state that the agent is not a script executor. - Explain why the task matters so the agent can prioritize appropriately. - Encourage judgment, curiosity, and care. ### Purpose and Intent - Describe the guide as supporting both human and AI team members. - Make priorities explicit, such as security over speed or correctness over completion. - Allow autonomy when repository contexts differ. - Frame uncertainty as a reason to collaborate rather than as failure. ### Prioritized Goals - List primary, secondary, quality, and human-in-the-loop objectives in order. - Explicit priorities help the agent resolve conflicts. - Including quality and collaboration prevents optimizing for speed alone. ### Step-by-Step Guidance with Judgment - Provide concrete actions, conditional logic, edge-case handling, and before-and-after examples. - Specify what must remain unchanged, including custom logic. - Define situations requiring escalation, such as unusual patterns, ambiguity, or possible security violations. - The framework combines procedural guidance with room for context-sensitive decisions. ## Practical Recommendation For complex migrations, security reviews, or architectural work, write prompts that define a collaborative role and decision-making framework—not just a list of commands. Give the agent context, priorities, preservation rules, examples, and clear escalation points so it can act autonomously while knowing when human judgment is required.

ai-agentgithub-copilotprompt-engineeringsoftware-development-lifecycle+3