azure-devops

2 posts

microsoft

Common annotated security keys (opens in new tab)

GitHub’s improved security-token format demonstrated that fixed signatures and checksums can sharply reduce both false positives and missed secret detections. Microsoft applies these ideas across its services and proposes the open-source Common Annotated Security Standard (CASK), a shared format for identifiable secrets. CASK is intended to make scanning faster, more accurate, and easier to apply across an entire ecosystem without disrupting developers. ## Identifiable Secrets and Better Detection - “Identifiable” keys combine: - A fixed signature that reliably identifies the format. - A checksum that validates whether a detected string is a real key. - These features reduce scanner noise and missed findings. - Microsoft can hard-block identifiable keys from being stored in source code, work items, and similar locations with high confidence. - Scanners can detect the common format first and classify the specific service provider later, if needed. ## The Common Annotated Security Standard - CASK defines platform-agnostic requirements for minted security keys. - It reserves space for individual platforms and providers to encode service-specific metadata. - Microsoft has defined Azure-specific metadata within this reserved area. - A shared standard lowers the effort required for security tools to protect multiple service providers. - Other providers can adopt the same core format. ## CASK Key Requirements ### Alphanumeric Encoding - Keys use only the BASE62 alphabet. - Avoiding special characters allows keys to be transmitted without escaping or additional encoding. ### Strong Entropy - Each key contains 52 randomized encoded characters. - This provides approximately 310 bits of entropy. - The design is intended to prevent brute-force attacks, including in a post-quantum environment. ### Fixed Signatures - Every CASK key includes: - The standard signature `JQQJ`. - A provider-specific signature. - Microsoft observed `JQQJ` to be rare in both open-source and internal code, enabling fast and accurate detection. - Azure DevOps uses `AZDO` as its provider signature. - These signatures allow tools to detect CASK keys generically while still supporting provider-specific classification. ### Metadata and Testing Support - Keys include their creation month and year. - Timestamps support incident response and key-rotation enforcement. - CASK reserves dedicated test keys so developers can test scanners and security controls without exposing real credentials. - Microsoft plans to provide more details about Azure-specific metadata. Microsoft recommends that service providers adopt CASK and contribute feedback as the standard evolves.

microsoft

Managed DevOps Pools – The Origin Story (opens in new tab)

Microsoft’s vast, diverse engineering organization had accumulated more than 5,000 self-hosted Azure DevOps pools, creating duplicated tooling, inconsistent reliability, security gaps, and compliance challenges. Its One Engineering System (1ES) team addressed this with 1ES Hosted Pools, a standardized service for flexible, secure, and scalable CI/CD infrastructure. Adoption reduced costs by more than 60%, cut remaining self-hosted pools to a few dozen, and eventually led to the external Managed DevOps Pools offering. ## The Scale and Challenges of Self-Hosted Infrastructure - Microsoft supports over 100,000 engineers across many businesses, programming languages, operating systems, hardware platforms, build engines, and test frameworks. - By 2021, teams had created: - More than 5,000 self-hosted Azure DevOps pools - Hundreds of thousands of agents - Teams needed capabilities unavailable from Microsoft-hosted agents, including: - Larger compute sizes - Private-network connectivity - Custom images - Stateful agents - Long-running tests - The decentralized approach caused: - Duplicate engineering effort - Uneven support and reliability - Poor resource utilization and higher costs - Inconsistent patching and security practices - Difficult and time-consuming compliance audits ## 1ES Hosted Pools - 1ES developed a standardized internal service for custom Azure DevOps infrastructure. - Teams could connect agents to private resources such as package registries, secret managers, and on-premises services. - They could bring custom images, using centrally maintained images as their base. - Business continuity features allowed backup pools and failover to other Azure regions. - Agents were stateless by default, but teams could reuse stateful agents for better performance through local caches. - Stateful agents were automatically recycled based on age or available disk space. - Teams could select Azure VM families and sizes suited to their workload. - Standby agents could be pre-warmed on schedules or automatically provisioned using historical demand. ## Operational and Business Benefits - **Lower costs:** Infrastructure bills fell by more than 60% through improved utilization, better SKU selection, and selective use of Azure Spot VMs. - **Faster development:** Teams spent less time maintaining CI/CD infrastructure and more time building products. - **Simpler compliance:** Standardized telemetry made audits easier and allowed security and compliance improvements to be deployed centrally. - **Greater mobility:** Developers changing teams no longer had to learn different infrastructure-management systems. - **Improved security:** Features such as Azure Confidential VMs, Trusted Launch, and Secure TPM became available across pools. - **Reduced fragmentation:** By 2024, Microsoft had reduced its remaining self-hosted pools from more than 5,000 to only a few dozen. ## From Internal Platform to Managed DevOps Pools - 1ES first built Hosted Pools as an internal “Host On Behalf Of” service to validate whether centralized management could reduce self-hosting. - Success inside Microsoft, combined with customer demand, led to the external **Managed DevOps Pools (MDP)** service. - Organizations using VM Scale Set agents or self-hosted agents can migrate to MDP to gain standardized scaling, security, compliance, and operational support. - The external offering initially does not include every feature available in 1ES Hosted Pools, though additional capabilities may be added later. Centralizing CI/CD infrastructure can eliminate redundant platform work while improving cost efficiency, security, compliance, and developer productivity. Managed DevOps Pools extends Microsoft’s internal solution to organizations facing similar self-hosting challenges.