The provided Microsoft Engineering blog URL no longer contains the article “Scaling AI for Silicon.” Instead, it returns a 404 error page, so the post’s argument, technical details, and conclusion cannot be determined from the supplied content.
## Missing Article
- The requested URL points to `devblogs.microsoft.com/engineering-at-microsoft/scaling-ai-for-silicon/`.
- Microsoft reports: “Oops, 404 Error! That page can’t be found.”
- No article text, author, publication date, or technical discussion is included.
## Available Resources
- Links are provided to Microsoft Docs, Visual Studio, Microsoft Learn, Developer Community, and the Dev Blogs FAQ.
- The page also lists Microsoft’s other technology blogs, including Azure, .NET, AI, engineering, and Windows-related blogs.
- A link to the Engineering@Microsoft blog homepage is available, but the requested post itself is not.
The article cannot be accurately summarized without its original text or an accessible replacement URL.
Workflows was originally designed for human-paced events, but autonomous agents now create and manage workflow instances at machine speed. To support this shift, the platform increased its limits substantially and redesigned its control plane for horizontal scalability. The new architecture replaces V1’s account-level bottleneck with distributed components while preserving durable execution, retries, and human-in-the-loop pauses.
## The Shift to Agent-Driven Workloads
- Workflows initially handled events such as sign-ups and purchases, typically requiring only one instance per person.
- Persistent agents can operate for hours or days and launch dozens of workflows from a single session.
- Concurrent agents can create thousands of workflow instances within seconds.
- Workflows also serve as durable execution harnesses for agent loops, maintaining progress across failures and supporting asynchronous work.
## Higher Workflows Capacity
The platform now supports:
- **50,000 concurrent instances**, up from 4,500.
- **300 instance creations per second per account**, up from 100.
- **2 million queued instances per workflow**, up from 1 million.
These increases were driven by observed usage patterns and a redesign of the control plane.
## V1: A Single Account-Level Bottleneck
- Each workflow consists of durable, independently retryable steps that can run tasks, wait for events, or sleep until a scheduled time.
- SQLite-backed Durable Objects provide execution, coordination, and storage.
- An **Engine Durable Object** is created for each workflow instance and handles execution, retries, and sleeping.
- A single **Account Durable Object** manages account-wide workflow and instance metadata.
- All create, update, and list operations passed through the Account object.
- High-volume customers could generate thousands of requests per second as instances started and completed, overwhelming the singleton.
- The original rate limits were therefore hard architectural limits rather than adjustable product settings.
## V2: Horizontal Scaling Principles
The redesigned control plane is based on several architectural changes:
- The instance’s **Engine is now the sole source of truth** for whether that instance exists.
- The system verifies that an Engine exists before queuing an instance, avoiding queued instances with no running execution object.
- Instance lifecycle and liveness operations are distributed across workflows and regions so they can scale horizontally.
- The Account singleton stores only essential metadata and has a bounded maximum number of concurrent requests.
- Limits are designed to be flexible and increaseable rather than constrained by one central bottleneck.
## SousChef and Gatekeeper
- V2 introduces two central components: **SousChef** and **Gatekeeper**.
- SousChef acts as a “second in command” to the Account, taking over work that previously concentrated all workflow and instance management in one Durable Object.
- Together, these components are intended to distribute control-plane responsibilities and enable higher creation rates and concurrency.
- The migration was performed with live traffic, allowing customers to move to the new architecture without interruption.
The redesign aligns Workflows with agentic workloads by moving coordination away from a single account-level Durable Object. Developers running high-volume or highly concurrent agents should benefit from the new limits and a control plane that can continue scaling independently.
Amazon S3 began in 2006 as a simple web service for storing and retrieving objects, but its emphasis on security, durability, availability, performance, and elasticity enabled it to become foundational infrastructure. Over two decades, it scaled from roughly one petabyte to hundreds of exabytes while preserving API compatibility, reducing prices, and expanding beyond object storage. Amazon’s long-term vision is for S3 to serve as a universal foundation for data, analytics, and AI workloads.
## The Original S3 Philosophy
- S3 introduced two basic operations:
- `PUT` to store an object
- `GET` to retrieve it
- The service abstracted away complex infrastructure so developers could focus on applications.
- Its five enduring design principles are:
- **Security:** Data is protected by default.
- **Durability:** Designed for 11 nines of durability, with a lossless operating model.
- **Availability:** Failure is assumed and handled throughout the system.
- **Performance:** Storage capacity can grow without degrading performance.
- **Elasticity:** Capacity expands and contracts automatically.
## From One Petabyte to Hundreds of Exabytes
- At launch, S3 had approximately:
- One petabyte of capacity
- 400 storage nodes across 15 racks and three data centers
- 15 Gbps of bandwidth
- A maximum object size of 5 GB
- A price of $0.15 per GB
- Today, S3:
- Stores more than 500 trillion objects.
- Serves over 200 million requests per second.
- Operates across 123 Availability Zones in 39 AWS Regions.
- Supports objects up to 50 TB—10,000 times larger than the original limit.
- Storage prices have fallen by roughly 85%, to slightly above 2 cents per GB.
- S3 Intelligent-Tiering has saved customers more than $6 billion in storage costs.
- The S3 API has become an industry standard, with many other storage systems offering compatible interfaces.
## Backward Compatibility and Long-Term Reliability
- Code written against S3 in 2006 still works without modification.
- AWS has repeatedly replaced disks, storage systems, and request-processing code while preserving access to older data.
- This compatibility reflects S3’s goal of remaining infrastructure that “just works” despite continuous internal change.
## Engineering for Durability and Scale
- Microservices continuously inspect every byte across the fleet.
- Auditor services detect degradation and automatically trigger repair and re-replication.
- Automated formal methods mathematically verify correctness in areas such as:
- The index subsystem
- Cross-Region replication
- Access policies
- AWS has progressively rewritten performance-critical components in Rust over the past eight years.
- Rust improves performance while preventing memory-safety bugs and other classes of errors at compile time.
- S3 follows the principle that scale should improve the service: larger, more distributed workloads become increasingly decorrelated, improving reliability for all customers.
## S3 as a Foundation for Data and AI
Amazon’s future vision is for customers to store data once in S3 and work with it directly, avoiding costly copies and specialized systems.
- **S3 Tables** provides managed Apache Iceberg tables with automated maintenance to improve query performance and reduce storage costs.
- **S3 Vectors** supports semantic search and retrieval-augmented generation, with up to 2 billion vectors per index and sub-100 ms query latency.
- Within five months of launch, customers created over 250,000 indexes, ingested more than 40 billion vectors, and executed over 1 billion queries.
- **S3 Metadata** enables centralized, faster data discovery without recursively listing large buckets.
These additions extend S3 from inexpensive object storage into a broader platform for analytics, search, and AI while retaining its scale and cost advantages.
Toss Design System (TDS) argues that as organizations scale, design systems often become a source of friction rather than efficiency, leading teams to bypass them through "forking" or "detaching" components. To prevent this, TDS treats the design system as a product that must adapt to user demand rather than a set of rigid constraints to be enforced. By shifting from a philosophy of control to one of flexible expansion, they ensure that the system remains a helpful tool rather than an obstacle.
### The Limits of Control and System Fragmentation
* When a design system is too rigid, product teams often fork packages to make minor adjustments, which breaks the link to central updates and creates UI inconsistencies.
* Treating "system bypasses" as user errors is ineffective; instead, they should be viewed as unmet needs in the system's "supply."
* The goal of a modern design system should be to reduce the reason to bypass the system by providing natural extension points.
### Comparing Flat and Compound API Patterns
* **Flat Pattern:** These components hide internal structures and use props to manage variations (e.g., `title`, `description`). While easy to use, they suffer from "prop bloat" as more edge cases are added, making long-term maintenance difficult.
* **Compound Pattern:** This approach provides sub-components (e.g., `Card.Header`, `Card.Body`) for the user to assemble manually. This offers high flexibility for unexpected layouts but increases the learning curve and the amount of boilerplate code required.
### The Hybrid API Strategy
* TDS employs a hybrid approach, offering both Flat APIs for common, simple use cases and Compound APIs for complex, customized needs.
* Developers can choose a `FlatCard` for speed or a `Compound Card` when they need to inject custom elements like badges or unique button placements.
* To avoid the burden of maintaining two separate codebases, TDS uses a "primitive" layer where the Flat API is simply a pre-assembled version of the Compound components.
Design systems should function as guardrails that guide developers toward consistency, rather than fences that stop them from solving product-specific problems. By providing flexible architecture that supports exceptions, a system can maintain its relevance and ensure that teams stay within the ecosystem even as their requirements evolve.
Netflix manages the massive surge of concurrent users during live events by utilizing a hybrid strategy of prefetching and real-time broadcasting to deliver synchronized recommendations. By decoupling data delivery from the live trigger, the system avoids the "thundering herd" effect that would otherwise overwhelm cloud infrastructure during record-breaking broadcasts. This architecture ensures that millions of global devices receive timely updates and visual cues without requiring linear, inefficient scaling of compute resources.
### The Constraint Optimization Problem
To maintain a seamless experience, Netflix engineers balance three primary technical constraints: time to update, request throughput, and compute cardinality.
* **Time:** The specific duration required to coordinate and push a recommendation update to the entire global fleet.
* **Throughput:** The maximum capacity of cloud services to handle incoming requests without service degradation.
* **Cardinality:** The variety and complexity of unique requests necessary to serve personalized updates to different user segments.
### Two-Phase Recommendation Delivery
The system splits the delivery process into two distinct stages to smooth out traffic spikes and ensure high availability.
* **Prefetching Phase:** While members browse the app normally before an event, the system downloads materialized recommendations, metadata, and artwork into the device's local cache.
* **Broadcasting Phase:** When the event begins, a low-cardinality "at least once" message is broadcast to all connected devices, triggering them to display the already-cached content instantaneously.
* **Traffic Smoothing:** This approach eliminates the need for massive, real-time data fetches at the moment of kickoff, distributing the heavy lifting of data transfer over a longer period.
### Live State Management and UI Synchronization
A dedicated Live State Management (LSM) system tracks event schedules in real time to ensure the user interface stays perfectly in sync with the production.
* **Dynamic Adjustments:** If a live event is delayed or ends early, the LSM adjusts the broadcast triggers to preserve accuracy and prevent "spoilers" or dead links.
* **Visual Cues:** The UI utilizes "Live" badging and dynamic artwork transitions to signal urgency and guide users toward the stream.
* **Frictionless Playback:** For members already on a title’s detail page, the system can trigger an automatic transition into the live player the moment the broadcast begins, reducing navigation latency.
To support global-scale live events, technical teams should prioritize edge-heavy strategies that pre-position assets on client devices. By shifting from a reactive request-response model to a proactive prefetch-and-trigger model, platforms can maintain high performance and reliability even during the most significant traffic peaks.
Discord’s September 2025 update focuses on enhancing user expression and scaling server infrastructure to unprecedented levels. By introducing massive server capacity increases and highly customizable interface features, the platform aims to better support its largest communities and most active power users. Ultimately, these changes provide a more dynamic social experience through improved profile visibility, expanded pin limits, and flexible multitasking tools.
### Enhanced User Profiles and Multitasking
- Desktop profiles now feature a refreshed layout designed to showcase a user's current activities and history more clearly.
- Multiple concurrent activities, such as playing a game while listening to music in a voice channel, are now displayed as a "stack of cards" on the profile.
- Activities can be moved into a pop-out floating window, allowing users to participate in shared experiences like "Watch Together" while navigating other servers or DMs.
- A new audio cue now plays whenever a user turns their camera on to provide immediate feedback that their video stream is live.
### Massive Scaling and Embed Improvements
- The default server member cap has been increased to 25 million, supported by engineering optimizations to member list loading speeds for "super-super-large" communities.
- The channel pin limit has been expanded fivefold, moving from a 50-message cap to 250 messages per channel.
- Native support for AV1 video attachments and embeds was integrated to improve video quality and loading performance.
- Tumblr link embeds have been overhauled to include detailed descriptions and metadata for hashtags used in the original post.
### Custom Themes and Aesthetic Upgrades
- Nitro users can now create custom gradient themes using up to five different colors, a feature that synchronizes across both desktop and mobile clients.
- Two new Server Tag badge packs—the Pet pack and the Flex pack—introduce new iconography for server roles, including animal icons and royalty-themed badges.
- Visual updates were made to Group DM icons, which the development team refers to as "facepiles," to better represent groups of friends in the chat list.
Users should explore the new custom gradient settings in their Nitro preferences to personalize their workspace and take advantage of the expanded pin limits to better manage information in high-traffic channels.
Google Research has introduced a novel parallel algorithm called MaxAdaptiveDegree (MAD) to enhance differentially private (DP) partition selection, a critical process for identifying common data items in massive datasets without compromising individual privacy. By utilizing an adaptive weighting mechanism, the algorithm optimizes the utility-privacy trade-off, allowing researchers to safely release significantly more data than previous non-adaptive methods. This breakthrough enables privacy-preserving analysis on datasets containing hundreds of billions of items, scaling up to three orders of magnitude larger than existing sequential approaches.
## The Role of DP Partition Selection
* DP partition selection identifies a meaningful subset of unique items from large collections based on their frequency across multiple users.
* The process ensures that no single individual's data can be identified in the final list by adding controlled noise and filtering out items that are not sufficiently common.
* This technique is a foundational step for various machine learning tasks, including extracting n-gram vocabularies for language models, analyzing private data streams, and increasing efficiency in private model fine-tuning.
## The Weight, Noise, and Filter Paradigm
* The standard approach to private partition selection begins by computing a "weight" for each item, typically representing its frequency, while ensuring "low sensitivity" so no single user has an outsized impact.
* Random Gaussian noise is added to these weights to obfuscate exact counts, preventing attackers from inferring the presence of specific individuals.
* A threshold determined by DP parameters is then applied; only items whose noisy weights exceed this threshold are included in the final output.
## Improving Utility via Adaptive Weighting
* Traditional non-adaptive methods often result in "wastage," where highly popular items receive significantly more weight than necessary to cross the selection threshold.
* The MaxAdaptiveDegree (MAD) algorithm introduces adaptivity by identifying items with excess weight and rerouting that weight to "under-allocated" items sitting just below the threshold.
* This strategic reallocation allows a larger number of less-frequent items to be safely released, significantly increasing the utility of the dataset without compromising privacy or computational efficiency.
## Scalability and Parallelization
* Unlike sequential algorithms that process data one piece at a time, MAD is designed as a parallel algorithm to handle the scale of modern user-based datasets.
* The algorithm can process datasets with hundreds of billions of items by breaking the problem down into smaller parts computed simultaneously across multiple processors.
* Google has open-sourced the implementation on GitHub to provide the research community with a tool that maintains robust privacy guarantees even at a massive scale.
Researchers and data scientists working with large-scale sensitive datasets should consider implementing the MaxAdaptiveDegree algorithm to maximize the amount of shareable data while strictly adhering to user-level differential privacy standards.
Discord scaled its data infrastructure to manage petabytes of data and over 2,500 models by moving beyond a standard dbt implementation. While the tool initially provided a modular and developer-friendly framework, the sheer volume of data and a high headcount of over 100 concurrent developers led to critical performance bottlenecks. To resolve these issues, Discord developed custom extensions to dbt’s core functionality, successfully reducing compilation times and automating complex data transformations.
### Strategic Adoption of dbt
* Discord integrated dbt into its stack to leverage software engineering principles like modular design and code reusability for SQL transformations.
* The tool’s open-source nature allowed the team to align with Discord’s internal philosophy of community-driven engineering.
* The framework offered seamless integration with other internal tools, such as the Dagster orchestrator, and provided a robust testing environment to ensure data quality.
### Scaling Bottlenecks and Performance Issues
* The project grew to a size where recompiling the entire dbt project took upwards of 20 minutes, severely hindering developer velocity.
* Standard incremental materialization strategies provided by dbt proved inefficient for the petabyte-scale data volumes generated by millions of concurrent users.
* Developer workflows often collided, resulting in teams inadvertently overwriting each other’s test tables and creating data silos or inconsistencies.
* The lack of specialized handling for complex backfills threatened the organization’s ability to deliver timely and accurate insights.
### Engineering Custom Extensions for Growth
* The team built a provider-agnostic layer over Google BigQuery to streamline complex calculations and automate massive data backfills.
* Custom optimizations were implemented to prevent breaking changes during the development cycle, ensuring that 100+ developers could work simultaneously without friction.
* By extending dbt’s core, Discord transformed slow development cycles into a rapid, automated system capable of serving as the backbone for their global analytics infrastructure.
For organizations operating at massive scale, standard open-source tools often require custom-built orchestration and optimization layers to remain viable. Prioritizing the automation of backfills and optimizing compilation logic is essential to maintaining developer productivity and data integrity when dealing with thousands of models and petabytes of information.
Datadog uses DDSketch-powered distribution metrics and heatmaps to reveal performance patterns that percentile lines can hide. Heatmaps preserve the full shape of latency distributions across hosts and time, making distinct behavioral modes, seasonality, and outliers visible. The visualization is designed to remain scalable and readable even with hundreds of trillions of underlying datapoints.
## Why Unaggregated Distributions Matter
- Line graphs reduce billions of events to a single value, such as p50, p99, or max.
- Multiple percentile lines provide more context, but the selected percentiles remain arbitrary and can obscure important behavior.
- Aggregated percentile changes may suggest that all requests are slowing when only one subset of traffic is changing.
- Heatmaps expose separate “modes”—distinct groups of measurements with different behavior.
- For example, periodic latency spikes may come from a low-latency benchmarking service rather than from a general degradation in the endpoint.
- Filtering out an identified mode can reveal other patterns, such as daily seasonality in the remaining traffic.
## Building Heatmaps with DDSketch
- DDSketch sacrifices a small amount of precision to represent extremely large numbers of observations efficiently.
- Datadog sends histogram bins and counts to the frontend instead of transmitting every individual datapoint.
- Limiting the number of bins keeps the payload size constant as traffic volume grows.
- Counts use `float32`, supporting values up to approximately `3 × 10^38` per bin—far beyond practical monitoring volumes.
- This allows heatmaps to represent massive datasets, including hundreds of trillions of datapoints.
## Preserving Resolution and Avoiding Aliasing
- Heatmap requests contain time buckets, distribution bins, and counts.
- Since bucket boundaries are shared across a request, Datadog stores those boundaries only once.
- Boundaries must be explicit because distributions may use logarithmic rather than linear scales.
- Time buckets need to align with the source data intervals.
- Misaligned intervals create aliasing artifacts: for example, grouping 10-second data into 7-second buckets produces repeating count patterns such as `[1, 1, 2, 1, 1, 2, …]`.
- Careful discretization preserves the resolution available in the original DDSketch data.
## Designing the Color Scale
- The default palette begins with light blue, consistent with other single-series Datadog visualizations.
- It transitions toward purple to match Datadog’s visual identity.
- The scale avoids lingering on red, which can imply negative alerts, and ends in orange for the hottest values.
- Color choices must communicate both the volume and structure of the distribution.
## Maintaining Dynamic Range
- A few high-count bins can dominate a linear color scale, leaving most of the heatmap visually indistinguishable.
- This is especially problematic for power-law distributions with a dense central mode and a long tail.
- A linear scale may clearly show the main mode around 20 ms while hiding a smaller mode near 1 second.
- Human brightness perception is nonlinear, approximately following a power law described by Stevens’ law.
- Applying nonlinear color interpolation improves the visibility of meaningful differences across both dense regions and long tails.
- This helps preserve distribution details that would otherwise be lost when the color range is dominated by outliers or highly concentrated buckets.
Datadog’s heatmap approach combines DDSketch compression, aligned high-resolution buckets, and perceptually informed color scaling. For systems where averages or a handful of percentiles conceal important subpopulations, distribution heatmaps provide a more reliable way to investigate performance at scale.
