Datadog’s page announces that the company was named a Leader in Gartner’s 2026 Magic Quadrant for Observability Platforms. However, the provided content contains only the page header, navigation links, and product categories—not the blog post itself—so its technical argument and supporting details cannot be reliably summarized.
## Available Content
- Announcement:
- Datadog was named a Leader in the Gartner® Magic Quadrant™ for Observability Platforms.
- The page links to a Gartner-related resource.
- Product areas listed:
- Infrastructure and application monitoring
- Logs, databases, and data observability
- Security and cloud security
- Digital experience monitoring
- CI/CD and software delivery
- Service management
- AI and observability tools
- The URL references a “noise suppression library,” but no corresponding article text was included.
Please provide the full blog post content for a substantive summary.
Datadog’s “Reliable Log Delivery” post explains how log-collection systems can avoid losing data when networks, destinations, or agents fail. Its central recommendation is to combine acknowledgments, buffering, retries, and controlled backpressure to provide at-least-once delivery without allowing outages to overwhelm the collector.
## Why Reliable Delivery Matters
- Logs are often needed during incidents, precisely when infrastructure and networks may be unstable.
- Temporary destination failures can cause data loss if collectors only keep logs in memory.
- Retrying without limits can create duplicate logs, unbounded memory usage, or cascading failures.
## Buffering and Persistence
- Collectors should buffer logs while downstream services are unavailable.
- In-memory buffers provide speed but cannot survive process crashes or host restarts.
- Disk-backed queues improve durability by preserving unsent logs across transient failures.
- Storage limits are necessary so a prolonged outage does not fill the host’s disk.
## Acknowledgments and Retries
- A log should be removed from the queue only after the destination confirms successful receipt.
- Failed or unacknowledged deliveries are retried, allowing temporary network and service failures to recover automatically.
- At-least-once delivery is the practical reliability target, meaning duplicates may occur and downstream systems should handle them safely.
- Retry policies should use delays and backoff rather than continuously retrying at full speed.
## Backpressure and Operational Trade-offs
- When downstream systems slow down, collectors must apply backpressure instead of accepting unlimited data.
- Backpressure can limit memory consumption and protect the rest of the host.
- Teams must define what happens when buffers reach capacity, such as dropping the oldest data, rejecting new logs, or prioritizing important streams.
- Reliability also requires monitoring queue size, delivery latency, retry rates, and dropped records.
A dependable logging pipeline is not built from retries alone. It requires durable buffering, explicit delivery acknowledgments, bounded resources, and clear failure behavior; organizations should choose retention and overflow policies according to the operational value of their logs.
The provided content does not include the blog post itself. It contains Datadog’s navigation menu and a link whose URL suggests an article about scaling configuration delivery to containers, but no article text or technical sections are available to summarize.
Please provide the post’s body or a readable URL extract, and I can summarize it in the requested format.
The supplied content does not include the blog post itself; it contains Datadog’s navigation menu and a link titled “Detecting Faulty Deployments.” As a result, there is not enough information to accurately summarize the article’s arguments, implementation details, or conclusions.
## Available context
- The linked article appears to concern identifying deployments that introduce faults or regressions.
- Datadog’s platform covers related capabilities such as:
- Application Performance Monitoring
- Metrics and infrastructure monitoring
- Logs and error tracking
- CI Visibility and software delivery monitoring
- Service-level objectives and incident response
- The page also promotes Datadog’s recognition as a Leader in the Gartner Magic Quadrant for Observability Platforms.
## Missing information
- The article’s detection methodology
- Metrics, queries, or deployment signals used
- Alerting, rollback, or remediation procedures
- Technical examples and conclusions
Please provide the article text or a page extract containing the post body for an accurate summary.
Discord’s original Elasticsearch-based search system worked well for billions of messages but became fragile as message volume and cluster size grew. Redis queues could drop messages, bulk operations failed too broadly, large clusters were difficult to operate, and individual indices could hit Lucene’s roughly two-billion-document limit. Discord’s response was to modernize the platform with Kubernetes, the Elastic Kubernetes Operator, and a multi-cluster “cell” architecture built from smaller clusters.
## The Original Search Architecture
- Messages were stored in Elasticsearch indices distributed across two clusters.
- Data was sharded by Discord server (guild) or direct message, keeping each guild’s messages together for efficient queries.
- Messages were indexed lazily because not every message is searched.
- Redis-backed queues supplied workers with message batches for Elasticsearch bulk indexing.
## Problems with the Existing System
### Redis Queue Message Loss
- The realtime indexing queue relied on Redis.
- When Elasticsearch failures caused the queue to back up, Redis CPU usage could reach its limit.
- Once overloaded, Redis began dropping messages, making the indexing pipeline unreliable.
### Fault-Intolerant Bulk Indexing
- A batch could contain messages belonging to many different Elasticsearch indices and nodes.
- A batch of 50 messages might fan out to dozens of nodes.
- If one message failed because its target node was unavailable, Elasticsearch treated the entire bulk request as failed.
- All messages were then re-enqueued, increasing queue pressure.
- In a 100-node cluster with batches of 50 messages, a single failed node gave each batch roughly a 40% chance of encountering a failure.
### Large-Cluster Overhead
- Adding nodes and indices enabled horizontal scaling but increased coordination overhead.
- Bulk operations fanned out across more nodes, slowing indexing.
- Larger clusters also had a higher probability that some node would fail.
### Difficult Upgrades and Restarts
- The system lacked sufficient resilience to individual node outages, making rolling restarts unsafe.
- Clusters exceeding 200 nodes and containing terabytes of data would have taken too long to drain gracefully.
- Discord therefore remained on outdated operating-system and Elasticsearch versions.
- Addressing the Log4Shell vulnerability required taking the entire search system offline while every node was restarted.
### Oversized Indices
- Some indices accumulated messages from extremely large guilds.
- Each Elasticsearch index is backed by a Lucene index with a limit of approximately two billion documents.
- Once that limit was reached, all further indexing failed.
- Discord temporarily recovered by identifying and deleting guilds created primarily for message spam, but this was not viable for legitimate high-volume communities.
## Moving Elasticsearch to Kubernetes
- Discord chose Kubernetes to improve operational flexibility and resource efficiency.
- The Elastic Cloud on Kubernetes (ECK) Operator could define cluster topology and configuration declaratively.
- Kubernetes would automate operating-system upgrades.
- ECK provided tools for safer rolling restarts and Elasticsearch upgrades.
- This marked Discord’s first move toward managing stateful Elasticsearch infrastructure on Kubernetes.
## Smaller Multi-Cluster Cells
- Discord planned to replace very large clusters with a larger number of smaller Elasticsearch clusters.
- Smaller clusters reduce coordination overhead and limit the impact of individual node failures.
- A cell-based design also provides a more manageable scaling and operational boundary than clusters with hundreds of nodes.
Discord’s experience demonstrates that scaling Elasticsearch is not only a matter of adding nodes. Reliable operation requires isolating failures, avoiding oversized indices and fan-out-heavy batches, and designing deployment infrastructure that supports upgrades without taking search offline.
Datadog built a Streaming Platform to make Kafka resilient and dynamically manageable across hundreds of clusters, thousands of topics, and millions of partitions. By decoupling applications from physical Kafka resources, the platform can reroute traffic, rebalance workloads, and fail over in seconds without redeployments. Its design trades strict processing order for parallelism and durability, while adding mechanisms to prevent poisoned events and backlogs from blocking entire partitions.
## A Fluid Control Plane for Multi-Cluster Orchestration
- Kafka infrastructure is treated as interchangeable commodity hardware rather than a fixed application dependency.
- Workloads can shift between clusters, availability zones, and Kubernetes environments automatically.
- The control plane continuously replaces unhealthy components and maintains uninterrupted data flow.
### Resilient Pipelines with Streams
- Streams abstract away Kafka topics and clusters behind stable identifiers.
- A single Stream can span multiple Kafka clusters, availability zones, and Kubernetes clusters.
- Producers and consumers do not need to know the underlying Kafka topology.
- Infrastructure can be reconfigured in real time without application changes or redeployments.
### Live Traffic Failovers
- When a cluster becomes unhealthy, the platform creates a replacement topic and redirects new traffic to it.
- Consumers drain the backlog from the original topic before transitioning to the replacement.
- The same mechanism supports proactive traffic redistribution, cluster decommissioning, capacity changes, and partition adjustments.
- Operations that traditionally take hours can be completed in seconds.
### Consumer Semantics for Scale and Durability
- Datadog uses at-least-once delivery rather than strict processing order.
- Events can be processed independently and in parallel.
- Ordering is restored later in the event store.
- This trade-off enables efficient processing of petabytes of data across distributed infrastructure.
### The Assigner Coordinator
- Kafka’s default group coordinator depends on session timeouts, making failover detection take tens of seconds or minutes.
- Datadog’s Assigner monitors cluster health, resource usage, and workload distribution continuously.
- It reacts in seconds to failures, traffic spikes, and capacity changes.
- Workloads are balanced using real metrics such as CPU load and available resources, allowing heterogeneous environments to be used efficiently.
## Preventing Head-of-Line Blocking
Kafka’s strict per-partition ordering means a single unprocessable event can block all later events. Datadog addresses this reliability problem through independent Stream lanes and a more flexible commit log.
### Stream Lanes and Quality of Service
- Streams contain separate lanes for different priority levels and traffic requirements.
- High-priority real-time traffic can be isolated from slower bursts or late-arriving data.
- A dedicated dead-letter queue lane receives poison pills that cannot be processed.
- Consumers can commit progress after moving failed events to the DLQ, preventing one bad event from blocking the partition without losing data.
### Advanced Commit Logging
- Kafka traditionally maintains one committed pointer per partition.
- That model prevents consumers from advancing past older events that are delayed or still being processed.
- Datadog uses Kafka’s commit metadata to record multiple offsets or offset ranges simultaneously.
- This allows consumers to continue processing live traffic while older events are handled separately, reducing backlog-related blocking.
## Overall Design Philosophy
- The Streaming Platform combines stable logical Streams, real-time orchestration, flexible consumer semantics, QoS isolation, and enhanced offset tracking.
- Together, these components make Kafka more self-healing and suitable for Datadog’s extreme scale.
- The approach prioritizes uninterrupted processing, rapid recovery, and operational flexibility over Kafka’s default assumptions of fixed topology and strict ordering.
Datadog’s design demonstrates that Kafka at very large scale requires a control layer around the broker infrastructure. Abstracting resources, automating failovers, and allowing independent progress through partitions can provide substantially better reliability and throughput than relying solely on Kafka’s native coordination model.
The provided content does not include the blog post itself. It contains Datadog’s navigation menu and a promotional link, including a URL suggesting an article about building a Kafka streaming platform with custom abstractions, but no article text or technical sections to summarize.
Please provide the full blog post content for an accurate summary.
The provided content does not include the blog post itself. It contains Datadog’s navigation links and a reference to an article at `/blog/engineering/husky-storage-compaction/`, but no substantive text about Husky or storage compaction.
## Available Information
- The article appears to be an engineering post about **Husky storage compaction**.
- The surrounding site navigation lists Datadog products across infrastructure, applications, data, logs, security, and AI.
- No technical details, arguments, implementation choices, or conclusions from the article are present.
Please provide the article’s body text or a complete excerpt for an accurate summary.
Postgres was crashing with segmentation faults when executing certain expensive queries on an Arm64 Kubernetes cluster. Investigators reduced the failure to a simple table scan and discovered that disabling JIT compilation prevented the crash. Assembly-level debugging ultimately traced the problem to a bug in LLVM’s Arm64 JIT support.
## Isolating the Crash
- The failures occurred across multiple EC2 nodes, ruling out faulty hardware.
- Query logs showed that the crashes consistently followed a small number of query patterns.
- The simplest reproducer was:
```sql
SELECT repo_id FROM repository;
```
- Core dumps had badly corrupted stacks, but surviving frames pointed to `ExecRunCompiledExpr`, suggesting a failure during JIT execution.
- The unusually short backtraces reinforced the suspicion that the stack itself had been corrupted.
## How PostgreSQL JIT Works
- PostgreSQL normally evaluates SQL expressions through a general-purpose interpreter.
- JIT compilation converts expressions such as `1+1` into native machine code, reducing interpreter overhead for large workloads.
- JIT can also optimize tuple deforming by converting disk tuples into in-memory values more efficiently.
- PostgreSQL uses LLVM to generate the compiled code.
- Because compilation adds overhead and compiled functions are not reused between queries, PostgreSQL enables JIT primarily for expensive queries based on cost thresholds.
## The Query of Death
- The affected query scanned a partitioned `repository` table with:
- 64 partitions
- More than 1.6 million rows
- 128 JIT-generated functions
- Its query plan enabled expression compilation and tuple deforming:
```text
JIT:
Functions: 128
Expressions: true
Deforming: true
Inlining: false
Optimization: false
```
- Running the query with:
```sql
SET jit = off;
```
completed successfully.
- Disabling JIT cluster-wide immediately stopped the crashes without noticeable query-latency effects.
## Root Cause Direction
- The release build of PostgreSQL offered limited debugging flexibility, so the team planned to reproduce the failure in a dedicated test environment.
- Further investigation eventually isolated the defect to JIT compilation on Arm64 systems.
- The underlying issue was identified as an LLVM bug rather than a PostgreSQL query or hardware problem.
- The investigation continued down to generated assembly and resulted in an upstream fix.
The immediate mitigation was to disable PostgreSQL JIT, while the durable solution was to adopt the LLVM fix addressing the Arm64 code-generation bug.
The provided content does not include the blog post itself. It contains Datadog’s navigation menu and a link titled “Formal Modeling and Simulation,” but no article text or technical explanation to summarize.
## Available Information
- Datadog promotes its observability platform across:
- Infrastructure and application monitoring
- Logs, databases, and data pipelines
- Security
- Digital experience monitoring
- Software delivery and service management
- AI-powered observability
- The page also advertises Datadog’s recognition as a Leader in the Gartner Magic Quadrant for Observability Platforms.
- The linked engineering article appears to concern formal modeling and simulation, but its subject, methods, and conclusions are not included.
## Conclusion
Please provide the blog post’s full text or the relevant article content to receive an accurate technical summary.
The provided content does not include the blog post itself; it consists primarily of Datadog’s navigation menu and a promotional banner announcing its Gartner recognition. As a result, the article’s argument, technical details, and conclusion cannot be reliably summarized.
## Visible Page Content
### Datadog’s Gartner Recognition
- Datadog promotes being named a **Leader in the Gartner Magic Quadrant for Observability Platforms**.
- The linked resource appears to concern the **2026** observability-platform evaluation.
### Datadog Product Areas
- Infrastructure and application monitoring
- Logs, metrics, databases, and data pipelines
- Security and cloud protection
- Real user monitoring and digital experience
- CI/CD and software delivery
- Incident response and service management
- AI capabilities, including Bits AI, agent observability, and GPU monitoring
### Article Reference
- The URL path indicates an article titled **“LLMs for Postmortems.”**
- However, no article text or sections about large language models, incident analysis, or postmortem generation are included in the supplied content.
Please provide the full article body for an accurate summary.
Figma migrated most of its core services from AWS ECS to Kubernetes in under 12 months because ECS was increasingly limiting its platform ambitions. Kubernetes offered better support for stateful workloads, Helm-based software, autoscaling, service networking, and the broader CNCF ecosystem. The migration was considered worthwhile because Figma had relatively few core services and had already containerized its workloads, making the transition more manageable.
## Figma’s Existing Compute Platform
- By early 2023, Figma was already running all services in containers on Amazon ECS.
- ECS had enabled rapid adoption of containerized workloads, but Figma’s growing infrastructure team began evaluating a more capable long-term platform.
- Figma is not organized around thousands of microservices:
- A small set of powerful core services provides modularization and traffic isolation.
- New product capabilities are usually added to existing services rather than creating new ones.
- This limited service count made a Kubernetes migration more practical.
## Limitations of ECS
- ECS lacked Kubernetes primitives needed for complex workloads.
- Running `etcd` on ECS required fragile custom startup code to manage cluster membership because ECS does not provide StatefulSets or persistent pod identity.
- Kubernetes StatefulSets provide stable identities and stateful networking for systems such as `etcd`.
- ECS did not natively support deploying groups of services packaged as Helm charts.
- Open-source tools such as Temporal would require manual conversion into Terraform configurations.
- This increased installation and maintenance effort.
- ECS also made routine infrastructure operations more cumbersome.
- For example, safely removing a malfunctioning EC2 instance was difficult.
- EKS can cordon a node and move its pods elsewhere while respecting graceful shutdown behavior.
## Access to the CNCF Ecosystem
- Kubernetes would give Figma access to a larger ecosystem of open-source cloud-native tools.
- Autoscaling was a major motivation:
- Figma was provisioning services for peak demand, wasting resources during lower-traffic periods.
- Kubernetes tooling such as KEDA supports scaling based on CPU, SQS queue length, and custom Datadog metrics.
- Figma expected to adopt a service mesh eventually.
- Existing AWS load balancer routing created operational drawbacks:
- Network Load Balancers could take several minutes to register or remove targets.
- This slowed emergency deployments and increased incident remediation time.
- Envoy offered more customization than AWS load balancers, including custom filters for shedding load during incidents.
- Figma had already deployed standalone Envoy machines for a major service and saw Kubernetes ecosystems such as Istio as a path toward fleet-wide service-mesh adoption.
Figma’s experience suggests that Kubernetes was justified not simply as a replacement for ECS, but as a foundation for more capable operations and broader platform tooling. Organizations considering a similar move should first assess their workload complexity, existing container maturity, and whether Kubernetes capabilities will materially reduce infrastructure work.
Datadog’s “Time Series Indexing at Scale” explains how an observability platform can index and query enormous numbers of time series without making tag-based searches prohibitively expensive. The central challenge is matching flexible combinations of metric names and tags while keeping ingestion, storage, and query latency predictable. The article presents indexing strategies and architectural trade-offs that allow Datadog to support high-cardinality telemetry at scale.
## The Challenge of Time-Series Indexing
- A time series is identified not only by its metric name but also by its complete set of tags.
- Modern monitoring systems may contain billions of series generated by containers, hosts, services, and dynamic infrastructure.
- Queries often filter on multiple tags, requiring the system to efficiently find the intersection of several large sets of series.
- Indexing must support both:
- Fast writes as new series appear
- Low-latency reads for interactive dashboards and alerts
- High-cardinality tags make naïve database indexes expensive in both storage and query processing.
## Inverted Indexes for Tags
- Datadog uses an inverted-index model that maps searchable terms—such as metric names and tag values—to the series containing them.
- A query can retrieve the posting list for each term and intersect those lists rather than scanning every time series.
- Common terms may correspond to very large lists, so the system must optimize how these lists are stored, compressed, and combined.
- The index separates metadata used to identify series from the time-series values stored for those series.
## Distributed Indexing
- Index data is partitioned across machines so that no single node must hold or process the entire dataset.
- Sharding enables horizontal scaling as the number of metrics, tags, and customers grows.
- Query coordination gathers results from multiple shards and combines them into a single response.
- The design must balance:
- Even distribution of index data
- Avoidance of hot shards
- Efficient fan-out during queries
- Resilience when individual nodes fail
## Managing Index Growth and Cardinality
- Dynamic environments continuously create and remove series, making index lifecycle management essential.
- Datadog must handle churn caused by short-lived containers, deployments, and changing tag values.
- Compression and compact data structures reduce the memory and storage required for posting lists.
- The system distinguishes between frequently queried data and less-used data to control resource consumption.
- Cardinality limits and indexing policies help prevent unusually large tag dimensions from overwhelming the system.
## Query Performance and Trade-offs
- Indexing every possible attribute would improve search flexibility but increase write, storage, and maintenance costs.
- The platform therefore makes trade-offs between indexing coverage, freshness, and query speed.
- Query execution can combine index filtering with additional processing over the remaining candidate series.
- Caching and reuse of intermediate results can reduce repeated work for common queries.
- The architecture is designed to maintain predictable latency even as data volume and query complexity increase.
## Operational Considerations
- Large-scale indexing requires monitoring the index itself, including shard balance, ingestion lag, memory usage, and query fan-out.
- Background processes must compact, expire, and rebalance index data without disrupting active queries.
- Fault tolerance is important because an index outage can affect dashboards and alerts even when the underlying metric data remains available.
- Separating indexing from time-series storage allows each subsystem to scale and evolve independently.
Datadog’s approach illustrates that scalable observability depends as much on metadata indexing as on storing metric values. Systems handling high-cardinality telemetry should use distributed inverted indexes, compact representations, careful lifecycle management, and explicit trade-offs between flexibility and operational cost.
Datadog migrated its static code analyzer from Java to Rust to improve performance, resource usage, and operational reliability. The rewrite addressed limitations that became increasingly significant as the analyzer processed larger codebases and ran more analyses in parallel. Rather than replacing everything at once, the team preserved existing behavior and introduced the Rust implementation incrementally.
## Why Move from Java to Rust
- Static analysis is computationally intensive and often runs across many files simultaneously.
- The Java implementation introduced overhead from:
- Garbage collection
- High memory consumption
- Startup and deployment costs
- Difficulty achieving predictable performance under heavy workloads
- Rust offered:
- Native performance
- More predictable memory usage
- Lightweight binaries
- Safe concurrency without a garbage collector
## Preserving Analyzer Behavior
- The primary challenge was maintaining compatibility with the existing analyzer and its rules.
- The migration had to preserve:
- Parsing behavior
- Finding locations and diagnostic messages
- Rule semantics
- Output formats consumed by Datadog’s products and integrations
- The team treated the existing implementation as the behavioral reference while rebuilding internal components in Rust.
## Incremental Migration Strategy
- Datadog avoided a risky “big bang” rewrite.
- Functionality was migrated in stages, allowing the team to:
- Compare Java and Rust results
- Detect behavioral differences
- Benchmark performance
- Roll back or isolate problematic changes
- Parallel validation helped ensure that improvements in speed did not produce inconsistent security findings.
## Engineering Trade-offs
- Rust improved control over memory and execution, but introduced a steeper learning curve and more explicit systems-level design.
- The team had to redesign interfaces between components rather than mechanically translate Java code.
- Particular attention was required for:
- Error handling
- Concurrency
- Cross-platform builds
- Dependency management
- Observability and debugging
## Results and Lessons
- The Rust implementation provided a stronger foundation for scaling static analysis workloads.
- More predictable resource usage makes it easier to run analyses reliably in CI and other automated environments.
- The migration demonstrated that large infrastructure rewrites are most manageable when correctness is continuously checked against the existing system.
The practical recommendation is to approach similar rewrites incrementally: define compatibility requirements first, compare old and new implementations continuously, and use measured performance and resource data—not language preference alone—to guide the migration.
Datadog announces that Gartner named it a Leader in the 2026 Magic Quadrant for Observability Platforms. The supplied content primarily consists of the announcement link and Datadog’s product navigation, so it does not provide Gartner’s evaluation details or the reasoning behind the placement. It does show the breadth of Datadog’s unified observability, security, software delivery, and AI platform.
## Gartner Recognition
- Datadog is identified as a “Leader” in Gartner’s Magic Quadrant for Observability Platforms 2026.
- The linked resource appears to contain the full announcement and Gartner report, but its substantive text is not included here.
## Broad Observability Platform
- **Infrastructure:** Infrastructure, container, Kubernetes, network, serverless, GPU, storage, and cloud cost monitoring.
- **Applications and data:** APM, universal service monitoring, profiling, dynamic instrumentation, database monitoring, data streams, jobs, and quality monitoring.
- **Logs:** Log Management, Sensitive Data Scanner, Audit Trail, and Observability Pipelines.
- **Digital experience:** Browser and mobile RUM, session replay, product analytics, synthetic monitoring, mobile testing, and error tracking.
## Security and Software Delivery
- Security offerings include code security, SAST, SCA, cloud security, SIEM, workload protection, vulnerability management, compliance, and application/API protection.
- Software delivery tools cover CI visibility, test optimization, continuous testing, code coverage, feature flags, IDE plugins, and internal developer portals.
- Service-management capabilities include incident response, event management, SLOs, case management, workflow automation, and service catalogs.
## AI and Platform Capabilities
- Datadog highlights AI features such as Bits AI agents, investigation tools, AI integrations, MCP Server, and agent observability.
- Platform features include dashboards, alerts, notebooks, Watchdog, access control, governance, fleet automation, and mobile access.
Datadog’s positioning is as a consolidated platform spanning telemetry, application and infrastructure monitoring, security, developer workflows, and AI operations. For the specific Gartner assessment, readers would need to consult the linked announcement or report.
