Log Management

57 posts

datadogJun 3, 20251 min readCurated summary

Detecting faulty deployments: Our journey from unlabeled data to supervised learning | Datadog

The supplied content does not include the blog post itself; it contains Datadog’s navigation menu and a link titled “Detecting Faulty Deployments.” As a result, there is not enough information to accurately summarize the article’s arguments, implementation details, or conclusions. ## Available context - The linked article appears to concern identifying deployments that introduce faults or regressions. - Datadog’s platform covers related capabilities such as: - Application Performance Monitoring - Metrics and infrastructure monitoring - Logs and error tracking - CI Visibility and software delivery monitoring - Service-level objectives and incident response - The page also promotes Datadog’s recognition as a Leader in the Gartner Magic Quadrant for Observability Platforms. ## Missing information - The article’s detection methodology - Metrics, queries, or deployment signals used - Alerting, rollback, or remediation procedures - Technical examples and conclusions Please provide the article text or a page extract containing the post body for an accurate summary.

Log Management Kubernetes Datadog Observability+4

datadogNov 20, 20241 min readCurated summary

How we use formal modeling, lightweight simulations, and chaos testing to design reliable distributed systems | Datadog

The provided content does not include the blog post itself. It contains Datadog’s navigation menu and a link titled “Formal Modeling and Simulation,” but no article text or technical explanation to summarize. ## Available Information - Datadog promotes its observability platform across: - Infrastructure and application monitoring - Logs, databases, and data pipelines - Security - Digital experience monitoring - Software delivery and service management - AI-powered observability - The page also advertises Datadog’s recognition as a Leader in the Gartner Magic Quadrant for Observability Platforms. - The linked engineering article appears to concern formal modeling and simulation, but its subject, methods, and conclusions are not included. ## Conclusion Please provide the blog post’s full text or the relevant article content to receive an accurate technical summary.

Log Management Kubernetes Datadog Observability+3

datadogSep 23, 20241 min readCurated summary

How we optimized LLM use for cost, quality, and safety to facilitate writing postmortems | Datadog

The provided content does not include the blog post itself; it consists primarily of Datadog’s navigation menu and a promotional banner announcing its Gartner recognition. As a result, the article’s argument, technical details, and conclusion cannot be reliably summarized. ## Visible Page Content ### Datadog’s Gartner Recognition - Datadog promotes being named a **Leader in the Gartner Magic Quadrant for Observability Platforms**. - The linked resource appears to concern the **2026** observability-platform evaluation. ### Datadog Product Areas - Infrastructure and application monitoring - Logs, metrics, databases, and data pipelines - Security and cloud protection - Real user monitoring and digital experience - CI/CD and software delivery - Incident response and service management - AI capabilities, including Bits AI, agent observability, and GPU monitoring ### Article Reference - The URL path indicates an article titled **“LLMs for Postmortems.”** - However, no article text or sections about large language models, incident analysis, or postmortem generation are included in the supplied content. Please provide the full article body for an accurate summary.

Log Management AI Agents Kubernetes Observability+4

datadogJun 28, 20243 min readCurated summary

Timeseries indexing at scale | Datadog

Datadog’s “Time Series Indexing at Scale” explains how an observability platform can index and query enormous numbers of time series without making tag-based searches prohibitively expensive. The central challenge is matching flexible combinations of metric names and tags while keeping ingestion, storage, and query latency predictable. The article presents indexing strategies and architectural trade-offs that allow Datadog to support high-cardinality telemetry at scale. ## The Challenge of Time-Series Indexing - A time series is identified not only by its metric name but also by its complete set of tags. - Modern monitoring systems may contain billions of series generated by containers, hosts, services, and dynamic infrastructure. - Queries often filter on multiple tags, requiring the system to efficiently find the intersection of several large sets of series. - Indexing must support both: - Fast writes as new series appear - Low-latency reads for interactive dashboards and alerts - High-cardinality tags make naïve database indexes expensive in both storage and query processing. ## Inverted Indexes for Tags - Datadog uses an inverted-index model that maps searchable terms—such as metric names and tag values—to the series containing them. - A query can retrieve the posting list for each term and intersect those lists rather than scanning every time series. - Common terms may correspond to very large lists, so the system must optimize how these lists are stored, compressed, and combined. - The index separates metadata used to identify series from the time-series values stored for those series. ## Distributed Indexing - Index data is partitioned across machines so that no single node must hold or process the entire dataset. - Sharding enables horizontal scaling as the number of metrics, tags, and customers grows. - Query coordination gathers results from multiple shards and combines them into a single response. - The design must balance: - Even distribution of index data - Avoidance of hot shards - Efficient fan-out during queries - Resilience when individual nodes fail ## Managing Index Growth and Cardinality - Dynamic environments continuously create and remove series, making index lifecycle management essential. - Datadog must handle churn caused by short-lived containers, deployments, and changing tag values. - Compression and compact data structures reduce the memory and storage required for posting lists. - The system distinguishes between frequently queried data and less-used data to control resource consumption. - Cardinality limits and indexing policies help prevent unusually large tag dimensions from overwhelming the system. ## Query Performance and Trade-offs - Indexing every possible attribute would improve search flexibility but increase write, storage, and maintenance costs. - The platform therefore makes trade-offs between indexing coverage, freshness, and query speed. - Query execution can combine index filtering with additional processing over the remaining candidate series. - Caching and reuse of intermediate results can reduce repeated work for common queries. - The architecture is designed to maintain predictable latency even as data volume and query complexity increase. ## Operational Considerations - Large-scale indexing requires monitoring the index itself, including shard balance, ingestion lag, memory usage, and query fan-out. - Background processes must compact, expire, and rebalance index data without disrupting active queries. - Fault tolerance is important because an index outage can affect dashboards and alerts even when the underlying metric data remains available. - Separating indexing from time-series storage allows each subsystem to scale and evolve independently. Datadog’s approach illustrates that scalable observability depends as much on metadata indexing as on storing metric values. Systems handling high-cardinality telemetry should use distributed inverted indexes, compact representations, careful lifecycle management, and explicit trade-offs between flexibility and operational cost.

Log Management Kubernetes Observability Application Performance Monitoring+4

datadogMay 23, 20242 min readCurated summary

How we migrated our static analyzer from Java to Rust | Datadog

Datadog migrated its static code analyzer from Java to Rust to improve performance, resource usage, and operational reliability. The rewrite addressed limitations that became increasingly significant as the analyzer processed larger codebases and ran more analyses in parallel. Rather than replacing everything at once, the team preserved existing behavior and introduced the Rust implementation incrementally. ## Why Move from Java to Rust - Static analysis is computationally intensive and often runs across many files simultaneously. - The Java implementation introduced overhead from: - Garbage collection - High memory consumption - Startup and deployment costs - Difficulty achieving predictable performance under heavy workloads - Rust offered: - Native performance - More predictable memory usage - Lightweight binaries - Safe concurrency without a garbage collector ## Preserving Analyzer Behavior - The primary challenge was maintaining compatibility with the existing analyzer and its rules. - The migration had to preserve: - Parsing behavior - Finding locations and diagnostic messages - Rule semantics - Output formats consumed by Datadog’s products and integrations - The team treated the existing implementation as the behavioral reference while rebuilding internal components in Rust. ## Incremental Migration Strategy - Datadog avoided a risky “big bang” rewrite. - Functionality was migrated in stages, allowing the team to: - Compare Java and Rust results - Detect behavioral differences - Benchmark performance - Roll back or isolate problematic changes - Parallel validation helped ensure that improvements in speed did not produce inconsistent security findings. ## Engineering Trade-offs - Rust improved control over memory and execution, but introduced a steeper learning curve and more explicit systems-level design. - The team had to redesign interfaces between components rather than mechanically translate Java code. - Particular attention was required for: - Error handling - Concurrency - Cross-platform builds - Dependency management - Observability and debugging ## Results and Lessons - The Rust implementation provided a stronger foundation for scaling static analysis workloads. - More predictable resource usage makes it easier to run analyses reliably in CI and other automated environments. - The migration demonstrated that large infrastructure rewrites are most manageable when correctness is continuously checked against the existing system. The practical recommendation is to approach similar rewrites incrementally: define compatibility requirements first, compare old and new implementations continuously, and use measured performance and resource data—not language preference alone—to guide the migration.

Log Management Kubernetes Datadog Observability+4

datadogMay 21, 20242 min readCurated summary

Engineering VP spotlight: Ivo Dimitrov | Datadog

Datadog announces that Gartner named it a Leader in the 2026 Magic Quadrant for Observability Platforms. The supplied content primarily consists of the announcement link and Datadog’s product navigation, so it does not provide Gartner’s evaluation details or the reasoning behind the placement. It does show the breadth of Datadog’s unified observability, security, software delivery, and AI platform. ## Gartner Recognition - Datadog is identified as a “Leader” in Gartner’s Magic Quadrant for Observability Platforms 2026. - The linked resource appears to contain the full announcement and Gartner report, but its substantive text is not included here. ## Broad Observability Platform - **Infrastructure:** Infrastructure, container, Kubernetes, network, serverless, GPU, storage, and cloud cost monitoring. - **Applications and data:** APM, universal service monitoring, profiling, dynamic instrumentation, database monitoring, data streams, jobs, and quality monitoring. - **Logs:** Log Management, Sensitive Data Scanner, Audit Trail, and Observability Pipelines. - **Digital experience:** Browser and mobile RUM, session replay, product analytics, synthetic monitoring, mobile testing, and error tracking. ## Security and Software Delivery - Security offerings include code security, SAST, SCA, cloud security, SIEM, workload protection, vulnerability management, compliance, and application/API protection. - Software delivery tools cover CI visibility, test optimization, continuous testing, code coverage, feature flags, IDE plugins, and internal developer portals. - Service-management capabilities include incident response, event management, SLOs, case management, workflow automation, and service catalogs. ## AI and Platform Capabilities - Datadog highlights AI features such as Bits AI agents, investigation tools, AI integrations, MCP Server, and agent observability. - Platform features include dashboards, alerts, notebooks, Watchdog, access control, governance, fleet automation, and mobile access. Datadog’s positioning is as a consolidated platform spanning telemetry, application and infrastructure monitoring, security, developer workflows, and AI operations. For the specific Gartner assessment, readers would need to consult the linked announcement or report.

Log Management Kubernetes Datadog Observability+3

datadogMay 20, 20241 min readCurated summary

.NET Continuous Profiler: Memory usage | Datadog

Datadog is presented as a Leader in the 2026 Gartner® Magic Quadrant™ for Observability Platforms. However, the provided content contains only the announcement headline, link, and website navigation; it does not include the blog post’s analysis, criteria, or supporting details. ## Announcement - Datadog’s headline claim is recognition as a “Leader” in Gartner’s 2026 Magic Quadrant for Observability Platforms. - The linked page appears to be a Datadog resource or announcement page. ## Available Product Scope The navigation indicates that Datadog’s observability platform spans: - Infrastructure monitoring, metrics, containers, Kubernetes, networks, serverless systems, and cloud costs - Application performance monitoring, profiling, and dynamic instrumentation - Logs, databases, data pipelines, and data quality - Security monitoring and cloud security - Real user monitoring, synthetic monitoring, session replay, and error tracking - CI/CD visibility, testing, developer portals, and software delivery - Incident management, service catalogs, SLOs, workflow automation, and AI-powered investigation Because the actual article text is missing, no further claims about Gartner’s evaluation or Datadog’s strengths can be reliably summarized.

Log Management Kubernetes Datadog Observability+4

datadogApr 18, 20241 min readCurated summary

How we brought Datadog's data visualization to iOS: A focus on performance | Datadog

The provided content does not include the blog post’s actual article text. It contains Datadog’s navigation menu, product links, and a reference to an engineering post about bringing Datadog data visualization to iOS performance, so its technical argument and conclusion cannot be reliably summarized. ## Available Content - Datadog promotes its recognition as a Leader in the Gartner Magic Quadrant for Observability Platforms. - The navigation lists products across: - Infrastructure and application monitoring - Logs, databases, and data observability - Security - Digital experience monitoring - Software delivery - Service management - AI and platform capabilities - The referenced engineering URL suggests a post focused on implementing Datadog data visualization for iOS performance monitoring, but no implementation details are provided. Please provide the article body or a readable extraction of the post for a substantive technical summary.

Log Management Kubernetes Datadog Observability+4

datadogMar 26, 20241 min readCurated summary

Engineering spotlight: Marie-Laure Bardonnet | Datadog

The provided content does not include the actual blog post. It contains Datadog’s navigation menu and a promotional banner announcing its recognition as a Leader in the 2026 Gartner® Magic Quadrant™ for Observability Platforms, but no article text to summarize. ## Datadog’s Observability Announcement - Datadog promotes its recognition as a Leader in Gartner’s 2026 Magic Quadrant for Observability Platforms. - The linked announcement appears to be a marketing resource rather than the blog post itself. ## Datadog Product Areas Listed - Infrastructure monitoring, metrics, containers, Kubernetes, networking, serverless, and cloud costs - Application performance monitoring, profiling, dynamic instrumentation, and database monitoring - Log management, observability pipelines, and sensitive data scanning - Security, including cloud security, SIEM, workload protection, and code security - Digital experience tools such as RUM, session replay, synthetic monitoring, and error tracking - Software delivery, CI visibility, testing, feature flags, and code coverage - Service management, incident response, workflows, dashboards, and AI capabilities The article body or source text is needed for a meaningful section-by-section summary.

Log Management Kubernetes Datadog Observability+2

datadogMar 26, 20243 min readCurated summary

Engineering spotlight: Marie-Laure Bardonnet

Marie-Laure Bardonnet’s Datadog career illustrates how engineers can grow through both technical and management paths. After working on Dashboards and Notebooks, she moved into distributed backend systems, eventually leading Datadog’s Logs engineering organization. Her approach emphasizes engineering-informed leadership, deliberate career planning, mentorship, and embracing unfamiliar challenges. ## From Web Engineering to Logs Leadership - Bardonnet joined Datadog full-time in 2017 after interning there. - She began on the Paris-based Dashboards team, where she helped: - Launch the Notebooks product. - Build the backend for a responsive Dashboard layout. - Encouraged by her manager, she transitioned into backend engineering and joined the Logs team. - Logs engineering involved real-time ingestion, processing, enrichment, storage, and querying of millions of log payloads daily. - As Datadog’s products developed shared technical requirements, Logs engineers collaborated closely with a centralized Platform team. - After one year as an individual contributor, Bardonnet became a team lead and later advanced to Engineering Manager II, overseeing both backend and frontend Logs teams. ## Balancing Product Delivery and Technical Health - Her role combines strategic planning, technical decision-making, people development, and recruiting. - At the start of each quarter, teams create OKRs that guide product and technical roadmaps. - Managers balance product priorities with: - Reliability and scalability. - Technical debt reduction. - Cross-team dependencies. - Bardonnet reviews RFCs, incident postmortems, and product documentation to help teams make sound decisions. - She supports both individual contributors and managers by identifying projects that build expertise and leadership skills. - She also participates in weekly hiring committees to recommend candidates and maintain consistent leveling. ## Structuring Teams for Future Growth - As organizations expand, Bardonnet focuses on restructuring teams to improve execution and create better growth opportunities. - Two questions guide this process: - What problems will the organization need to solve about a year from now? - How can everyone progress toward their next career step? - Logs leadership works with Product Management on a three-horizons plan to align future investments with customer needs. - Team design also considers whether each person has appropriately scoped work, meaningful challenges, and sufficient mentorship. ## Building a Self-Directed Career Path - Career planning begins by separating current responsibilities from the work someone ultimately wants to do. - Engineers should reflect on: - What work brings them satisfaction. - What they do well. - What they want to learn. - What legacy they want to leave. - Career goals should be reviewed continuously, organized across different planning horizons, and discussed with leaders. - A strong career path balances personal interests, strengths, learning opportunities, team needs, organizational priorities, and feedback. - Career direction is self-driven and may change over time, but managers and organizational leaders can help identify opportunities and create a suitable path. ## Growth Through Uncertainty - Datadog’s expanding platform and variety of engineering teams mean that career paths differ widely between employees. - Progression may be nonlinear and can require taking risks or accepting unfamiliar challenges. - Growth comes from leaving one’s comfort zone and learning through difficult problems. - Peer feedback helps employees assess whether they are progressing and feel supported. - Regardless of role or trajectory, employees contribute to Datadog’s culture by modeling high standards for quality and delivery. Bardonnet’s experience suggests that career growth is most effective when employees take ownership of their direction while seeking feedback, mentorship, and challenging opportunities from their organization.

Log Management Datadog Okrs Backend Systems+4

datadogDec 21, 20231 min readCurated summary

Scaling Self-Serve Analytics: The Tools Empowering 5,000 Employees | Datadog

The provided content does not include the blog post itself. It contains Datadog’s navigation menu and a promotional banner announcing its position as a Leader in the 2026 Gartner® Magic Quadrant™ for Observability Platforms, but no technical discussion from the article. ## Available Content - The page appears to be a Datadog engineering blog post related to a CrunchConf talk on self-serve analytics. - The supplied text primarily lists Datadog products across: - Infrastructure and application monitoring - Data and log management - Security - Digital experience - Software delivery - Service management - AI capabilities - A separate banner promotes Datadog’s recognition by Gartner. ## Missing Article Details - No article title, introduction, body sections, technical examples, architecture, or conclusions are included. - The available text is insufficient to accurately summarize the post’s arguments or implementation details. Please provide the blog post’s main text or a complete extraction of the page for a substantive summary.

Log Management Kubernetes Datadog Observability+4

datadogOct 27, 20232 min readCurated summary

Engineering spotlight: Jeromy Carriere | Datadog

Datadog announces that Gartner has named it a Leader in the 2026 Magic Quadrant for Observability Platforms. The provided content does not include the report’s evaluation criteria or detailed rationale, but it presents Datadog as a broad observability platform spanning infrastructure, applications, data, logs, security, digital experience, software delivery, and AI. ## Gartner Recognition - Datadog highlights its position as a Leader in Gartner’s 2026 Magic Quadrant for Observability Platforms. - The linked resource appears to provide the full Gartner-related announcement and assessment. ## Broad Observability Coverage - **Infrastructure:** infrastructure, container, network, serverless, GPU, storage, and cloud-cost monitoring. - **Applications:** APM, service monitoring, continuous profiling, dynamic instrumentation, and agent observability. - **Data and logs:** database monitoring, data-stream and job monitoring, log management, sensitive-data scanning, and observability pipelines. - **Security:** code, cloud, workload, vulnerability, compliance, SIEM, and application/API protection. - **Digital experience:** real-user monitoring, session replay, synthetic monitoring, product analytics, mobile testing, and error tracking. - **Software delivery and service management:** CI visibility, test optimization, feature flags, incident response, SLOs, workflow automation, and developer portals. - **AI capabilities:** AI agents, investigation tools, GPU monitoring, integrations, MCP support, and AI-assisted chat and coding. Datadog’s positioning rests on consolidating telemetry and operational workflows across the technology stack. To understand the Gartner recognition in depth, readers would need the linked report, since the supplied excerpt contains the announcement and product navigation but not the supporting analysis.

Log Management Kubernetes Datadog Observability+3

datadogMay 26, 20232 min readCurated summary

Not just another network latency issue: How we unraveled a series of hidden bottlenecks | Datadog

Datadog announces that Gartner has named it a Leader in the 2026 Magic Quadrant for Observability Platforms. The provided content primarily consists of Datadog’s product navigation rather than the full blog post, so it does not include Gartner’s evaluation criteria, Datadog’s specific strengths, or comparative analysis. ## Gartner Recognition - Datadog is identified as a **Leader** in Gartner’s **Magic Quadrant for Observability Platforms 2026**. - The announcement links to a Gartner-related resource hosted by Datadog. - No details are provided in the supplied text about: - Gartner’s assessment of Datadog’s execution or vision - The other vendors evaluated - Specific capabilities that led to the ranking - Any limitations or cautions noted by Gartner ## Datadog’s Observability Offering The surrounding navigation shows that Datadog positions observability as part of a broad platform covering: - Infrastructure and cloud monitoring - Application performance monitoring and profiling - Logs, metrics, databases, and data pipelines - Real user, synthetic, and mobile monitoring - Security and cloud protection - CI/CD and software delivery - Incident response, service management, and automation - AI-powered investigation and agent observability The supplied content does not explain how these products are integrated or how they support the Leader designation. Overall, the material communicates Datadog’s recognition as a Gartner observability-platform Leader, but the full article or report is needed for a meaningful technical summary.

Log Management Kubernetes Datadog Observability+3

datadogFeb 22, 20231 min readCurated summary

Husky: Exactly-once ingestion and multi-tenancy at scale | Datadog

The provided content does not include the blog post itself. It contains Datadog’s navigation menu and a promotional banner announcing its recognition as a Leader in the 2026 Gartner® Magic Quadrant™ for Observability Platforms, but no article text about “Husky.” ## Content Available - The page links to Datadog products covering: - Infrastructure and application monitoring - Logs, databases, and data observability - Security and digital experience - Software delivery and service management - AI and platform capabilities - The banner promotes Datadog’s observability-platform recognition. - No technical details, sections, architecture diagrams, implementation discussion, or conclusions from the Husky article are included. Please provide the article text or a complete page extract for an accurate summary.

Log Management Kubernetes Datadog Observability+2

datadogSep 29, 20221 min readCurated summary

DRUIDS, the design system that powers Datadog | Datadog

The provided content does not include the blog post itself; it contains Datadog navigation links and a URL titled “Druids: The Design System That Powers Datadog.” As a result, the article’s arguments, implementation details, and conclusions cannot be reliably summarized without inventing information. ## Available Information - The page appears to be a Datadog engineering blog post about **Druids**, Datadog’s design system. - The surrounding navigation lists Datadog products across: - Infrastructure and application monitoring - Logs, security, and digital experience - Software delivery and service management - AI and platform capabilities - No article sections, examples, technical architecture, or conclusions are included in the supplied text. ## Practical Conclusion Please provide the article body or a working copy of the page content for an accurate section-by-section summary.

Log Management Kubernetes Datadog Observability+2