Yaml | Techlist.io

datadogJun 3, 20193 min readCurated summary

Secure publication of Datadog Agent integrations with TUF and in-toto

Datadog built a compromise-resilient CI/CD system to publish Agent integrations independently of full Agent releases. It combines in-toto for end-to-end supply-chain verification with TUF for secure key and metadata distribution. Together, these technologies ensure that users install only integrations derived from developer-approved source code, even if parts of the infrastructure are compromised. ## Independent Integration Publishing - Agent integrations were traditionally bundled into full Agent releases. - This delayed important integration updates and prevented users from trying new integrations immediately. - Datadog wanted automation to build and publish integrations on demand without fully trusting the automation itself. ## End-to-End Verification with in-toto - TLS and GPG package signatures help prevent man-in-the-middle attacks but do not protect against compromised build or publishing infrastructure. - in-toto defines the software supply chain as a fixed sequence of signed steps. - Each step records the inputs it received and the outputs it produced, allowing the Agent to verify that only authorized parties performed the required work. - The integration pipeline includes: - Developers signing Python and YAML source files. - CI/CD packaging the source into Python wheels without modifying existing wheels. - A signing step applying TUF signatures to the wheels. - The Datadog Agent verifying that the downloaded wheel matches the developer-signed source. ## Secure Distribution with TUF - in-toto does not itself provide a secure way to distribute, revoke, or replace verification keys. - TUF supplies signed, compromise-resilient metadata for: - The root of trust for wheels and supply-chain metadata. - The in-toto-defined workflow. - Public keys used to verify the workflow. - TUF protects against tampering, rollback attacks, and indefinite replay of outdated metadata. - Offline trust bootstrapping and protected developer keys are essential to the overall security model. ## Hardware-Protected Developer Signing - Developers use Yubikeys to generate and store GPG signing keys. - Private keys cannot be exported from the device, assuming correct firmware. - Signing requires both a secret PIN and physical interaction with the Yubikey. - A command-line tool integrates in-toto and GPG, preserving a convenient developer workflow while reducing key-compromise risk. ## Transparent Verification for Users - The Datadog Agent automatically invokes TUF and in-toto when downloading or updating integrations. - Users need no workflow changes under normal conditions. - If metadata, signatures, or supply-chain steps fail verification, installation is blocked and the Agent reports the failure. Datadog’s approach demonstrates that secure automated publishing requires layered controls: in-toto verifies how software was produced, while TUF securely manages the trust and distribution mechanisms needed to validate it.

Read original

Yaml Database Design Datadog CI/CD+4

datadogJun 15, 20173 min readCurated summary

Hackathon project: Viewing Datadog metrics in Minecraft

Datadog engineers used a two-day hackathon to display real-time Datadog metrics inside Minecraft. They connected Minecraft’s Python API with Datadog’s metrics API, then built configurable, live-updating graphs and monitor indicators in the game world. The project demonstrated that even an unconventional visualization environment can be practical to prototype with familiar tools. ## Controlling Minecraft with Python - The team used Raspberry Juice and a Minecraft Pi Edition server to expose Minecraft controls. - They ran the setup on laptops for better performance and faster development. - The `py3minepi` library enabled Python code to create, remove, and query blocks. - Creating a block required only a connection to the server and a call such as `mc.setBlock(...)`. ## Retrieving Datadog Metrics - The Datadog Python library provided access to the Metrics API. - The prototype authenticated with an API key and application key. - It queried recent data, such as average system CPU idle time over the previous five minutes. - The Minecraft and Datadog components were then combined so metric values could be rendered as blocks and structures. - Monitor status indicators changed between green and red depending on whether an alert was active. ## YAML-Based Dashboard Configuration - The team moved dashboard definitions out of Python code into YAML files. - Configuration specified: - Graph position, size, and orientation - Visual properties such as colors, transparency, and borders - Datadog queries and time ranges - Monitor IDs and display locations - This allowed complete dashboards containing multiple graphs and monitor indicators to be updated in real time. ## Handling Minecraft’s Persistence - Minecraft blocks remain in the world after being created, while metric graphs change constantly. - Early experiments left behind random cubes that made the world difficult to navigate. - The team implemented “vacuum” functions to remove everything generated by the visualization code before redrawing it. ## Rendering and Performance Challenges - Without browser technologies such as JavaScript and CSS, graphs had to be reduced to rows of data and represented with Minecraft blocks. - Large graphs could overwhelm the data pipeline. - Caching was added to reduce bandwidth usage and avoid repeatedly requesting the same data. - The performance concerns mirrored Datadog’s everyday engineering work, where caching and efficient data handling are essential. ## Hackathon Experience - The first four hours focused on configuring the environment and connecting the systems. - The remaining time was spent experimenting with building, viewing, destroying, and rebuilding metric displays. - The project’s main value was creative exploration rather than production monitoring. The prototype shows how quickly APIs can be combined to create unusual monitoring interfaces. While Minecraft is not intended to replace conventional dashboards, the project is a playful demonstration of real-time data visualization and rapid experimentation.

Read original

Yaml Database Design Datadog Python+3