terraform

15 posts

gitlab

GitLab Secrets Manager adds ESO, Terraform, API support (opens in new tab)

GitLab Secrets Manager expands beyond CI/CD by supporting Kubernetes, Terraform/OpenTofu, CLI tools, and external automation. Built on OpenBao and compatible with Vault APIs, it provides one centrally managed secret store with consistent access controls and auditing. The result is fewer duplicated credential stores and safer secret retrieval across the software delivery lifecycle. ## Kubernetes with External Secrets Operator - ESO uses its Vault provider to retrieve secrets from GitLab Secrets Manager. - A Kubernetes workload uses a short-lived GitLab-minted JWT to authenticate with OpenBao. - A `SecretStore` configures: - The Vault-compatible server and KV v2 mount - The GitLab organization, group, and project namespace - JWT authentication and the Kubernetes secret containing the token - An `ExternalSecret` maps remote secrets to a Kubernetes `Secret`. - ESO refreshes values according to `refreshInterval`, allowing rotated credentials to reach workloads without redeployment. - `remoteRef.key`, `property`, and `secretKey` define the source path, field, and destination key. ## Terraform and OpenTofu Integration - Terraform can retrieve secrets at plan or apply time instead of storing them in `.tfvars` files or CI/CD variables. - A script obtains a minted JWT and connection metadata through Terraform’s `external` data source. - The Vault provider uses that JWT to authenticate against GitLab Secrets Manager. - The `vault_kv_secret_v2` data source reads the required secret. - Outputs containing secrets should be marked `sensitive`, though downstream Terraform state handling still requires care. ## OpenBao and Vault CLI - Existing Vault-compatible scripts can access GitLab Secrets Manager without using the API directly. - Users configure `VAULT_ADDR` and `VAULT_NAMESPACE`. - A minted JWT is exchanged for an OpenBao client token through the configured JWT authentication path. - The `vault kv get` command then retrieves secrets from the KV mount. ## Secrets Manager API - The API supports automation outside GitLab CI/CD, Kubernetes, and Terraform. - A service account requests an access token through GitLab’s project API. - The response supplies the Vault server, namespace, mount, secrets path, JWT authentication path, and role. - External systems can use this information to authenticate and fetch secrets without hardcoded credentials or separate variable files. GitLab Secrets Manager is most useful when multiple deployment tools need the same credentials. Centralizing secrets in the OpenBao-backed store, using short-lived JWT authentication, and integrating through ESO, Terraform, CLI, or the API can reduce duplication and improve rotation and auditing.

cloudflare

Cloudflare Internal DNS is now generally available (opens in new tab)

Cloudflare Internal DNS is now generally available as a unified platform for public and private DNS. It combines recursive resolution, authoritative private zones, DNS policy enforcement, and Zero Trust controls on Cloudflare’s global network. The goal is to eliminate fragmented DNS systems and simplify split-horizon management without duplicated configurations or synchronization drift. ## Problems with Traditional Internal DNS - Organizations often manage public DNS, internal DNS, and cloud-provider DNS separately. - Separate control planes create inconsistent policies, limited visibility, and operational overhead. - Split-horizon DNS typically requires parallel environments that can drift and cause outages. - Legacy appliances introduce hardware refresh cycles and scaling constraints. ## Unified DNS and Zero Trust - Public and private DNS share one platform, API, audit trail, and policy layer. - Internal DNS is included for Enterprise customers using Cloudflare Gateway. - Gateway policies determine which users and devices can resolve specific DNS views. - Private name resolution becomes part of the broader Zero Trust architecture. ## Internal DNS Architecture Cloudflare Internal DNS has two main components: - **Gateway Resolver** - Performs recursive resolution and evaluates DNS policies. - Can filter queries or redirect them to different upstream sources. - Provides centralized logging, auditing, and policy management. - **Internal Authoritative DNS** - Serves records for private zones using Cloudflare’s authoritative DNS infrastructure. - Stores resources such as internal applications, databases, and service endpoints. The main configuration objects are: - **Internal Zones:** Authoritative records for private resources. - **DNS Views:** Resolution contexts containing one or more zones. - **Resolver Policies:** Gateway rules that route matching queries to a specific view. - **Zone references:** Allow one shared zone to be reused across multiple views without duplicating records. ## Query Resolution and Change Propagation - Queries first reach the Gateway Resolver for policy evaluation. - Matching policies route queries to an internal DNS view. - Blocked queries are dropped. - Unmatched queries use public resolution through 1.1.1.1. - Views can fall back to public DNS when a name is not found internally. - Changes from the dashboard, API, or Terraform use the same DNS Records API. - Records are validated, persisted, replicated globally, and propagated within seconds as caches are invalidated. ## Getting Started - Enterprise customers using Gateway can access Internal DNS from **Networking → Internal DNS**. - Initial setup generally requires: - Creating an internal zone and records. - Creating a DNS view and associating the zone. - Creating a Gateway resolver policy that routes users or devices to the view. - Terraform is supported and follows the same API ingestion and propagation path. ## Connectivity Cloud Integration - Internal DNS works with Cloudflare One Client, DoH, DoT, standard DNS, PAC files, and Cloudflare WAN. - Cloudflare WAN enables devices across branches, data centers, cloud environments, and remote networks to resolve internal names without installing the client on every device. - The service extends Cloudflare’s existing Connectivity Cloud rather than operating as an isolated DNS product. Cloudflare’s recommendation is to consolidate public DNS, private DNS, and DNS security policies on one control plane, particularly for organizations already using Cloudflare Gateway or WAN.

cloudflare

Improving Smart Tiered Cache for public cloud regions (opens in new tab)

Cloudflare’s Smart Tiered Cache efficiently routes cache misses through one optimal upper-tier data center, but anycast and regional cloud origins make latency-based selection unreliable. Smart Tiered Cache for Public Cloud Regions addresses this by using a customer-provided cloud region hint to choose geographically appropriate primary and fallback tiers. The result is better cache efficiency, fewer origin requests, and reduced cross-continent “hairpin” traffic. ## Smarter Tiered Cache Topology - Smart Tiered Cache is available on all Cloudflare plans at no additional cost. - It selects a single upper tier based on real-time latency when an origin has a fixed, unicast IP. - Previous enhancements added support for: - **R2**, selecting an upper tier near the bucket’s actual location. - **Load Balancing**, selecting one tier for an entire pool so origins share cache content. - The new feature extends this approach to public cloud origins hosted on AWS, GCP, Azure, and Oracle Cloud. ## Why Anycast Origins Cause Problems - Anycast and regional unicast IPs represent cloud provider front ends rather than one physical origin location. - Multiple Cloudflare data centers may appear equally close to the same IP, preventing reliable selection of one upper tier. - The wrong selection can cause hairpin traffic. For example, an origin in Singapore might be assigned to an upper tier in Chicago, forcing Asian users’ requests across the Pacific and back. - This can add hundreds of milliseconds, increase origin connections, and reduce cache hit ratios. - Cloudflare detects likely anycast origins by comparing probe latencies against the physical limits imposed by the speed of light. - When anycast is detected without additional information, Smart Tiered Cache safely uses multiple upper tiers instead of pinning traffic to one. ## Configuring a Cloud Region Hint - In the dashboard, go to **Caching > Tiered Cache > Origin Configuration**. - Select an origin IP and choose **Set Region Hint**. - Enter a region such as: - `aws:us-east-1` - `gcp:europe-west1` - Hints can be configured individually or applied in bulk. - The same configuration is available through the API and Terraform. - Dashboard hints are currently available only for IPs Cloudflare has identified as anycast. - The initial release supports AWS, GCP, Azure, and Oracle Cloud, with more providers planned. ## How Region-Based Selection Works - Cloudflare periodically downloads each provider’s current IP range files. - These files map cloud regions to their active IP prefixes, allowing Cloudflare to track subnet changes. - Cloudflare matches those prefixes against an upper-tier database updated through latency probing every 15 minutes. - Each matching subnet contributes a weighted vote for its current upper-tier assignment. - The strongest signal determines the region’s primary upper tier. - Primary and fallback tiers are always placed in different Cloudflare points of presence to avoid a single-PoP failure eliminating both options. ## Practical Recommendation For public cloud origins behind anycast or regional unicast IPs, configure the correct cloud region hint through the dashboard, API, or Terraform. This lets Smart Tiered Cache retain a single, nearby primary tier while maintaining a separate fallback, improving cache efficiency and avoiding unnecessary cross-region traffic.

cloudflare

Turning Cloudflare’s threat indicators into real-time WAF rules (opens in new tab)

Cloudflare now lets security teams turn live Threat Events intelligence into proactive WAF rules. Instead of manually blocking known malicious IPs, organizations can filter traffic by threat actor, targeted industry or country, attack type, and recency. The integration separates always-on detection from mitigation, allowing teams to validate intelligence before enforcing blocks without sacrificing visibility. ## Always-on Threat Detection - The system builds on Cloudflare’s always-on Attack Signature Detection framework. - Threat intelligence runs continuously in the background and enriches HTTP requests with metadata before a WAF action is taken. - This removes the traditional “log versus block” trade-off: teams can observe detections in analytics and later create blocking rules. - Cloudforce One subscribers can review threat actors, targeted industries, and traffic patterns before enabling mitigation. - Detection is designed to add negligible latency. - The initial release supports IP-based matching, with planned support for JA3 fingerprints and domain-based indicators to identify attackers who rotate IP addresses. ## New WAF Intelligence Fields Cloudflare exposes threat context through fields that can be used in WAF custom rules and rate-limiting policies: - `cf.intel.ip.attacker_names`: Known threat groups, such as `CRAVENFLEA`. - `cf.intel.ip.target_industries`: Industries previously targeted by the IP. - `cf.intel.ip.attacker_countries`: Countries associated with the threat event. - `cf.intel.ip.target_countries`: Countries targeted by the event. - `cf.intel.ip.datasets`: Intelligence sources or attack categories, such as `ddos` and `waf`. Because an IP may have multiple associated actors, industries, or datasets, these values are arrays. Rules use `any()` with the `[*]` wildcard to match individual values. ## Example WAF Rules - Block IPs involved in DDoS activity against France: ```text any(cf.intel.ip.target_countries[*] == "FR") and any(cf.intel.ip.datasets[*] == "ddos") ``` - Block BLACKBASTA-associated traffic targeting banking and financial services: ```text any(cf.intel.ip.target_industries[*] == "Banking & Financial Services") and any(cf.intel.ip.attacker_names[*] == "BLACKBASTA") ``` - Apply broad protection against traffic originating from Iran: ```text any(cf.intel.ip.attacker_countries[*] == "IR") ``` ## Deployment Through Existing Workflows - The fields are available in the WAF rule builder for custom rules and rate limiting. - Teams can configure them through the Cloudflare API or Terraform. - Rules can be automated across individual domains or an entire account. - Security Analytics records matches, including the triggered rule and the specific intelligence indicator involved. - Threat Events users can create Saved Views—such as IPs targeting the financial sector within the last seven days—and convert them into WAF rules with one click. ## Global Distribution and Performance - Cloudflare compresses threat intelligence datasets into a high-performance format. - The data is distributed to Cloudflare data centers worldwide. - The WAF can therefore evaluate threat indicators at the edge without requiring local IP lists or introducing significant request latency. - The post begins to describe this lookup process as O(1), but the provided content ends before explaining the implementation in detail. Cloudflare’s integration gives teams a practical path from threat intelligence to enforcement: investigate indicators, validate them in analytics, and deploy precise WAF or rate-limiting rules through the UI, API, or Terraform. Teams should use the always-on visibility to tune criteria before blocking, while preparing for future indicators such as JA3 fingerprints and malicious domains.

aws

Introducing the next generation of AWS Resilience Hub for generative AI-based SRE resilience journey | Amazon Web Services (opens in new tab)

AWS introduces the next generation of AWS Resilience Hub to help organizations manage application resilience consistently across large portfolios. The update adds a new business-oriented application model, modular resilience policies, dependency discovery, generative AI failure-mode analysis, and organization-wide reporting. Its goal is to help SREs and development teams define expectations, identify weaknesses, implement fixes, and demonstrate resilience through testing. ## Modular Resilience Policies - Policies are built from composable requirements rather than a single fixed policy type. - Requirements can include: - Availability service-level objectives (SLOs) - Multi-AZ and multi-Region disaster recovery - Recovery time objectives (RTOs) - Recovery point objectives (RPOs) - Backup data recovery requirements - Example policies can define 99.95% availability, a 15-minute RTO, and a 5-minute RPO for financial applications. ## Business-Oriented Application Modeling - A **system** represents a business application. - **User journeys** describe critical end-user paths and business outcomes. - **Services** represent deployable units such as microservices, AWS resources, code, and observability components. - Resilience Hub automatically maps resource relationships into a topology showing data flow, containment, and permissions. ## Dependency Discovery - Resilience Hub analyzes VPC DNS query logs to identify dependencies that may not be documented. - It can discover: - AWS services - Internal endpoints - Third-party endpoints - Unexpected cross-Region calls - Critical external dependencies - Dependency discovery can be enabled or disabled for individual services. ## Generative AI Failure-Mode Assessments - Assessments analyze services against: - Configured resilience policies - AWS Well-Architected best practices - The AWS Resilience Analysis Framework - Findings explain the failure mode, its architectural impact, recommended remediation, and related policy requirement. - Users can add or modify assertions to guide the AI agents and improve assessment accuracy. - Findings can be marked as resolved after remediation or irrelevant when they do not apply. ## Getting Started and Enterprise Management - Users must configure an invoker IAM role granting read-only access to AWS resources. - Cross-account roles or AWS Organizations service-linked roles can support multi-account environments. - AWS Organizations integration allows a delegated administrator to assess resilience across the enterprise without signing into individual accounts. - A typical workflow is: - Create a resilience policy. - Create a system and service. - Associate resources through tags, CloudFormation, Terraform, or Amazon EKS. - Enable dependency discovery. - Run a failure-mode assessment. - Review and implement recommendations. ## Migration, Availability, and Pricing - Migration APIs convert older Resilience Hub policies and applications into the new model, including mapping multiple related applications into one system with multiple services. - The new version is generally available in supported AWS commercial Regions. - Pricing uses a service-based model and includes two failure-mode assessments per service per month, with automated dependency assessment available optionally. - AWS offers a free trial. The updated Resilience Hub is most useful for organizations that need consistent resilience standards and evidence across many AWS accounts and applications. Teams should begin by defining reusable policies, modeling critical services and dependencies, and using the AI assessments to prioritize remediation.

cloudflare

Building a CLI for all of Cloudflare (opens in new tab)

Cloudflare is rebuilding Wrangler into a unified CLI for its entire platform, motivated by the growing role of coding agents in configuring and deploying Cloudflare applications. The technical preview, available as `npx cf` or the globally installed `cf` package, currently covers only a subset of products but is intended to support the full API surface. The effort depends on a new TypeScript-based schema system that can generate consistent commands, configuration, bindings, documentation, and agent-oriented interfaces. ## A CLI for all of Cloudflare - Cloudflare offers more than 100 products and nearly 3,000 HTTP API operations. - Agents increasingly use Cloudflare APIs to: - Build and deploy applications - Configure accounts - Query analytics and logs - Create agents and platforms - Cloudflare aims to expose its products consistently through: - CLI commands - Workers Bindings - SDKs - Configuration files - Terraform - Documentation and OpenAPI schemas - MCP servers and Agent Skills - The new Wrangler technical preview can be tried with: - `npx cf` - `npm install -g cf` - A broader internal version already supports the full Cloudflare API, with ongoing work to make command output useful for both humans and agents. ## A new schema and code-generation pipeline - Existing OpenAPI schemas already generate: - Cloudflare SDKs - The Terraform provider - The Code Mode MCP server - Other interfaces, including Wrangler commands, Workers Bindings, configuration, documentation, and Agent Skills, were previously maintained manually. - Manual synchronization was error-prone and could not scale to Cloudflare’s full product range. - OpenAPI alone is insufficient because it primarily describes REST APIs, while Cloudflare also needs to represent: - Interactive CLI workflows - Multiple local and remote actions - RPC-style Workers Bindings - Agent Skills and related documentation - Cloudflare therefore created a TypeScript schema format containing: - API definitions - CLI commands and arguments - Context required to generate different interfaces - Conventions, linting, and guardrails enforce consistency while allowing the schema to generate OpenAPI and future interfaces. ## Consistency for agents and humans - Agents depend on predictable command names and flags. Inconsistent syntax can cause them to call commands that do not exist. - Cloudflare is enforcing conventions at the schema layer, including: - `get`, never `info` - `--force`, never `--skip-confirmations` - `--json`, never `--format` - Applying these rules across interfaces avoids discrepancies between the CLI, REST APIs, and SDKs. - Wrangler must also clearly distinguish local and remote resources. - This is especially important for D1, R2, and KV, where local simulation and remote bindings can coexist. - Clear defaults and output indicating whether an operation targets local or remote resources help agents avoid modifying the wrong environment. ## Local Explorer for simulated resources - Local Explorer is available in open beta through Wrangler and the Cloudflare Vite plugin. - It lets developers inspect locally simulated: - KV - R2 - D1 - Durable Objects - Workflows - Local resources use the same underlying API structure as Cloudflare’s remote APIs and Dashboard. - Cloudflare’s local development environment runs Workers APIs locally, including D1 backed by SQLite through Miniflare. - Previously, developers had to inspect `.wrangler/state` or use third-party tools to understand local data. - Local Explorer provides an interface showing: - Which bindings are attached to a Worker - What data those bindings contain - It can be opened with the `e` keyboard shortcut and helps developers or agents verify schemas, seed test data, and reset local databases. Cloudflare’s direction is to make Wrangler a consistent, machine-readable interface to the entire platform. The technical preview is early, but the new schema-driven system and Local Explorer establish the foundation for a CLI that is easier for both developers and coding agents to use safely.

cloudflare

A one-line Kubernetes fix that saved 600 hours a year (opens in new tab)

Atlantis restarts were taking about 30 minutes, blocking infrastructure changes and consuming more than 50 engineering hours monthly. The delay was caused by Kubernetes recursively changing ownership on a large Ceph-backed PersistentVolume containing millions of files. Setting `fsGroupChangePolicy: OnRootMismatch` avoided unnecessary recursive ownership changes and reduced restart time dramatically. ### The Restart Bottleneck - Atlantis runs as a singleton Kubernetes `StatefulSet`. - Its PersistentVolume stores repository and Terraform state. - Credential rotations, onboarding, and offboarding required restarting Atlantis. - With roughly 100 restarts per month, each 30-minute delay created more than 600 hours of annual lost engineering time. - The volume had grown large enough to exhaust inodes, making storage expansion and pod restarts necessary. ### Kubernetes Made the Delay Look Like a Scheduling Problem - `kubectl rollout restart statefulset atlantis` terminated the old pod and created a replacement. - The new pod was scheduled quickly but remained stuck in `Init:0/1`. - Kubernetes events showed the image pulling successfully, but revealed no obvious cause for the long gap. - Kubelet logs showed the PersistentVolume mounting successfully, followed by repeated `context deadline exceeded` errors while syncing the pod. ### The Hidden Cost of `fsGroup` - Searching logs using the PersistentVolume name exposed the relevant message: - Kubernetes was “setting volume ownership” because an `fsGroup` was configured. - Kubernetes warned that ownership changes could be slow when a volume contained many files. - The default behavior recursively changed ownership across the entire mounted volume. - As Atlantis’s volume accumulated millions of files, this initialization step became the 30-minute bottleneck. ### The One-Line Fix - The volume configuration was changed to: ```yaml fsGroupChangePolicy: OnRootMismatch ``` - With this policy, Kubernetes checks the root directory’s ownership and only performs recursive changes when necessary. - Existing volumes with the correct ownership no longer require a full filesystem traversal during every restart. The practical lesson is to inspect kubelet and volume logs when a pod appears scheduled but remains stuck before initialization. For large persistent volumes, explicitly setting `fsGroupChangePolicy: OnRootMismatch` can eliminate costly recursive ownership changes and prevent substantial operational downtime.

github

GitHub expands application security coverage with AI‑powered detections (opens in new tab)

GitHub is expanding application security coverage with AI-powered detections that complement CodeQL’s traditional static analysis. The approach targets languages and frameworks that are difficult to support through semantic analysis alone, including Bash, Dockerfiles, Terraform, and PHP. Planned for public preview in early Q2, the system brings detection, automated remediation, and enforcement directly into pull requests. ## Hybrid Static Analysis and AI Detection - CodeQL remains the primary tool for deep analysis of supported languages. - AI-powered detections extend coverage to scripts, infrastructure definitions, and less-supported ecosystems. - The system can identify vulnerabilities and suggest fixes within the pull request workflow. - Internal testing analyzed more than 170,000 findings in 30 days, receiving positive feedback from over 80% of developers. - Early supported areas include: - Shell/Bash - Dockerfiles - Terraform/HCL - PHP - The capability is part of GitHub’s broader agentic detection platform, which also supports code quality and code review. ## Security Findings in Pull Requests - GitHub automatically analyzes changes when a pull request is opened. - It selects CodeQL or AI-powered detection based on the code being reviewed. - Findings appear alongside existing code-scanning results, without requiring developers to switch tools. - Example risks include: - Unsafe string-built SQL queries or commands - Weak cryptographic algorithms - Infrastructure configurations exposing sensitive resources - Detecting issues during review allows teams to address vulnerabilities before code is merged or deployed. ## Copilot Autofix for Remediation - GitHub connects detection with Copilot Autofix, which proposes fixes developers can review, test, and apply. - Autofix resolved more than 460,000 security alerts in 2025. - Alerts were resolved in an average of 0.66 hours with Autofix, compared with 1.29 hours without it. - This reduces the gap between discovering a vulnerability and correcting it. ## Security Enforcement at Merge - GitHub positions pull requests as the point where security policies can be enforced. - Detection, remediation, and governance operate within the same workflow. - Teams can reduce risk without adding separate post-deployment review steps. - GitHub plans to demonstrate the technology at RSAC, highlighting hybrid detection and developer-native remediation. GitHub’s recommendation is effectively to combine CodeQL’s precision with AI-based coverage for modern, diverse repositories, while using Copilot Autofix and merge policies to turn findings into timely, enforceable fixes.

cloudflare

Beyond the blank slate: how Cloudflare accelerates your Zero Trust journey (opens in new tab)

Cloudflare argues that a Zero Trust platform’s blank-slate flexibility can become an adoption barrier when customers must configure countless policies and security controls themselves. Project Helix addresses this by codifying Cloudflare experts’ best practices into automated Terraform templates, delivered through a simple web interface. The result is a faster, more consistent way to deploy a secure Cloudflare One baseline within minutes rather than hours. ## The complexity barrier of a blank slate - Cloudflare One offers extensive capabilities across DNS protection, network security, Secure Web Gateway, TLS inspection, DLP, antivirus scanning, and Zero Trust Access. - Tenants are generally provisioned with minimal defaults because enabling advanced protections immediately could disrupt existing traffic and applications. - Customers must therefore manually activate numerous settings, policies, and routing changes. - Some features require coordinated configuration: - Enabling private application access by hostname requires both a platform setting and a specific CGNAT range in the client’s split-tunnel configuration. - Traffic from applications such as Zoom may need to bypass Cloudflare and go directly to the Internet. - Captive portal exceptions can be important for users connecting from hotels, airlines, and other public networks. - Initial setup guides and scenario-based wizards helped, but customers using multiple scenarios still had to complete each workflow separately. ## Project Helix: Turning expertise into automation - Cloudflare gathered deployment knowledge from Solutions Engineers, Professional Services Engineers, and partners. - The team documented desired proof-of-concept and production outcomes, including: - Baseline DNS, network, and HTTP security protections - TLS inspection - QUIC and HTTP/3 security - Remote Browser Isolation for risky categories such as newly registered domains - Visibility and controls for AI applications - Tenant Control policies restricting users to approved SaaS instances - Helix packages these recommendations in a repeatable, codified format that can be applied with a button click. - This avoids relying on individually maintained documentation or the memory of experienced administrators. ## Problems with manual deployment - Configuring the complete baseline on a new tenant can take several hours. - Documentation must be continually updated as Cloudflare features and best practices change. - Repetitive manual steps increase the risk of configuration errors and inconsistent deployments. - Manual work also makes it harder for less experienced users to benefit from Cloudflare One’s full capabilities. ## Terraform, Workers, and ephemeral provisioning - Helix uses scalable Terraform templates to define Cloudflare One settings, configuration snippets, and security policies. - A web interface hosted on Cloudflare Workers accepts basic customer inputs and executes the Terraform configuration. - Cloudflare Containers support the provisioning workflow. - The process uses no persistent storage, reducing risks associated with retaining Terraform logs or authentication tokens. - Within minutes, users can deploy an advanced baseline configuration and review additional recommended policies to enable. ## Layered security configuration - Helix begins with DNS security policies that: - Support corporate DNS for Zero Trust - Block malicious or questionable categories before they resolve - It then applies network policies to protect users across ports and protocols. - The broader configuration also incorporates traffic-routing exceptions, application controls, and user-experience improvements such as captive portal handling. Project Helix’s practical recommendation is to replace manual, blank-slate configuration with expert-designed, automated baselines. This lets customers adopt Cloudflare One’s advanced protections quickly while preserving the flexibility to customize policies for their own environments.

cloudflare

The truly programmable SASE platform (opens in new tab)

Cloudflare argues that true SASE programmability goes beyond APIs, Terraform, webhooks, and alerts. It means intercepting security events, enriching them with external context, and making real-time decisions through custom logic. By running Cloudflare One and its Developer Platform on the same global edge network, Cloudflare aims to let customers apply programmable, low-latency policies without stitching together separate infrastructure. ## What “Programmability” Means - Traditional programmability supports configuration and automation, such as sending Slack alerts when policies trigger. - True programmability allows security systems to: - Inspect an event before access is granted. - Query external systems for additional context. - Make or change an access decision in real time. - Example: a request to a regulated application could be checked against a learning management system to confirm that the user’s compliance training is current. Expired or missing certification would result in denial and redirection to training. ## Cloudflare’s Programmable SASE Architecture - Cloudflare’s network spans more than 330 cities and reaches approximately 95% of Internet-connected users within 50 milliseconds. - Cloudflare One and the Developer Platform run on the same infrastructure and use shared network primitives. - This enables Workers to extend inline services such as Access without requiring separate cloud infrastructure. - Customers can: - Call external risk APIs. - Add dynamic request headers. - Validate browser attributes. - Route traffic according to custom business logic. - Running custom logic at the edge reduces latency and avoids the operational overhead of webhook-based integrations and disconnected systems. ## Custom Actions in Security Policies - Conventional gateways generally limit policy outcomes to actions such as allow, block, isolate, or quarantine. - Cloudflare is expanding policies to support managed and custom actions. - Potential uses include: - Injecting headers based on user identity claims. - Obtaining real-time verdicts from external risk engines. - Restricting access based on location or working hours. - Updating risk lists based on scheduled analysis of user activity. - Custom actions can invoke a Worker when a Gateway HTTP policy matches, giving the code access to request context and allowing decisions in milliseconds. - Managed actions will offer templates for common use cases such as IT service management, redirects, and compliance workflows. ## Automated Device Session Revocation - One customer needed periodic re-authentication for Cloudflare One Client users, similar to traditional VPN session expiration. - Cloudflare’s built-in session controls were application-specific rather than global and time-based. - The customer implemented a scheduled Worker that: - Queries the Cloudflare Devices API. - Handles cursor-based pagination to retrieve registrations. - Calculates how long each device has been inactive. - Deletes registrations exceeding a configured inactivity threshold. - Forces affected users to authenticate again through their identity provider. - The example also supports environment-based configuration and a dry-run mode for testing before revocations are applied. Cloudflare’s recommendation is to treat SASE policies as programmable decision points rather than fixed allow-or-block rules. Combining Cloudflare One with edge Workers can provide faster, more context-aware security automation while reducing integration complexity.

figma

How We Migrated onto K8s in Less Than 12 months | Figma Blog (opens in new tab)

Figma migrated most of its core services from AWS ECS to Kubernetes in under 12 months because ECS was increasingly limiting its platform ambitions. Kubernetes offered better support for stateful workloads, Helm-based software, autoscaling, service networking, and the broader CNCF ecosystem. The migration was considered worthwhile because Figma had relatively few core services and had already containerized its workloads, making the transition more manageable. ## Figma’s Existing Compute Platform - By early 2023, Figma was already running all services in containers on Amazon ECS. - ECS had enabled rapid adoption of containerized workloads, but Figma’s growing infrastructure team began evaluating a more capable long-term platform. - Figma is not organized around thousands of microservices: - A small set of powerful core services provides modularization and traffic isolation. - New product capabilities are usually added to existing services rather than creating new ones. - This limited service count made a Kubernetes migration more practical. ## Limitations of ECS - ECS lacked Kubernetes primitives needed for complex workloads. - Running `etcd` on ECS required fragile custom startup code to manage cluster membership because ECS does not provide StatefulSets or persistent pod identity. - Kubernetes StatefulSets provide stable identities and stateful networking for systems such as `etcd`. - ECS did not natively support deploying groups of services packaged as Helm charts. - Open-source tools such as Temporal would require manual conversion into Terraform configurations. - This increased installation and maintenance effort. - ECS also made routine infrastructure operations more cumbersome. - For example, safely removing a malfunctioning EC2 instance was difficult. - EKS can cordon a node and move its pods elsewhere while respecting graceful shutdown behavior. ## Access to the CNCF Ecosystem - Kubernetes would give Figma access to a larger ecosystem of open-source cloud-native tools. - Autoscaling was a major motivation: - Figma was provisioning services for peak demand, wasting resources during lower-traffic periods. - Kubernetes tooling such as KEDA supports scaling based on CPU, SQS queue length, and custom Datadog metrics. - Figma expected to adopt a service mesh eventually. - Existing AWS load balancer routing created operational drawbacks: - Network Load Balancers could take several minutes to register or remove targets. - This slowed emergency deployments and increased incident remediation time. - Envoy offered more customization than AWS load balancers, including custom filters for shedding load during incidents. - Figma had already deployed standalone Envoy machines for a major service and saw Kubernetes ecosystems such as Istio as a path toward fleet-wide service-mesh adoption. Figma’s experience suggests that Kubernetes was justified not simply as a replacement for ECS, but as a foundation for more capable operations and broader platform tooling. Organizations considering a similar move should first assess their workload complexity, existing container maturity, and whether Kubernetes capabilities will materially reduce infrastructure work.

figma

Inside Figma: securing internal web apps | Figma Blog (opens in new tab)

Figma built a reusable system for securely exposing internal web applications while preserving a smooth employee experience. The design combines AWS Application Load Balancers, Cognito, Okta, SAML, Lambda, and Terraform to enforce zero-trust authentication and centralized authorization. Its main conclusion is that carefully integrated, managed cloud components can provide strong security without creating excessive operational work for a small security team. ## Security Requirements for Internal Applications Figma’s internal web tools support critical workflows such as software deployment and customer support, making them attractive targets for attackers. The system was designed around five requirements: - **Smooth user experience:** Authentication should be fast, reliable, and convenient. - **Zero-trust access:** Network location alone should not establish trust. - **Modern authentication:** Applications should be able to use protections such as WebAuthn. - **Centralized authorization:** IT and security teams should centrally assign, monitor, and revoke permissions. - **Low operational overhead:** The system should minimize ongoing SRE and security-team toil. ## Technologies Used The architecture relies primarily on managed services and infrastructure-as-code: - **SAML:** Exchanges identity, group membership, and role assertions between services. - **AWS Application Load Balancer:** Acts as a managed reverse proxy and routes authenticated HTTP/HTTPS traffic. - **AWS Cognito:** Provides user-pool functionality and integrates with federated SAML identity providers. - **AWS Lambda:** Runs code in response to configured events without managing servers. - **Terraform:** Defines and automates AWS and Okta configuration through reusable modules. ## Application Load Balancers, Cognito, and Okta Figma uses AWS for infrastructure and Okta for employee authentication and authorization. - ALBs can authenticate traffic using OIDC, but Okta charges extra for OIDC support. - Figma instead uses the ALB’s SAML authentication capability together with an AWS Cognito user pool. - Terraform modules automate the creation and configuration of the required ALB and Cognito resources. - These modules allow infrastructure engineers to quickly deploy internal applications protected by the company’s Okta environment. - The Cognito user pool disables self-registration. - Each pool connects to a Cognito identity provider backed by a SAML Okta application created for the specific internal application. - Attribute mappings are configured so user information such as `email` and `profile` is passed through the authentication flow.

datadog

Scaling support with Vagrant and Terraform | Datadog (opens in new tab)

The provided content does not include the blog post itself. It consists primarily of Datadog’s navigation menu and a promotional banner announcing its Gartner recognition, while the linked page suggests an article about scaling support with Vagrant and Terraform. ### Visible Content - Datadog announces that it was named a **Leader in the Gartner Magic Quadrant for Observability Platforms**. - The page navigation lists products across: - Infrastructure and application monitoring - Logs, security, and digital experience - CI/CD, service management, and AI - The URL references an engineering post titled **“Scaling Support With Vagrant and Terraform,”** but no article text is present. A meaningful technical summary requires the article body or a complete extract of the post.

datadog

Scaling support with Vagrant and Terraform (opens in new tab)

Datadog’s Solutions Team uses reproducible virtual environments to investigate customer issues across diverse operating systems, kernels, and integrations. Vagrant simplifies local VM creation, while provisioning scripts eliminate repeated installation and configuration work. Terraform extends the same approach to shared AWS environments, enabling teams to provision, preserve, and collaborate on sandboxes quickly. ## Reproducing Customer Environments with Vagrant - Containers are useful, but virtual machines are better when reproducing specific operating systems, kernels, orchestrators, or complex infrastructure. - Vagrant provides a simple workflow: - `vagrant init` - `vagrant up` - `vagrant ssh` - The main challenge is not creating a VM, but installing and configuring the technologies needed to match a customer’s environment. - With more than 200 integrations, engineers cannot be experts in every technology they may need to troubleshoot. ## Standardizing Setup with Provisioning Scripts - Vagrant provisioning supports tools such as Chef, Puppet, Ansible, and ordinary shell scripts. - Datadog stores reusable reproduction environments in a shared GitHub repository. - Each sandbox includes: - A `Vagrantfile` - A `setup.sh` provisioning script - A `data` directory for configuration files and supporting scripts - A `README.md` with usage information - Engineer-specific values, such as hostnames and tags, are kept in a local `.sandbox.conf.sh` file. - Once a sandbox exists, an engineer can run `vagrant up` and begin reproducing the customer issue within minutes. - The directory hierarchy organizes sandboxes by operating system, version or provider, and technology—for example, Ubuntu Xenial with Kafka. ## Sharing Remote Environments with Terraform - Terraform provides similar infrastructure management for remote cloud instances, including AWS EC2. - The team reuses the same `setup.sh` and `data` files for both Vagrant and Terraform, avoiding duplicate configuration work. - Each sandbox adds a `.tf` file that: - Creates an EC2 instance - Copies required data files - Executes the provisioning script remotely - A shared Terraform module handles common infrastructure tasks, while a `tf.example` file helps engineers create new configurations. - This preserves the same repository structure and workflow while extending sandboxes from local VMs to remote environments. ## Benefits for Team Collaboration - Remote sandboxes can remain available without consuming engineers’ local RAM. - Proper network security allows teammates to access and share environments. - Engineers can reproduce previously configured integrations during live customer interactions. - Investigations can continue across time zones, allowing teams to hand off urgent issues without rebuilding the environment. The overall recommendation is to treat reproduction environments as reusable infrastructure: encode installation and configuration steps once, store them in version control, and use Vagrant for local testing and Terraform for persistent, shared cloud sandboxes.

datadog

Improving cloud security visibility with ChatOps (opens in new tab)

Datadog built a largely serverless AWS security monitoring pipeline to detect suspicious API activity across more than 15 AWS accounts. Rather than process every CloudTrail event equally, it categorizes actions as log, notify, or alert, reducing false positives while preserving broad visibility. The system centralizes events, batches activity, and uses automated workflows to verify engineers’ actions or escalate potential compromises. ## The Security Monitoring Challenge - Datadog manages extensive AWS infrastructure across multiple accounts and nearly 200 geographically distributed engineers. - Every console or CLI operation generates an AWS API call, creating a high-volume CloudTrail data stream. - The monitoring system must detect malicious activity while also catching accidental exposure caused by configuration mistakes. - Processing every event manually would require an expensive, constantly staffed Security Operations Center. ## Prioritizing Relevant API Calls - Datadog maintains a focused list of security-relevant AWS API calls and assigns each to one of three categories: - **Log:** Lower-risk events retained for investigation, such as `CreateGroup` or `UpdateUser`. - **Notify:** Events that require the initiating engineer to confirm their identity and intent, such as `CreateUser` or `PutUserPolicy`. - **Alert:** Rare, dangerous, or clearly misconfigured actions sent directly to the security team. - A representative alert is `AuthorizeSecurityGroupIngress` with `0.0.0.0/0`, which exposes an EC2 security group to the entire Internet. - User verification reduces false positives and helps identify compromised AWS credentials. ## Cross-Account Event Pipeline - CloudTrail records API activity in each AWS account. - CloudWatch Event Rules filter for the selected API calls and publish matching events to SNS. - SNS forwards events across accounts to an SQS queue in a dedicated security AWS account. - Centralization is necessary because CloudWatch could not directly send events cross-account to SQS. - The queue also supports batching, which is important when Terraform generates many AWS changes in a short period. - A CloudWatch rule triggers a Lambda function every two minutes to drain the SQS queue and forward events to the security orchestration layer. ## Automated Decision-Making with Komand - Datadog uses Komand, a security orchestration and automation platform, to construct workflows from built-in and custom plugins. - A custom decision plugin evaluates: - The calling user - The event’s age - Request parameters and their content - Other contextual details - Based on the analysis, the workflow silently logs the event, notifies the engineer, or pages the security team through PagerDuty. ## Engineer Verification and Escalation - For notification-level events, the engineer receives an interactive Slack message containing API call details. - Confirming the action triggers a Duo push for second-factor identity verification. - If the engineer denies the action or fails to respond promptly, the workflow alerts the security team. - Komand coordinates the Slack, Duo, PagerDuty, and custom integration logic in one centralized workflow. ## Visibility and Continuous Improvement - Every workflow execution is logged and sent to Elasticsearch. - The resulting data helps Datadog visualize security events, measure detection effectiveness, identify behavioral trends, and improve alerting. - The pipeline is designed to provide actionable security intelligence without overwhelming engineers or security personnel. Datadog’s approach combines selective event filtering, cross-account centralization, batching, and automated identity verification. Organizations facing similar AWS-scale monitoring challenges can use the same principles to reduce alert fatigue while maintaining strong detection and response capabilities.