From Automation to AI with Infrastructure as Code (IaC): Adopting OpenTofu and ChatOps
LY Corporation’s LINE Plus SRE team migrated Verda cloud infrastructure and IMON monitoring resources from scattered, manual management into an Infrastructure as Code (IaC) and GitOps workflow. Using OpenTofu and Terragrunt, they now manage roughly 1,500 resources across seven services through pull requests, CI/CD, and daily drift detection. The migration required careful automation for importing existing resources, normalizing state, and handling provider limitations and resource dependencies. ## Why IaC Was Needed - Teams previously managed infrastructure through different methods: - Verda’s web dashboard - Scripts - Wiki-based procedures - Personal documents and GitHub repositories - As the number of services and resources grew, this caused: - Inconsistent management practices - Difficult-to-track configuration changes - Greater risk of manual errors - Poor reproducibility and reviewability - The team adopted GitOps so that: - Desired infrastructure state is declared in Git. - All changes go through pull requests. - Infrastructure history is versioned and auditable. - CI/CD applies approved changes automatically. - Their goal was to manage infrastructure with the same engineering standards as application code: reviewable, version-controlled, and reproducible. ## Choosing OpenTofu and Terragrunt - OpenTofu was selected as an open-source Terraform fork. - It retains: - Terraform’s HCL syntax - Provider compatibility - Familiar module and configuration patterns - The team created reusable modules for: - Virtual machines - Load balancers - Monitoring alerts - Modules were versioned so updates could be adopted explicitly rather than affecting every environment immediately. - Terragrunt was added to reduce repetition in environment configuration. - Shared settings are defined once in a parent `root.hcl`. - Individual environments contain only their differing inputs. - OpenTofu reduces duplication in resource definitions, while Terragrunt reduces duplication in environment and backend configuration. ## Planning the Migration - The most difficult part of introducing IaC into an existing environment was importing resources that were already running. - Manual import was considered impractical for hundreds of VMs, load balancers, and DNS records because it would be slow and error-prone. - The migration was split into two phases: - **Phase one:** Automate imports, select one service for a pilot, and establish the complete OpenTofu/Terragrunt pipeline. - **Phase two:** Reuse the validated modules and import scripts to roll the approach out to the remaining services. ## Designing the Import Process - Import scripts were designed to: - Query existing resources - Decide which resources should be managed by IaC - Convert resource data into the desired code structure - Generate Terragrunt configuration - Connect resources to OpenTofu state - Run `plan` to verify that no unintended changes would occur - A key requirement was keeping three representations aligned: - Configuration code - OpenTofu state - Actual cloud resources - Normalization was added because equivalent values could be represented differently—for example, network or image IDs—causing OpenTofu to report misleading differences after import. ## Resource-Specific Import Strategies - Resources could not all be imported using the same procedure. - Different resource types have different identifiers, dependencies, and ownership models: - **VMs:** Imported individually, while distinguishing manually created instances from Kubernetes-managed instances. - **Load balancers:** Imported together with related listeners and pools. - **DNS:** Imported while preserving zone and record relationships. - **Kubernetes:** Structured around clusters and node pools. - **IMON:** Imported according to its hierarchy of teams, alert groups, alert rules, and monitors. - Each resource followed the same broad five-step process, but its implementation was adapted to the resource’s characteristics. ## Problems Discovered During Migration ### Kubernetes-Managed VMs - OpenStack contained both manually created VMs and VMs automatically created by Kubernetes. - Importing Kubernetes-managed VMs into IaC could cause conflicts between OpenTofu and Kubernetes. - The scripts excluded these VMs using naming patterns and metadata. ### IMON’s Hierarchical Structure - IMON alerts are organized as: `Team → Alert Group → Alert Rule → Monitor` - A flat import would lose these relationships. - The team mirrored the hierarchy in the directory structure so ownership and relationships were visible from the file layout. ### Provider and Regional Identifier Issues - The actual cloud platform allowed both hyphens and underscores in load balancer names, but the provider validation logic rejected underscores. - The team fixed this by modifying the provider’s validation logic and contributing the change upstream. - Resource UUIDs such as `flavor_id`, `image_id`, and `network_id` differed by region. - This produced unnecessary changes in `plan`. - The modules added regional mapping logic, allowing users to specify readable names while resolving them to region-specific IDs. The migration demonstrates that successful IaC adoption requires more than writing configuration files: existing infrastructure must be filtered, normalized, modeled according to its dependencies, and validated against real provider behavior. OpenTofu and Terragrunt provided the foundation for scalable GitOps management, while custom import automation and provider improvements made the transition safe for production resources.
Read original(opens in new tab)