How we used AI agents to migrate GitLab rate limiting
GitLab used a three-person engineering pod and AI agents to migrate 121 application-level rate-limit keys into a shared `labkit-ruby` implementation. The migration succeeded because humans retained ownership of architecture, scope, rollout decisions, and final review while agents handled mechanical coding, tests, and reviews. The main lesson was that disciplined workflows and meaningful observability mattered more than the agents themselves. ## Migration Setup - GitLab was consolidating two production rate-limiting systems: - `Gitlab::ApplicationRateLimiter`, with 121 keys - A separate Rack-level implementation - The target was a single observable, testable, and consistently operated implementation in `labkit-ruby`. - A three-person pod divided responsibilities across the monolith, the gem, architecture, and project scope. - AI agents: - Read project context - Drafted specifications - Implemented bounded changes - Wrote tests - Pre-reviewed merge requests - Humans controlled scope, architecture, rollout strategy, and final approvals. ## The Specification and Review Loop - The team followed a repeatable process: - Read the epic - Write a specification - Conduct adversarial review - Implement only after blockers were resolved - Verify with explicit evidence - Review the merge request adversarially - Escalate to human review - Merge - Adversarial review was limited to two resolution rounds before requiring human involvement. - The project produced 14 numbered specifications and more than 30 merge requests. - This structured loop made agents useful on legacy code without allowing them to make high-impact decisions independently. ## Successful Rollouts - The first cohort covered five heavily used keys, including: - `pipelines_create` - `notes_create` - `user_sign_in` - Rollout progressed from 1% to 10%, 50%, and finally 100% over two days. - Engineers compared the old and new implementations during rollout and deliberately generated traffic to test behavior above the configured limits. - The second cohort consolidated 95 call sites: - 83 in the monolith - 12 in Enterprise Edition - Agents were especially effective at this repetitive, large-scale codebase work, avoiding roughly 95 individual feature-flag changes and 190 YAML edits. ## Observability and Shadow-Mode Failure - During Cohort 2, an adapter dropped an identifier on an unauthenticated path by incorrectly packing three strings into two primitive slots. - Some users briefly received generic failures when enforcement began. - Shadow comparison had detected divergence, but the dashboards did not distinguish structural identifier collisions from ordinary disagreements. - The team disabled enforcement immediately and shipped a short-term fix two days later. - The deeper cleanup will replace array-based scopes with named characteristics when calling `ApplicationLimiter`. - The incident showed that having observability is insufficient if it cannot identify the failure modes that require action. ## Missed Rate Limits and Infrastructure Constraints - An audit revealed that the original five-cohort plan had missed 17 of the 121 keys. - The omissions included: - Enterprise-only limits - Registry entries - Webhook keys - `partner_*` sub-second limits - Orphaned adapter rows - The team had not maintained a complete inventory count, making it possible for keys to become effectively invisible. - A sixth cohort was added to cover the missed cases. - Redis capacity also became a constraint: - The rate-limiting service used a four-shard cluster. - `maxclients` was increased incrementally. - Rollout stopped at 75,000 connections rather than 100,000 because primary CPU usage approached saturation. - Redis command execution was limited by one core per primary, leaving no simple vertical scaling solution. ## How AI Changed the Work - Agents made code generation faster, shifting the bottleneck to: - Human review capacity - Rollout judgment - Operational monitoring - Reviewer and operator attention - Agent collaboration was not always efficient; engineers sometimes spent longer guiding agents than they would have spent coding directly. - Engineers also had to develop new skills for specifying, reviewing, and correcting agent-generated work. - Agents could execute a request mechanically—such as creating dozens of feature flags—but could not decide whether that design was appropriate. - Human judgment remained essential for simplifying the rollout and avoiding unnecessary per-key flags. ## Outcome - By mid-June, all six cohorts had reached 100%. - All 121 application rate-limit keys were running through the new framework. - The migration demonstrated that AI agents can safely support complex legacy-system changes when paired with bounded tasks, adversarial review, gradual rollouts, complete inventories, and failure-specific observability. A practical recommendation is to use agents for repetitive implementation and verification, but keep architecture, risk assessment, rollout control, and operational decisions firmly with experienced humans.
Read original(opens in new tab)
