software-migration

2 posts

gitlab

Forrester Consulting: GitLab Duo Agent Platform delivers 400% ROI (opens in new tab)

GitLab-commissioned Forrester research found that organizations using GitLab Duo Agent Platform could achieve a 400% three-year ROI, $7.5 million in net present value, and payback in under six months. The study argues that agentic coding creates the greatest business value when integrated across the software lifecycle—not merely used to generate code. Benefits included faster onboarding, shorter migrations, quicker security remediation, and more developer time for feature work. ## Study Scope and Financial Model - Forrester interviewed four organizations across financial services, software, entertainment, and insurance. - Their experiences were modeled as a composite global company with: - $3 billion in annual revenue - 3,000 employees - GitLab Duo Agent Platform adoption growing from 150 to 250 users - Three-year risk-adjusted costs totaled approximately $1.9 million: - $1.3 million in consumption credits - $589,000 for implementation, training, support, and internal labor - Quantified benefits reached $9.4 million, producing: - 400% ROI - $7.5 million net present value - Payback in less than six months ## Problems Before Adoption - Manual processes and ad-hoc knowledge sharing slowed development. - New developers depended heavily on senior engineers for context and troubleshooting. - Security fixes waited in queues for specialists with the necessary expertise. - Code review was often a larger bottleneck than writing code. - Senior engineers were repeatedly interrupted to unblock other team members. ## Quantified Benefits - **80% faster developer onboarding** - Agentic chat in IDEs and repositories helped new hires understand unfamiliar codebases independently. - Estimated savings: $582,000. - **75% shorter migration timeline** - A migration from on-premises GitLab to GitLab SaaS finished in two months instead of eight. - Agents helped diagnose pipeline failures and resolve issues during the migration. - Estimated labor savings: $157,000. - **40% more time for security and QA engineers** - Contextual explanations and suggested fixes reduced remediation effort and reliance on senior staff. - Estimated three-year savings: $1.3 million. - **20% more developer capacity for feature work** - Agents supported code review, testing, and troubleshooting. - Estimated combined benefit: $7.4 million. - Additional unquantified benefits included reduced need for overlapping AI tools, improved developer satisfaction, and stronger knowledge sharing. ## Broader Impact on Software Delivery The study found that organizations shipped features in days rather than weeks, resolved vulnerabilities more quickly, onboarded staff faster, and compressed major infrastructure work. Its central conclusion is that AI productivity gains compound when agentic coding is connected to infrastructure supporting the entire software delivery lifecycle. The findings are based on interviewed organizations and a composite financial model, so actual results will vary. Companies evaluating agentic development should use the study as a framework for estimating benefits across productivity, security, onboarding, and operational efficiency—not as a guaranteed ROI.

microsoft

The Interaction Changes Everything: Treating AI Agents as Collaborators, Not Automation (opens in new tab)

The article argues that effective AI agents should be treated as engineering collaborators, not automation scripts. Microsoft applied this approach to migrate hundreds of repositories from Entra SDK v1 to v2, reducing work from 4–6 weeks per repository to under two hours with 80–90% accuracy. The key improvement came from giving the agent a role, mission, priorities, and permission to exercise judgment. ## The Entra SDK Migration Challenge - The migration involved hundreds of repositories and sensitive authentication security boundaries. - Traditional migrations required extensive human review and took 4–6 weeks per repository. - The AI agent completed comparable work in under two hours while achieving 80–90% accuracy. - The goal was not merely speed, but reliable handling of custom configurations, edge cases, and security concerns. ## Problems with Automation Thinking - Initial attempts treated the agent like a script executor: - Detailed transformation instructions were provided. - Every anticipated edge case was documented. - The agent was expected to follow a checklist. - This approach repeatedly failed because complex migrations require: - Context-dependent decisions - Handling of undocumented patterns - Security-boundary evaluation - Trade-offs between correctness, speed, and preservation of custom logic - The central lesson is that judgment cannot be fully automated, but it can be supported through collaboration with an intelligent agent. ## Identity Instead of Instructions - The team reframed the agent as a member of the migration team rather than a tool. - The prompt described the agent as a “co-creative engineer” expected to: - Use judgment - Stay curious - Act carefully - Ask for help when uncertain - This change improved accuracy and edge-case handling. - The agent was more likely to surface uncertainty instead of guessing or failing silently. ## The Co-Creative Partnership Framework ### Identity and Mission - Establish the agent’s team, mission, and the importance of the work. - Explicitly state that the agent is not a script executor. - Explain why the task matters so the agent can prioritize appropriately. - Encourage judgment, curiosity, and care. ### Purpose and Intent - Describe the guide as supporting both human and AI team members. - Make priorities explicit, such as security over speed or correctness over completion. - Allow autonomy when repository contexts differ. - Frame uncertainty as a reason to collaborate rather than as failure. ### Prioritized Goals - List primary, secondary, quality, and human-in-the-loop objectives in order. - Explicit priorities help the agent resolve conflicts. - Including quality and collaboration prevents optimizing for speed alone. ### Step-by-Step Guidance with Judgment - Provide concrete actions, conditional logic, edge-case handling, and before-and-after examples. - Specify what must remain unchanged, including custom logic. - Define situations requiring escalation, such as unusual patterns, ambiguity, or possible security violations. - The framework combines procedural guidance with room for context-sensitive decisions. ## Practical Recommendation For complex migrations, security reviews, or architectural work, write prompts that define a collaborative role and decision-making framework—not just a list of commands. Give the agent context, priorities, preservation rules, examples, and clear escalation points so it can act autonomously while knowing when human judgment is required.