Software Automation | Techlist.io

gitlabJul 16, 20262 min readCurated summary

Turn multi-step software delivery into agentic flows you can trust

GitLab Duo Agent Platform’s Custom Flows are now generally available, enabling teams to automate multi-step software delivery rather than relying on chat responses, scripts, or manual runbooks. Flows can start from GitLab events or Agentic Chat, execute tasks across projects, and include human approval checkpoints. The goal is dependable, repeatable automation for work such as implementing issues, repairing pipelines, and reviewing merge requests. ## Why Software Delivery Remains Manual - Real development tasks involve multiple steps: gathering context, changing code, opening a merge request, waiting for CI, and responding to reviews. - Without workflows, people must manually coordinate each handoff and remember undocumented team practices. - Teams previously hesitated to formalize these processes because Custom Flows lacked production readiness and sufficient event coverage. ## What Agentic Flows Provide - Automate trusted sequences triggered by events such as: - Mentions and assignments - Pipeline activity - Merge request lifecycle changes - Work item updates - Run under a composite identity, keeping permissions bounded and actions attributable. - Launch specialist workflows from Agentic Chat, including: - Developer Flow for implementing changes - Code Review Flow for reviewing merge requests - Fix CI/CD Pipeline Flow for diagnosing and repairing failures - Require user approval before handing work from chat to a flow, while displaying progress inline. - Use review exclusions to avoid automatically reviewing bot-authored or branch-pattern merge requests. - Apply custom review instructions so automated reviews match team standards. ## Custom Flow Configuration - Create flows from a project or the AI Catalog. - Set visibility, enable flows for selected projects, and attach GitLab event triggers. - Add human-in-the-loop checkpoints for sensitive operations. - GitLab 19.2 adds: - A “work item status changed” trigger - Bulk enablement of public flows across up to 100 projects - A future Flow Creation Agent is planned to generate runnable flow definitions from plain-language descriptions. ## Costs and Adoption - Event-driven flows consume GitLab credits according to the work they perform. - GitLab recommends testing flows in a small number of projects before broad deployment. - Access is available through a GitLab Duo Agent Platform trial or through included credits for Premium and Ultimate subscribers. Teams should begin by encoding a few well-understood delivery processes as Custom Flows, adding approval gates where necessary, and expanding gradually while monitoring credit usage and automation quality.

Read original

Software Automation AI Agents CI/CD Gitlab Duo Agent Platform+3

spotifyFeb 19, 20263 min readCurated summary

Background Coding Agents: Predictable Results Through Strong Feedback Loops (Honk, Part 3) | Spotify Engineering

Spotify argues that unsupervised coding agents become reliable only when surrounded by strong, automated feedback loops. Its “Honk” system uses component-specific verifiers, mandatory pre-PR checks, and an LLM judge to catch build failures, test failures, scope creep, and functionally incorrect changes. The conclusion is that constrained, sandboxed agents with rich verification are more predictable than flexible agents operating independently. ## Failure Modes at Scale - Agents may fail to produce a pull request, which is inconvenient but usually manageable. - They may produce PRs that fail CI, leaving engineers to repair incomplete work. - Most seriously, they may produce PRs that pass CI but are functionally wrong and potentially reach production. - These failures are more likely when components lack tests, agents modify code beyond the prompt, or agents cannot correctly run builds and tests. - Reviewing invalid or nonsensical PRs can become a significant engineering time sink. ## Verification Loops - Honk uses independent verifiers that provide incremental feedback while the agent works. - Verifiers activate automatically based on the repository contents; for example, a Maven verifier runs when a root-level `pom.xml` is present. - The agent sees an abstract MCP tool rather than the implementation details of Maven, test runners, or build systems. - Verifiers handle formatting, compilation, testing, and output parsing, returning concise error messages instead of consuming the agent’s context with raw logs. - All applicable verifiers run before a PR is opened. In Claude Code, this is enforced with a stop hook. - If verification fails, the PR is blocked and the user receives an error. ## An LLM as a Judge - Deterministic checks cannot detect every problem, especially when an agent makes unnecessary refactors or disables flaky tests. - Honk therefore sends the original prompt and proposed diff to a separate LLM judge. - The judge runs after the regular verifiers and can veto changes that exceed the requested scope. - Across thousands of sessions, the judge rejects roughly one quarter of proposed changes. - Agents successfully correct about half of the vetoed changes. - Spotify has not yet built formal evaluations for the judge, but observed that scope violations are its most common reason for rejection. ## Constrained Agents and Sandboxing - The agent has limited responsibilities: inspect the relevant code, edit files, and invoke verification tools. - Surrounding infrastructure handles prompt creation, pushing code, and user communication through systems such as Slack. - Restricting the agent’s capabilities improves predictability and provides security benefits. - Agents run in heavily sandboxed containers with limited permissions, few installed binaries, and almost no access to surrounding systems. - Spotify reports that agents solve increasingly complex tasks reliably when these feedback loops are present, but often produce unusable code without them. ## Future Expansion - Spotify plans to support more hardware and operating systems. - Current verifiers run only on Linux x86, limiting support for systems that require macOS, such as iOS applications, or ARM64 environments. - The company also intends to integrate Honk more deeply with existing CI/CD pipelines. The practical recommendation is to treat autonomous coding as an infrastructure and verification problem, not merely a prompting problem: keep agents narrowly scoped, isolate them securely, and require layered automated checks before accepting their changes.

Read original

Software Automation Large Language Models AI Agents CI/CD+4

githubFeb 5, 20263 min readCurated summary

Continuous AI in practice: What developers can automate today with agentic CI

Continuous AI extends CI into software-engineering tasks that require judgment, context, and interpretation rather than deterministic rules. It uses continuously running agents guided by natural-language instructions to review repositories, identify issues, and produce reviewable artifacts such as patches, issues, or reports. GitHub’s central argument is that AI should complement—not replace—traditional CI, while operating within explicit permissions and developer oversight. ## Why CI Isn’t Enough - CI is effective for binary, rule-based checks: - Tests pass or fail. - Builds succeed or fail. - Linters detect defined violations. - Many important engineering tasks depend on intent and context, including: - Finding discrepancies between documentation and implementation. - Detecting confusing accessibility text that passes linting. - Identifying behavioral changes caused by dependency updates. - Spotting subtle performance regressions, such as compiling a regular expression inside a loop. - Recognizing UI regressions that only appear during interaction. - GitHub describes this as a shift from AI-generated code toward AI handling cognitively demanding maintenance work. ## What Continuous AI Means - Continuous AI is a pattern, not a replacement for CI: - **Natural-language rules + agentic reasoning, executed continuously inside a repository.** - Developers describe expectations in natural language, especially when those expectations are difficult to encode with schemas, heuristics, or YAML. - Example workflows include: - Comparing documented behavior with implementation and proposing fixes. - Producing weekly reports on project activity, bug trends, and code churn. - Detecting performance regressions in critical paths. - Finding semantic regressions in user flows. - Workflows are refined collaboratively with agents by adding intent, constraints, and acceptable outputs rather than being authored as a perfect single instruction. ## Guardrails and Safe Outputs - Agents operate with read-only repository access by default. - They cannot modify content, create issues, or open pull requests unless explicitly authorized. - “Safe Outputs” defines the exact artifacts an agent may produce and the constraints governing them. - Agent activity is sanitized, logged, and auditable. - The goal is to keep the potential impact predictable even when agents make mistakes or behave unexpectedly. ## Natural Language Complements YAML - Deterministic problems should remain in CI, using YAML, schemas, tests, and heuristics. - Some expectations—such as determining whether documentation and code still express the same behavior—require semantic understanding. - Natural-language instructions let agents reason about intent without forcing that intent into brittle rules. - Continuous AI therefore expands automation into judgment-heavy tasks while preserving CI as the foundation for deterministic validation. ## Developers Remain in the Loop - Agents do not make unrestricted autonomous commits. - Depending on permissions, they can produce pull requests, issues, comments, discussions, or other reviewable artifacts. - Pull requests are especially useful because they fit existing developer review and collaboration practices. - The broader vision is to delegate recurring maintenance work while allowing developers to retain judgment, taste, and final control. Continuous AI is best adopted alongside traditional CI: use conventional automation wherever rules are sufficient, and use guarded, continuously running agents for tasks involving interpretation, synthesis, and evolving intent.

Read original

Software Automation AI Agents CI/CD Github+3