Orchestration | Techlist.io

githubJun 12, 20263 min readCurated summary

How we made GitHub Copilot CLI more selective about delegation

GitHub improved Copilot CLI by making subagent delegation more selective rather than treating delegation as inherently beneficial. The new orchestration policy keeps narrow tasks with the main agent, delegates broad or independent work, and encourages parallel execution instead of waiting. After full production rollout, it reduced tool failures by 23% and improved high-percentile wait times without reducing quality. ## The Cost of Unnecessary Delegation - Subagents help with complex investigations, large repositories, and parallel work, but every handoff adds tool calls, coordination, and latency. - Copilot sometimes delegated simple, well-scoped tasks that the main agent could complete directly. - Common problems included: - Repeated or overlapping repository searches. - Subagents re-discovering context already available to the main agent. - Sequential delegation that left the main agent idle. - Stale paths, incorrect relative paths, and workspace mismatches. - The result was slower execution and more tool failures for tasks that should have required only a few steps. ## How the Problem Was Identified - GitHub used LLMs to analyze complete agent trajectories rather than manually reviewing sessions. - The analysis found that delegation was frequently used for narrow, obvious, or fully described tasks. - This led to a clear target: - Keep focused discovery-and-edit work with the main agent. - Reserve subagents for broad exploration, cross-cutting tasks, or genuinely independent work. ## A More Selective Orchestration Policy - Copilot now starts with the narrowest effective workflow: - Find and read the relevant file. - Make the targeted change. - Verify the result. - Delegation becomes appropriate when additional context, uncertainty, or parallel execution creates real value. - Subagents are treated as a parallelism mechanism, not a reason for the main agent to pause. - Handoffs should clearly specify: - The user’s request. - What the main agent already knows. - Which work the subagent owns. - What result the subagent should return. ## Evaluation and Production Results - GitHub tested the change with generated regression cases and existing benchmarks before rollout. - Staff and public A/B tests measured reliability, responsiveness, subagent workload, and quality. - Production results showed: - 23% fewer tool failures per session. - 27% fewer search-tool failures. - 18% fewer edit-tool failures. - 5% lower P95 wait time. - 3% lower P75 wait time. - No quality regression. - The improvements came mainly from avoiding unnecessary subagent paths and reducing orchestration overhead, not from making individual model calls faster. Copilot CLI users can access the improvement by running `/update` and upgrading to version 1.0.42 or later. The broader recommendation is to delegate selectively: use the main agent for focused tasks and subagents only when independent context or parallel work provides meaningful leverage.

Read original

Orchestration Large Language Models Github Copilot Cli Ab Testing+2

kakaoApr 21, 20263 min readCurated summary

Kanana Scala 1st Seminar On-site Sketch

Kakao’s first Kanana Scholar seminar brought together seven leading AI professors and Kakao researchers to discuss the company’s independent AI strategy. Kakao presented its from-scratch Kanana foundation models, emphasizing data efficiency, Korean-language capability, and multimodal processing. The discussion concluded that Kakao should focus less on generic benchmark scores and more on technology sovereignty, personalized agents, and practical execution in real services. ## Kanana Foundation Models - Kakao is developing its own foundation-model lineup to strengthen competitiveness and reduce dependence on overseas providers. - Kanana reportedly achieved strong performance using 11 trillion training tokens, compared with 23 trillion tokens for a similarly sized global-target model. - Kakao attributed this efficiency to the quality and refinement of its training data. - The company also demonstrated **Kanana-o**, an omni model capable of processing text, images, and audio in real time. - The model handled emotional speech and multi-speaker conversations naturally, receiving praise for its Korean fluency. ## Technology Sovereignty and Customization - Kakao argued that proprietary models protect it from external risks such as changing licensing policies and closed technologies. - Owning the technology enables Kakao to build efficient, customized models optimized for its services. - Participating professors agreed that control over Korean cultural context and local issues is essential for technological sovereignty. - They viewed an independent model as a strategic asset for long-term service stability. ## Digital World Models and Personalized Agents - Kakao aims to understand users’ behavioral context within KakaoTalk and provide highly personalized assistance. - On-device AI could protect private conversations while allowing agents to respond immediately to user needs. - The professors suggested expanding the idea of “physical AI” into a **digital world model** that predicts interactions and causal relationships across a platform. - This direction could create an area of AI differentiation uniquely suited to Kakao’s ecosystem. ## Evaluating Practical Agentic Intelligence - Kakao is prioritizing AI systems that can create multi-step plans, call necessary tools, and complete tasks independently. - It plans to use an internally developed orchestration benchmark to evaluate real-world problem-solving ability. - The professors cited Claude as an example of how users perceive intelligence through successful completion of complex requests, not merely high benchmark scores. - They recommended competing through practical execution in real service environments rather than focusing only on text-generation performance. ## Industry-Academic Cooperation - Kakao plans to explore GPU support for university research labs and undergraduate AI clubs. - Possible support could include credits, project-based resources, and other forms of infrastructure assistance. - The seminar marked the beginning of a broader collaboration aimed at advancing Korea’s AI ecosystem and developing future talent. Kakao’s recommended path is to combine proprietary, efficient models with privacy-preserving personalization and strong agentic execution. Success will depend on how effectively Kanana turns technical depth into useful intelligence that users can experience in everyday services.

Read original

Orchestration AI AI Agents Multimodal AI+4

lineApr 10, 20261 min readCurated summary

List of Articles for ‘Orchestration Development Workshop,’ an Internal Workshop to Enhance AI Utilization Skills

LY Corporation runs the “Orchestration Development Workshop” to help engineers apply AI more effectively in real development work. The workshop focuses on connecting multiple AI tools and agents to amplify creativity through collaborative learning and creation. A related blog series will share workshop topics, beginning with an example of using AI to reduce PR review delays and improve review culture. ## Orchestration Development Workshop - Targets engineers involved in development at LY Corporation. - Emphasizes practical, workplace-oriented AI skills rather than purely theoretical knowledge. - Uses “orchestrating multiple AIs to maximize creativity” as its central theme. - Builds on the idea that organizational learning is essential for successfully adopting AI. ## Blog Series - The series will publish workshop content incrementally. - The article list will be updated as new posts are released. - The listed update date is April 10, 2026. ## AI-Assisted PR Reviews - The first installment addresses bottlenecks and delays in pull-request reviews. - It explains how AI-assisted PR review support can help resolve review stagnation. - It also describes an internal workshop designed to change team review practices and culture. The series presents AI adoption as an organizational and collaborative practice, with PR review automation serving as an initial example of measurable workflow improvement.

Read original

Orchestration AI Team Collaboration Workshop+2

lineApr 8, 20263 min readCurated summary

The Key to AI Utilization Lies in 'Organizational Learning' - The Start of the Orchestration Development Workshop

LY Corporation is moving from simply adopting AI tools to building with AI as a collaborative development partner. Its new Orchestration Development Workshop teaches engineers to coordinate multiple AI systems across coding, testing, reviews, incident analysis, and other workflows. The initiative aims not only to improve efficiency but to free engineers from repetitive work so they can focus on more creative, high-value challenges. ## From AI Adoption to AI Collaboration - AI-assisted development and operations are spreading rapidly across LY Corporation. - Engineers use generative AI for code generation and testing, while combining it with non-generative AI for analysis and operational optimization. - Despite broader adoption, employees differ significantly in how deeply they use AI in their daily work. - The workshop was created to help the organization evolve from “using AI” to “creating alongside AI.” ## Orchestration: Coordinating Multiple AI Systems - “Orchestration” refers to combining multiple AIs, along with human input, to produce a complete outcome. - Example workflows include: - Generating code automatically from a Jira ticket. - Having AI run tests, conduct reviews, and create a pull request. - Analyzing a Slack incident report, estimating the cause, and proposing a fix. - The workshop turns these emerging practices into hands-on learning rather than passive demonstrations. ## A Hands-On, Interactive Learning Model - Participants follow instructors in real time and perform the same tasks themselves. - Zoom conversations and Slack questions create two-way communication during the session. - Instructors and representative participants explore solutions to problems as they arise. - The goal is for attendees to gain skills they can reproduce in their own projects, not merely acquire theoretical knowledge. ## Organization-Wide Support Through Guilds and DevRel - The initiative is designed to avoid depending on individual enthusiasm. - Three complementary functions support continuous growth: - **DevRel:** Drives the program and promotes adoption. - **Guilds:** Contribute practical insights from engineering teams. - **TD:** Helps maintain quality and reproducibility. - This structure supports consistent content quality and enables AI knowledge to spread across the company. ## Beyond Efficiency: Unlocking Engineering Creativity - LY Corporation views AI as more than a way to complete tasks faster. - By delegating repetitive work to AI, engineers can spend more time on creative and strategically valuable activities. - The organization aims to move beyond a model where AI writes code and humans only review it. - Instead, engineers should collaborate with AI from the design stage through implementation. ## Future Direction - LY Corporation plans to share lessons from the workshops through external channels such as its technology blog. - Future topics will include both generative and non-generative AI. - The broader goal is to provide practical guidance for engineers building new workflows with AI. The workshop represents a structured way to turn AI experimentation into repeatable organizational practice, helping engineers coordinate multiple AI tools while preserving human creativity and judgment.

Read original

Orchestration AI Database Design PostgreSQL+4

githubMar 10, 20262 min readCurated summary

The era of “AI as text” is over. Execution is the new interface.

The post argues that AI is moving beyond text-based question-and-answer interactions toward embedded execution. The GitHub Copilot SDK lets applications use Copilot’s planning, tool use, file modification, command execution, and error recovery capabilities directly. This enables teams to build adaptable AI workflows without creating their own orchestration infrastructure. ## Delegating Multi-Step Work to Agents - Applications can express intent and constraints instead of hard-coding every workflow step. - For a task such as “Prepare this repository for release,” an agent can: - Explore the repository - Plan the necessary work - Modify files - Run commands - Recover and adapt when failures occur - This approach is more flexible than scripts, which become brittle when workflows depend on changing context or unexpected errors. - Teams can use agentic execution while maintaining defined boundaries and observability. ## Using Structured Runtime Context - Relying on prompts to contain system logic makes workflows difficult to test, maintain, and evolve. - The Copilot SDK supports structured, composable context through: - Domain-specific tools and agent skills - Model Context Protocol (MCP) - Runtime retrieval of relevant data - Agents can directly access systems such as: - Service ownership records - Historical decisions - Dependency graphs - Internal APIs - Permissioned tools and real-time data provide more reliable grounding than embedding organizational knowledge in prompts. ## Embedding Agents Beyond the IDE - Agentic capabilities can be integrated into: - Desktop applications - Internal operational tools - Background services - SaaS products - Event-driven systems - Applications can invoke Copilot in response to events such as file changes, deployments, or user actions. - Execution happens within the product itself rather than in a separate IDE or terminal interface. - This turns AI from an auxiliary developer tool into application infrastructure available wherever the software operates. ## Execution as a New Interface - Agentic workflows are programmable planning-and-execution loops that: - Integrate with real systems - Operate under constraints - Adapt during runtime - Use tools to complete tasks - The Copilot SDK provides this execution layer so teams can focus on defining outcomes instead of rebuilding orchestration systems. The practical recommendation is to treat AI as an executable application capability rather than merely a text interface. Teams can start by identifying multi-step workflows or event-driven tasks where structured tools, runtime context, and adaptive execution would provide more value than fixed scripts.

Read original

Orchestration AI Agents Model Context Protocol Github Copilot+2

spotifyFeb 9, 20263 min readCurated summary

Our Multi-Agent Architecture for Smarter Advertising | Spotify Engineering

The post argues that fragmented advertising workflows, not backend infrastructure, are the core problem. Although buying channels share services and data, their planning and optimization logic is repeatedly reimplemented across channels and surfaces, causing drift and technical debt. The proposed solution is a shared agentic decision layer that interprets advertiser goals, orchestrates existing Ads APIs, and applies consistent reasoning across products. ## Fragmented Workflows Across a Shared Backend - Direct, Self-Serve, and Programmatic buying use largely consolidated infrastructure but retain different workflows and decision logic. - Spotify Ads Manager, Salesforce, Slack, and internal tools contain overlapping automation. - Budget allocation, inventory selection, reach, efficiency, and STR decisions are repeatedly implemented in different places. - Incremental workflow changes therefore create duplicated maintenance work and inconsistent behavior. ## Why Conventional Workflow Services Fall Short - Hard-coded state machines and REST services are poorly suited to combinatorial planning tasks. - Campaign planning depends on: - User and advertiser characteristics - Available inventory and audiences - Business priorities - Forecasts, performance, and optimization goals - A workflow optimized for one channel or “happy path” will not adapt well as requirements change. - Improvements to decision logic must be replicated across every product surface, increasing the risk of divergence. ## The Missing Intent Layer - Existing systems can perform individual actions such as creating line items, running forecasts, and retrieving insights. - They do not consistently translate high-level objectives into: - A sequence of tool calls - Explicit tradeoffs - Validation and safety checks - An objective such as maximizing reach in Brazil while protecting video inventory and meeting STR requires coordinated reasoning across multiple capabilities. ## A Modular Agentic Architecture - Campaign planning and management are modeled as cooperating specialized agents. - Agents use shared signals, including: - Inventory - Audiences - STR - Quality and risk - Historical performance - They jointly optimize advertiser goals and Spotify’s business constraints. - Existing Ads services become tools that agents orchestrate, rather than capabilities being rebuilt in each workflow. - A long-running orchestration layer delegates tasks while agents share context and evaluation logic. - The same decision engine can support every buying channel and surface. ## Engineering Implications - APIs need to be designed as agent tools, rather than only as CRUD interfaces. - Testing must include behavioral evaluation in addition to unit and integration tests. - Observability should explain what an agent decided and why, not merely track latency and errors. - Safety requires guardrails for semi-autonomous decisions, beyond ordinary input validation. - The approach avoids both duplicated deterministic workflows and a brittle, centralized rules engine for probabilistic, ML-heavy advertising logic. The overall recommendation is to centralize campaign decision-making in a reusable agentic platform while keeping existing services as specialized tools. This should reduce duplicated workflow logic, make improvements consistent across products, and allow advertising workflows to evolve without repeatedly rebuilding them.

Read original

Orchestration AI Agents Machine Learning API Design+3

netflixDec 16, 2025Original article

How Temporal Powers Reliable Cloud Operations at Netflix | by Netflix Technology Blog | Dec, 2025 | Netflix TechBlog (opens in new tab)

Netflix has significantly enhanced the reliability of its global continuous delivery platform, Spinnaker, by adopting Temporal for durable execution of cloud operations. By migrating away from a fragile, polling-based orchestration model between its internal services, the engineering team successfully reduced transient deployment failures from 4% to a remarkable 0.0001%. This shift has allowed developers to write complex, long-running operational logic as standard code while the underlying platform handles state persistence and fault recovery. ### Limitations of Legacy Orchestration * **The Polling Bottleneck:** Originally, Netflix's orchestration engine (Orca) communicated with its cloud interface (Clouddriver) via a synchronous POST request followed by continuous polling of a GET endpoint to track task status. * **State Fragility:** Clouddriver utilized an internal orchestration engine that relied on in-memory state or volatile Redis storage, meaning if a Clouddriver instance crashed mid-operation, the deployment state was often lost, leading to "zombie" tasks or failed deployments. * **Manual Error Handling:** Developers had to manually implement complex retry logic, exponential backoffs, and state checkpointing for every cloud operation, which was both error-prone and difficult to maintain. ### Transitioning to Durable Execution with Temporal * **Abstraction of Failures:** Temporal provides a "Durable Execution" platform where the state of a workflow—including local variables and thread stacks—is automatically persisted. This allows code to run "as if failures don’t exist," as the system can resume exactly where it left off after a process crash or network interruption. * **Workflows and Activities:** Netflix re-architected cloud operations into Temporal Workflows (orchestration logic) and Activities (idempotent units of work like calling an AWS API). This separation ensures that the orchestration logic remains deterministic while external side effects are handled reliably. * **Eliminating Polling:** By using Temporal’s signaling and long-running execution capabilities, Netflix moved away from the heavy overhead of thousands of services polling for status updates, replacing them with a push-based, event-driven model. ### Impact on Cloud Operations * **Dramatic Reliability Gains:** The most significant outcome was the near-elimination of transient failures, moving from a 4% failure rate to 0.0001%, ensuring that critical updates to the Open Connect CDN and Live streaming infrastructure are executed with high confidence. * **Developer Productivity:** Using Temporal’s SDKs, Netflix engineers can now write standard Java or Go code to define complex deployment strategies (like canary releases or blue-green deployments) without building custom state machines or management layers. * **Operational Visibility:** Temporal provides a native UI and history audit log for every workflow, giving operators deep visibility into exactly which step of a deployment failed and why, along with the ability to retry specific failed steps manually if necessary. For organizations managing complex, distributed cloud infrastructure, adopting a durable execution framework like Temporal is highly recommended. It moves the burden of state management and fault tolerance from the application layer to the platform, allowing engineers to focus on business logic rather than the mechanics of distributed systems failure.

Orchestration CI/CD Microservices Durable Execution+4

naverDec 3, 2025Original article

Naver TV (opens in new tab)

Naver’s VLOps framework introduces an event-driven approach to MLOps, designed to overcome the rigidity of traditional pipeline-based systems like Kubeflow. By shifting from a monolithic pipeline structure to a system governed by autonomous sensors and typed messages, Naver has achieved a highly decoupled and scalable environment for multimodal AI development. This architecture allows for seamless functional expansion and cross-cloud compatibility, ultimately simplifying the transition from model training to large-scale evaluation and deployment. ### Event-Driven MLOps Architecture * Operations such as training, evaluation, and deployment are defined as "Typed Messages," which serve as the primary units of communication within the system. * An "Event Sensor" acts as the core logic hub, autonomously detecting these messages and triggering the corresponding tasks without requiring a predefined, end-to-end pipeline. * The system eliminates the need for complex version management of entire pipelines, as new features can be integrated simply by adding new message types. * This approach ensures loose coupling between evaluation and deployment systems, facilitating easier maintenance and infrastructure flexibility. ### Omni-Evaluator and Unified Benchmarking * The Omni-Evaluator serves as a centralized platform that integrates various evaluation engines and benchmarks into a single workflow. * It supports real-time monitoring of model performance, allowing researchers to track progress during the training and validation phases. * The system is designed specifically to handle the complexities of Multimodal LLMs, providing a standardized environment for diverse testing scenarios. * User-driven triggers are supported, enabling developers to initiate specific evaluation cycles manually when necessary. ### VLOps Dashboard and User Experience * The VLOps Dashboard acts as a central hub where users can manage the entire ML lifecycle without needing deep knowledge of the underlying orchestration logic. * Users can trigger complex pipelines simply by issuing a message, abstracting the technical difficulties of cloud infrastructure. * The dashboard provides a visual interface for monitoring events, message flows, and evaluation results, improving overall transparency for data scientists and researchers. For organizations managing large-scale multimodal models, moving toward an event-driven architecture is highly recommended. This model reduces the overhead of maintaining rigid pipelines and allows engineering teams to focus on model quality rather than infrastructure orchestration.

Orchestration Multimodal AI Mlops Llm Evaluation+4

awsDec 2, 2025Original article

Build multi-step applications and AI workflows with AWS Lambda durable functions (opens in new tab)

AWS Lambda durable functions introduce a simplified way to manage complex, long-running workflows directly within the standard Lambda experience. By utilizing a checkpoint and replay mechanism, developers can now write sequential code for multi-step processes that automatically handle state management and retries without the need for external orchestration services. This feature significantly reduces the cost of long-running tasks by allowing functions to suspend execution for up to one year without incurring compute charges during idle periods. ### Durable Execution Mechanism * The system uses a "durable execution" model based on checkpointing and replay to maintain state across function restarts. * When a function is interrupted or resumes from a pause, Lambda re-executes the handler from the beginning but skips already-completed operations by referencing saved checkpoints. * This architecture ensures that business logic remains resilient to failures and can survive execution environment recycles. * The execution state can be maintained for extended periods, supporting workflows that require human intervention or long-duration external processes. ### Programming Primitives and SDK * The feature requires the inclusion of a new open-source durable execution SDK in the function code. * **Steps:** The `context.step()` method defines specific blocks of logic that the system checkpoints and automatically retries upon failure. * **Wait:** The `context.wait()` primitive allows the function to terminate and release compute resources while waiting for a specified duration, resuming only when the time elapses. * **Callbacks:** Developers can use `create_callback()` to pause execution until an external event, such as an API response or a manual approval, is received. * **Advanced Control:** The SDK includes `wait_for_condition()` for polling external statuses and `parallel()` or `map()` operations for managing concurrent execution paths. ### Configuration and Setup * Durable execution must be enabled at the time of the Lambda function's creation; it cannot be retroactively enabled for existing functions. * Once enabled, the function maintains the same event handler structure and service integrations as a standard Lambda function. * The environment is specifically optimized for high-reliability use cases like payment processing, AI agent orchestration, and complex order management. AWS Lambda durable functions represent a major shift for developers who need the power of stateful orchestration but prefer to keep their logic within a single code-based environment. It is highly recommended for building AI workflows and multi-step business processes where state persistence and cost-efficiency are critical requirements.

Orchestration Serverless Aws Lambda State Management+3