line·Aug 7, 2026·4 min readCurated summary

What Is the Next Step in Personal AI Use? Conditions for Introducing an AIDD Organization Explored Through an AIDD Workshop at LY Corporation

LY Corporation argues that AI-driven development (AIDD) must evolve beyond individual experimentation into a repeatable organizational practice. AIDD integrates AI across requirements, design, implementation, and review, with AI producing drafts while people provide context, make decisions, and maintain accountability. Its workshop showed that successful adoption depends less on distributing tools than on preparing shared context, workflows, responsibilities, and decision-making structures. ## Defining AIDD - AIDD uses AI as a collaborator throughout the development lifecycle, from requirements clarification through code review. - It is neither fully delegating development to AI nor using AI as an isolated productivity assistant. - The intended workflow is: - AI creates an initial draft. - People provide intent, constraints, and judgment. - Results are reviewed and carried into subsequent development stages. - The central challenge is designing how people and AI work together across the entire process. ## Why LY Corporation Held the Workshop - Individual use of AI coding agents has become common for: - Code completion - Research - Testing - Documentation - Organizational adoption often stalls because: - Individual usage is not connected to team workflows. - Review standards for AI output are unclear. - Teams are unsure how to apply AI to existing products. - Successful experiments remain personal know-how. - “Convenience” does not translate into investment or adoption decisions. - The workshop aimed to move teams from personal AI usage toward organization-wide “AI Ready” conditions. - It involved 21 teams and 112 participants, including LINE Plus, who brought real work topics for evaluation. ## Why Participation Was Team-Based - AI creates value through workflow design, not just prompt-writing skill. - Teams must decide: - What information AI receives - Where human review occurs - Which output becomes the official deliverable - How feedback enters the existing process - Engineers alone cannot resolve these questions. Product, planning, design, leadership, and decision-makers contribute essential perspectives. - Team participation exposed hidden disagreements about consensus, ownership, review responsibilities, and decision boundaries. ## Workshop Structure - The two-day program combined learning with practical validation using real team projects. - Day one focused on: - Defining problems - Organizing requirements and context - Clarifying assumptions and priorities - Day two focused on autonomous experimentation and producing workflows applicable to actual work. - Orchestration Guild members, Developer Relations, and Technical Directors provided mentoring and helped make the learning reproducible across the company. - Informal conversations during breaks and meals also helped reveal issues and next steps that formal meetings often miss. ## Four Major Lessons ### The Greatest Value Often Comes Before Implementation - Teams initially focused on how quickly AI could write code. - In practice, the more important benefits came earlier in the process: - Breaking vague requirements into concrete issues - Defining requirements in clear language - Aligning team understanding - Identifying which decisions must come first - Turning decisions into manageable work units - AI can accelerate progress, but people must establish the problem definition and make critical judgments. ### Context, Not Tools, Is the Main Bottleneck - AI output quality depends heavily on the quality of its context. - Important context includes: - Specifications - Terminology - Constraints - Design intent - Relationships to existing code - Operational rules - Without this information, AI may generate plausible but impractical results, increasing review effort. - Organizing context must therefore be treated as core infrastructure for AI adoption, not optional preparation. ### Team Participation Reveals Organizational Issues - Individual experiments rarely expose the full set of coordination problems. - Working on a shared topic helps teams determine: - Where AI should be used - Who reviews its output - Which artifacts are authoritative - How AI-assisted work fits into existing processes - Collaboration across business, planning, design, engineering, and leadership makes implicit knowledge and conflicting assumptions visible. ### Decision-Maker Involvement Improves Follow-Through - Teams with leaders or decision-makers were more likely to turn workshop outcomes into concrete actions. - Organizational adoption requires decisions about: - Which areas to start with - Where to invest time - What to standardize - How deeply AI should be embedded into operations - Leadership participation prevents the workshop from ending as an interesting experiment and helps connect it to implementation. ## Conditions for Successful Adoption - Start with manageable topics, such as: - Requirements or issue clarification - Work requiring stakeholder alignment - Projects with accessible existing information - Small efforts where one complete cycle can be tested - Create lightweight entry points, such as applying AI to one feature, one requirements document, or one review checklist. - Make context preparation an official responsibility: - Document specifications, terminology, constraints, design intent, and decision rationale. - Allocate team and organizational time for this work rather than relying on individual goodwill. - Treat context organization as a long-term engineering asset that improves development even beyond AI use. The practical recommendation is to adopt AIDD incrementally through real team projects, while simultaneously improving shared context, review processes, role definitions, and leadership involvement. The goal is not merely to use more powerful tools, but to redesign the development system so AI-assisted work can be repeated and sustained across the organization.

Read original(opens in new tab)

Software DevelopmentCode ReviewSoftware DesignContext Engineering+4

line·Jul 31, 2026·5 min readCurated summary

Analyzing Incident Causes with Natural Language in Grafana: Developing an LLM Agent-Based SRELens

SRELens is a Grafana-based natural-language observability assistant created by LY Corporation’s Home SRE team. It connects metrics, logs, traces, and profiles so engineers can investigate incidents without switching between tools or manually transferring context. The project’s central conclusion is that production reliability depends less on natural-language querying itself and more on controlling the LLM’s tools, prompts, permissions, cost, and failure behavior through backend code and policy. ## The Observability Analysis Problem - Incident investigation traditionally requires moving among: - Grafana or IMON for metrics - LaaS or IU for logs - IMON Trace or Tempo for traces - A separate profiling system - Engineers must manually connect: - Error-rate increases - Error messages - Trace IDs and slow requests - Relevant time ranges, services, and labels - This context switching is especially costly during outages. - The team first consolidated data with a self-hosted LGTM-P stack: - Mimir for metrics - Loki for logs - Tempo for traces - Pyroscope for profiles - OpenTelemetry Collector as the ingestion layer - Centralizing the data helped, but engineers still needed to know the correct datasource, labels, query syntax, and relationships between signals. ## Why an Existing Open-Source PoC Was Not Enough The team initially evaluated an open-source Grafana LLM plugin, but identified several production limitations: - It could not reliably propagate Grafana-authenticated user context for chat history, permissions, and usage limits. - System prompts could not be controlled strongly enough to enforce organizational policies. - Short tool-call limits interrupted multi-step investigations. - Datasource-specific naming differences often produced empty results: - Metrics might use `service_name` - Tempo might require `resource.service.name` - Loki might require JSON parsing or structured metadata filters - Modifying and deploying the solution internally raised operational and licensing concerns. The PoC showed that the key requirement was not merely asking questions in natural language, but retaining control over how the agent operates. ## SRELens Architecture - SRELens runs as a Grafana application plugin. - The frontend provides the chat interface. - The backend handles: - LLM requests - Tool orchestration - Prompt composition - Usage and quota enforcement - Observability queries are executed through an MCP gateway. - A `CompositeClient` combines: - Upstream FlavaMCP observability tools - Local Grafana tools such as `find_grafana_panel` and `render_grafana_panel` - The backend is an orchestration and policy layer, not just a proxy. ## Three-Layer System Prompt Design ### Base System Prompt Defines organization-wide behavior and safety rules, including: - Tool-call ordering - Safe handling of dashboard creation, modification, and deletion - Fallback behavior for empty results - Re-querying with aggregation when results are truncated - Response structure and evidence requirements Only administrators can change this layer. ### Datasource Fragment Encodes environment-specific operational knowledge in YAML: - Preferred Mimir, Loki, and Tempo datasource UIDs - Candidate service-name labels - Loki parsing and filtering rules This prevents the agent from wasting tool-call rounds discovering basic datasource conventions. ### User Prompt Stores personal or team-specific context in Redis, such as: - Owned services - Preferred response formats - Frequently used dashboards User preferences are added as context but cannot override organizational safety policies. ## Backend Tool Orchestration and Guardrails The backend exclusively assembles system prompts and runs the agent loop: 1. Send the user’s question to the LLM. 2. Execute requested MCP or local tools. 3. Return tool results to the LLM. 4. Repeat until a final answer is produced. Safety and reliability controls include: - A default maximum of 10 tool-call rounds - Duplicate-call prevention using call hashes - A default retry limit of two attempts per tool - Per-tool result-size limits - Trimming older tool results when the request history becomes too large - Preserving `tool_call_id` relationships when trimming history - Hints that encourage changing labels, time ranges, or datasources after empty results These safeguards reduce dependence on the LLM making perfect decisions. ## Usage Limits and Degraded Operation - Per-user daily token quotas - Per-user requests-per-minute limits - HTTP 429 responses after limits are exceeded - Post-response accounting based on actual prompt and completion tokens returned by OpenAI - Daily quota reset at midnight in the Asia/Seoul timezone - Redis stores conversation history, user prompts, and quotas. - If Redis is unavailable, personalization and history are reduced, but a single chat request can still proceed. ## Incident Analysis Scenario In one beta service, SRELens was asked to investigate an error spike between 09:50 and 10:05. - Instead of separately searching alerts, logs, and traces, the agent examined the relevant dashboard and observability data together. - It narrowed the incident to a surge in `CopyMedia` requests. - The analysis was intended to connect the request pattern with the underlying errors and supporting telemetry, demonstrating how SRELens can move from an aggregate error spike toward a specific API-level cause. SRELens demonstrates that an LLM can accelerate incident analysis when it is grounded in an integrated observability stack and constrained by explicit backend policies. For production use, organizations should treat prompt control, tool orchestration, permissions, quotas, retries, and failure handling as core system components rather than leaving them entirely to the model.

Read original(opens in new tab)

ObservabilityModel Context ProtocolRedisGrafana+3

line·Jul 30, 2026·4 min readCurated summary

Android CLI for AI Agents: Applying It to Large-Scale Mobile Development Environments

LINE’s Android app is a large monorepo with hundreds of Gradle modules and developers, making unrestricted AI-agent searches expensive and unreliable. Generic tools such as `grep` and `glob` often return excessive, semantically weak results, causing agents to waste tokens and retry. The team therefore built thin wrappers, skills, and prompts around Android CLI to provide efficient documentation lookup and Android Studio’s semantic capabilities across multiple agents. ## Why Generic Search Breaks Down at Scale - Large repositories can return huge numbers of search results from a single request. - Search output consumes agent context and increases costs. - Text search cannot reliably answer semantic questions such as: - Where a symbol is declared or used - Whether a file contains IDE-detectable problems - Whether code is unused - As the number of modules grows, agents are more likely to rely on irrelevant results and repeat searches. ## Replacing MCP Documentation Search with Android CLI - The team first adopted Android CLI for official documentation search. - It provides current documentation for Android, Jetpack Compose, AndroidX, Firebase, and related technologies. - This helps reduce hallucinations caused by outdated pretrained knowledge. - Android CLI’s `docs` commands are exposed through the `get-android-dev-knowledge` skill: - `docs search` finds relevant documentation. - `docs fetch` retrieves the document body from its Knowledge Base URL. - Compared with the previous Google Cloud Knowledge MCP setup, Android CLI eliminates: - Per-developer Google Cloud authentication - An authentication proxy - Quota-management and workaround logic - The result is fresher documentation with fewer tokens and less supporting infrastructure. ## Bundling the Android CLI Binary The team stores the Android CLI binary in the repository and invokes it from a fixed path such as `.agents/tools/android-cli/android`. - **Consistent environments** - Developers, CI systems, and agent hosts use the same pinned version. - Installation differences in version, path, and platform are reduced. - **Security enforcement** - The wrapper automatically adds `--no-metrics`. - This prevents agents from accidentally omitting the company-required telemetry setting. - A fixed binary location makes reliable wrapper enforcement possible. - **Manageable repository cost** - Existing use of Git LFS makes storing the binary relatively inexpensive. ## Handling the Android CLI Metrics Bug - Android CLI 1.0 initializes metrics tracking before honoring `--no-metrics`. - It may still attempt to write under `~/.android/cli`. - In restricted sandboxes, this causes a multi-page Java stack trace, wasting agent context. - The wrapper now probes write access before invoking the CLI: - It creates `~/.android/cli`. - It attempts to create a temporary probe file. - If writing is blocked, it emits a concise, parseable error explaining the required permission. - This converts a noisy failure into an actionable one-line message. ## Android Studio Integration Android CLI 1.0 added integration with running Android Studio instances, enabling IDE-level semantic operations from the command line. - `studio check` - Verifies that Android Studio is running. - Confirms that the target project is open and indexing is complete. - `analyze-file` - Runs IDE inspections on a single file without a build. - Detects semantic issues such as unused code. - `find-declaration` - Locates symbol declarations in the project and inside `.aar` or `.jar` dependencies. - `find-usages` - Finds references to a symbol. - `render-compose-preview` - Renders Compose `@Preview` functions as PNG images. ## Wrapping Studio Features as Skills - The team does not expose raw Android CLI behavior directly to agents. - Each capability is wrapped in a lightweight script and presented as an agent skill. - The first skill created was `studio-check`. - This follows the same design used for documentation search and ensures failures are concise, predictable, and easier for agents to interpret. ## Practical Recommendation For large Android repositories, use Android CLI behind repository-pinned wrappers and agent skills rather than exposing generic search or raw CLI commands directly. Enforce security flags, validate filesystem prerequisites early, and prefer IDE-backed semantic operations when agents need declarations, usages, inspections, or Compose previews.

Read original(opens in new tab)

Model Context ProtocolAndroidGradleAi Coding Agents+3

line·Jul 16, 2026·3 min readCurated summary

Developing a Model to Assess Harmfulness from Open Chat Names and Descriptions

The AI Services Lab developed a model to automatically detect harmful LINE OpenChat names and descriptions, reducing the need for manual review. The project improved an existing moderation system by cleaning inconsistent labels, selecting a lightweight safety-tuned decoder model, and adapting it to predict both penalty levels and reasons. Granite Guardian 3.1 2B was ultimately fine-tuned with LoRA and deployed using token-probability-based inference. ## OpenChat Monitoring - Users must provide an OpenChat name and may add a description. - Names and descriptions are reviewed whenever they are created or modified. - LINE processes a large volume of global OpenChats, making fully manual moderation impractical. - The project aimed to: - Expand automated moderation to countries requiring more detailed judgments. - Improve accuracy in regions already using automation. - Reduce the amount of content requiring human review. ## Data Cleansing - Training data consisted of previously manually reviewed OpenChat names and descriptions. - Only records reviewed under the current moderation guidelines were used. - Identical name-description pairs sometimes had conflicting penalty outcomes. - Labels were consolidated using these rules: - Select the most severe penalty if it appeared at least twice. - If it appeared only once, treat it as possible noise and select the second-most-severe penalty. - Choose the most frequent penalty reason. - If reasons were tied, choose the globally rarer reason, following a TF-IDF-like principle that rarer reasons may be more specific. - This process produced a single, consistent label for each identical input. ## Selecting the Pretrained Model The team evaluated models according to four requirements: - Decoder-based architecture. - Fine-tuned for safety moderation. - Approximately 2 billion parameters. - Apache license for commercial use. Granite Guardian 3.1 2B was selected because: - It is designed to classify harmfulness through the probabilities of “Yes” and “No” tokens. - Restricting predictions to predefined tokens avoids unpredictable free-form responses. - Token probabilities provide confidence scores that can be thresholded for operational needs. - Its relatively small size supports lower serving costs and faster responses. ## Fine-Tuning for Penalty Prediction - A simple harmful/not-harmful classification was insufficient because moderation decisions include different penalty levels and reasons. - The model was trained to produce structured responses containing: - An `Action` penalty code. - A `Reason` penalty reason. - Cross-entropy loss was calculated only over the assistant’s response tokens, not the entire prompt. - This focuses training on predicting moderation decisions rather than reproducing the input text. - LoRA was used instead of full-parameter fine-tuning: - The base model parameters remained frozen. - Only small trainable matrices representing parameter updates were optimized. - This reduced memory and training costs while preserving pretrained capabilities. ## Inference Design - During inference, the model calculates logits for all possible next tokens. - The system extracts only the logits corresponding to valid penalty-code tokens, converts them to probabilities, and selects the highest-scoring code. - It then predicts the penalty reason in a second step. - Existing operational codes consisted of arbitrary letters and numbers that tokenized into multiple pieces. - To simplify probability calculations, penalty codes and reasons were mapped to meaningful natural-language tokens, each represented by a single tokenizer token. - KV caching was used between the penalty-code and penalty-reason predictions to improve efficiency. The resulting approach combines cleaned moderation labels, lightweight decoder-model fine-tuning, structured output targets, and constrained token-level inference. It is intended to broaden automated OpenChat moderation while maintaining the accuracy and response speed required for real-time LINE operations.

Read original(opens in new tab)

TransformerPytorchLoraTokenization+4

line·Jul 10, 2026·4 min readCurated summary

One Million Events per Second: Implementing End-to-End Encryption with Apache Kafka in the LINE App

LINE handles billions of messages daily, including highly sensitive personal data. While Kafka already provides TLS, authentication, and authorization, those controls do not protect message contents stored in brokers from privileged access. LY Corporation therefore introduced Kafka client-to-client end-to-end encryption, keeping payloads encrypted from producers through consumers while supporting large-scale traffic, flexible consumers, and minimal overhead. ## Limits of Kafka’s Existing Security Model - TLS protects data in transit between clients and brokers. - SASL authenticates clients before they connect. - ACLs control which users or groups can publish to or consume from topics. - These mechanisms primarily control access and communication channels; broker-stored payloads may still exist in plaintext. - End-to-end encryption adds a defense-in-depth layer by encrypting data at production and decrypting it only at authorized consumers. ## Record-Level Encryption - LY Corporation chose record-level rather than batch-level encryption. - Batch encryption offers better compression and lower CPU overhead, but would require modifying Kafka client internals because standard extension points operate at the record level. - Record encryption works with Kafka interceptors, serializers, and deserializers without modifying existing Kafka clients. - Using standard APIs also improves compatibility with future Kafka upgrades, despite somewhat larger messages and reduced compression efficiency. ## DEK–KEK Key Architecture - Payloads are encrypted with a symmetric AES-GCM data encryption key (DEK). - The DEK is encrypted with an ECC-based key encryption key (KEK), using ECIES and the `secp521r1` curve. - KEKs are managed through a key management service (KMS). - Producers use the KEK’s public key, while authorized consumers obtain the private key from KMS. - This hybrid approach: - Avoids the high cost of encrypting large payloads with asymmetric cryptography. - Keeps message size effectively independent of the number of consumers. - Separates encryption and decryption permissions according to the least-privilege principle. ## Encrypted Kafka Message Structure - **Key:** The existing Kafka message key remains unchanged for partitioning. - **Header:** Contains the KEK identifier and the DEK encrypted with that KEK. - **Body:** Contains the payload encrypted with the DEK. - Embedding metadata directly in each message avoids dependencies on external databases or caches. - Consumers identify the appropriate KEK, decrypt the DEK, and then decrypt the payload. ## Producer and Consumer Architecture ### Producer Encryption - Interceptors generate or select the DEK and place the encrypted DEK in the message header. - A wrapper serializer encrypts the serialized payload with the DEK. - The interceptor and serializer share the DEK through `ThreadLocal`, since they run on the same thread. - DEKs are cached for a limited period rather than regenerated and re-encrypted for every message, reducing asymmetric cryptographic overhead. ### Consumer Decryption - Consumers retrieve authorized private KEKs from KMS. - The deserializer reads the encrypted DEK from the header, decrypts it with the private KEK, and decrypts the payload. - Consumers cache encrypted-DEK/plain-DEK pairs, allowing repeated messages from the same producer to bypass redundant DEK decryption. - The existing deserialization process is wrapped so decryption occurs before normal deserialization. ### KMS Operations - Topic owners generate and register KEK key pairs. - Producers retrieve public keys, while authorized consumers retrieve private keys. - New consumers must request access to the private key and receive approval from the topic owner. - KMS manages key distribution, access control, and key rotation. ## Scaling Optimizations ### Shared KEKs - Assigning a unique KEK to every consumer would cause message headers to grow with the consumer count. - This would reduce Kafka batch sizes and increase network, CPU, and memory usage, especially for topics reaching up to one million messages per second. - Multiple consumers therefore share a single KEK, keeping the header size constant. - The trade-off is reduced per-consumer key isolation, mitigated through: - KMS authorization controls. - Mandatory periodic key rotation. - Centralized key management by the topic owner. ### Zero-Downtime Migration - During migration, encrypted and plaintext messages must coexist. - The consumer deserializer checks whether encryption metadata exists: - If headers are present, it decrypts the message. - If headers are absent, it processes the message using the existing plaintext path. - The migration sequence is: - Deploy compatible consumers first. - Enable producer encryption after all consumers support both formats. - Monitor the plaintext-message ratio and complete the migration once it reaches zero. - Producer encryption is intended to be enabled progressively rather than switched to 100% immediately, reducing the risk of unexpected performance or cryptographic failures. ## Practical Conclusion Kafka’s built-in security controls should be supplemented with payload-level encryption when brokers handle highly sensitive data. A record-level AES-GCM design combined with DEK–KEK key wrapping, KMS authorization, caching, shared KEKs, fallback processing, and gradual rollout provides a practical balance between confidentiality, scalability, and operational continuity.

Read original(opens in new tab)

Apache KafkaEnd To End EncryptionZero Downtime MigrationKey Management Service+3

line·Jul 2, 2026·3 min readCurated summary

Building a Group Video Calling Service Inside a LINE App with AI, Without a Web Engineer

LINE’s LIFF enables web services to run directly inside the LINE app, turning a LINE Official Account into an interactive service platform rather than merely a notification channel. The LINE Planet team demonstrated this by building a group video-calling service with only a PM and an Android engineer, without a web engineer. The core architecture combines a LIFF web app, a small token-issuing app server, and LINE’s managed authentication and WebRTC infrastructure. ## Services Enabled by LINE OA and LIFF - **Professional consultations:** One-to-one video sessions with lawyers, financial planners, counselors, and other experts. - **Remote education:** Scheduled video lessons with separate rooms for multiple teachers and students, including screen sharing. - **Interactive live broadcasts:** Events for 500 to 10,000 simultaneous participants, including audience members joining conversations as panelists. - **In-game voice chat:** Real-time communication with LINE friends without switching to another app. ## Overall Architecture - **LIFF** provides the in-app web interface and automatically exposes LINE login information such as `userId` and `displayName`. - **LINE Planet** handles WebRTC media processing and global network infrastructure. - **The web app** uses the LINE Planet SDK to implement the group call experience. - **The app server** issues LINE Planet access tokens. - Firebase Cloud Functions can provide the app-server layer without managing separate server infrastructure. - The developers are responsible mainly for connecting these components; LINE handles authentication, media transport, and much of the underlying infrastructure. ## Required Preparation - Use Node.js 20 LTS or later with npm. - Deploy over HTTPS; local development can use ngrok. - Create a Business ID, developer account, and Provider in the LINE Developers Console. - Create a LINE Official Account and enable its Messaging API. - Create a LINE Login channel under the same Provider and register a LIFF app. - Record the LIFF ID because it is required for initialization. - Set the LINE Login channel to **Published** for the `shareTargetPicker` API, which supports inviting LINE friends. - Request a LINE Planet Console account and service ID from the LINE Planet team. ## Designing and Generating Room IDs - LIFF can collect call setup information and register it with the app server, reducing the amount of pre-call configuration. - Users can join simply by entering or following a room ID. - The example generates a random 16-character alphanumeric ID using `crypto.randomUUID()`. - If a `roomId` query parameter exists in an invitation link, the app restores and uses that room instead. - The same design can later support fixed rooms based on interests or automatically generated rooms for user groups. ## Building the Preview Screen - The preview screen lets users check their camera and microphone before entering a call. - Instead of directly calling `getUserMedia`, the example uses PlanetKit’s `MediaStreamManager`. - A single `MediaStreamManager` instance is reused from preview through the conference, avoiding repeated permission requests. - Camera input is created with `createMediaStream()` or replaced with `changeVideoInputDevice()`. - Microphone muting changes the audio track’s `enabled` flag, preventing another permission prompt in mobile webviews. - Mobile users can switch between front and rear cameras by resolving the appropriate device ID. - The sample UI includes camera and microphone toggles, camera-flip controls for mobile devices, and an “Enter” action. - The article notes that the sample focuses on the essential flow; production applications still need stronger security, error handling, and performance optimization. The practical recommendation is to treat LIFF and LINE Planet as managed building blocks: implement the web call interface and a minimal token server, while relying on LINE for user identity and PlanetKit for real-time media.

Read original(opens in new tab)

ReactTypeScriptNode.JsWebrtc+4

line·Jun 26, 2026·3 min readCurated summary

From Prompting to Workflows: Boosting Frontend Development Productivity with AI

Frontend development is increasingly shifting from a coding problem to an orchestration problem. Requirements, designs, documentation, discussions, and existing code are scattered across tools, while LLMs can now connect these sources through repeatable workflows. The article argues that structured, reviewable workflows—rather than clever one-off prompts—are the key to scaling AI-assisted development and improving implementation quality. ## From Prompting to Repeatable Workflows - A prompt may produce a useful result once, but it does not create a reusable process. - A workflow defines a repeatable path from inputs to outputs: - Collect context from Jira, Confluence, Slack, and the codebase. - Summarize the actual requirements. - Identify ambiguities and unresolved decisions. - Propose an implementation plan. - Wait for human review before modifying code. - The LLM acts as the engine executing the workflow. - LY Corporation’s Noah MCP connects systems such as Jira, Confluence, Slack, and GitHub, allowing AI agents to access real organizational context instead of relying on manually copied prompts. - Once established, the same workflow pattern can be applied across many tickets, even when the specific inputs differ. ## Example: Planning a List Page The example Jira ticket requests a list page with search, filtering, sorting, and role-based filter visibility. - In the traditional process, a developer manually: - Reads the Jira ticket and identifies missing details. - Searches Figma for loading, empty, and no-results states. - Finds role-based filter rules in Confluence. - Searches Slack for prior decisions. - Inspects the codebase for reusable hooks and components. - Copies findings into notes and assembles an implementation plan. - Implements the feature, resolves bugs and edge cases, and submits a PR. - An AI workflow performs these steps systematically before coding. - The generated plan identifies: - A new `FeatureListPage` route and `FeatureList` component. - Reuse of `useTableFilters` and `useUrlState`. - Existing API support through `GET /api/<feature>`. - URL synchronization for filters, sorting, and pagination. - Role-based visibility using `useCurrentUserRole()`. - Required loading, empty, and no-results states. ## Surfacing Hidden Requirements The workflow improves quality by exposing information that might otherwise appear late in development. - A Slack decision establishes that filter and sort state should use URL parameters rather than `localStorage`, enabling shareable and reloadable views. - Existing hooks such as `useTableFilters` and `useUrlState` are discovered before new code is written, preventing unnecessary duplication. - Unresolved questions are explicitly listed for human review, including: - Whether filter and sort state belongs in URL parameters or `localStorage`. - Which empty-state design should be used when Figma contains multiple variants. - Resolving these questions early reduces rework during implementation or PR review. ## Closed-Loop Verification The workflow should continue after coding rather than stopping when the first implementation is complete. - The agent compares the implementation with the original plan. - It runs: - Type checks. - Linting. - Related unit tests. - Relevant smoke tests or local verification flows. - It reports: - Successful checks. - Failures that were fixed. - Items that could not be verified automatically. - UI screenshots or state notes. - Remaining risks before opening a PR. - This creates a closed-loop development cycle in which AI not only writes code but also validates its work against the intended requirements. Teams should treat AI as a workflow and context-orchestration layer, not merely a code generator. The most effective process gathers information across systems, obtains human approval for the plan, implements with existing project patterns, and automatically verifies the result before review.

Read original(opens in new tab)

Large Language ModelsModel Context ProtocolReactTypeScript+4

line·Jun 26, 2026·4 min readCurated summary

Solving the Cold-Start Problem in Search Reranking Through Embedding Stabilization: A LINE Part Time Jobs Case Study

LY Corporation improved LINE Part Time Jobs’ real-time search reranking by stabilizing user and item embeddings produced by a two-tower recommendation model. The approach addressed both cold-start degradation and daily embedding-space drift without changing the underlying model or training pipeline. Offline and online evaluations showed substantial gains, including a 4.7% overall KPI increase and 6.5% revenue growth. ## Search Reranking at LINE Part Time Jobs - Search consists of: - Retrieval, which finds listings matching a query. - Reranking, which orders the retrieved candidates. - The previous system ranked listings by cosine similarity between precomputed user-to-item two-tower embeddings. - This approach was computationally simple and captured broad user preferences, but: - It ignored query-specific information, such as the distance from a selected station. - Its embeddings combined behavior from multiple services and recommendation modules, not just search activity. - The team therefore introduced a dedicated real-time reranking model. ## Challenges with the Dedicated Reranking Model ### Cold Start - Most job listings are replaced at the beginning of each month. - New listings initially lack sufficient interaction data. - As a result, reranking quality dropped until enough training data accumulated. ### Embedding-Space Drift - Two-tower models were regularly retrained from random initialization. - Each training run produced a different embedding space. - Using embeddings as downstream features caused a mismatch between training-time and inference-time data, reducing model performance. ## Stabilizing the Embedding Space - Each day’s embeddings are aligned with the previous day’s stabilized embeddings. - The first day’s embeddings are used without stabilization. - This preserves continuity across retraining cycles and allows embeddings generated on different days to remain comparable. - Downstream models and embedding generation no longer need perfectly synchronized update schedules. ### Low-Rank SVD - User and item embeddings are converted into a more standardized low-dimensional representation. - Instead of decomposing the enormous user-item score matrix directly, transformation matrices are derived from the embedding matrices. - This makes the procedure practical for large-scale data. ### Orthogonal Procrustes Alignment - The transformed embeddings are aligned to the previous day’s stabilized space. - The orthogonal transformation only rotates or reflects the space. - Distances and inner-product relationships are therefore largely preserved, maintaining the two-tower model’s scoring behavior. ## Scalable Implementation - The algorithm was implemented with Apache Spark to handle LINE Part Time Jobs’ large datasets. - For low-rank SVD: - The original QR decomposition was optimized using Cholesky decomposition. - The Gram matrix \(G=A^\top A\) is decomposed to obtain the same upper-triangular matrix \(R\) as QR decomposition. - For Procrustes alignment: - The large matrix multiplication \(M=B^\top A\) is distributed across Spark. - The resulting \(e \times e\) matrix is small enough for SVD on a single node using NumPy. ## Evaluation Results ### Embedding Stability - Before stabilization, embeddings from randomly selected days had correlations close to zero. - After stabilization: - Similarity remained around 0.88 after one week. - Similarity remained around 0.87 after one month. - This reduced performance loss caused by embedding drift. ### Offline Evaluation - Unstabilized embeddings reduced nDCG by approximately 1–5% when training and inference used different days. - Stabilized embeddings improved: - Conversion nDCG by about 9.0%. - Click nDCG by about 4.5%. ### Online A/B Test - Search-page KPIs alone did not show statistically significant improvement. - Across the entire service: - KPIs increased by 4.7%. - Revenue increased by 6.5%. - The results suggest that the embeddings captured long-term user preferences that influenced later actions across the service, not only behavior on the search page. - The added embedding features also helped mitigate the initial cold-start problem. ## Practical Benefits and Future Work - The solution required no changes to the two-tower model itself. - Stabilization was added as post-processing, minimizing changes to existing pipelines and reducing deployment risk. - LY Corporation plans to test the method as the service expands its sources of job listings and to reuse the approach across other services through its internal machine-learning platform. Overall, sequential low-rank SVD and orthogonal Procrustes alignment provide a relatively simple way to make frequently retrained embeddings reliable downstream features while improving real-time reranking and business outcomes.

Read original(opens in new tab)

Machine LearningApache SparkOnline LearningTwo Tower Embedding+4

line·Jun 25, 2026·4 min readCurated summary

What If AI Agents Debated Each Other? Redesigning the Development Process Through Multi-Agent Collaboration

AI coding’s main bottleneck is no longer code generation but the human coordination surrounding it: clarifying intent, validating assumptions, testing implementations, and preparing trustworthy pull requests. LY Corporation proposes an AI-native pipeline in which specialized “proposer” and “challenger” agents debate across three stages—specification, build, and delivery—while an orchestrator decides whether to revise, escalate, or proceed. The goal is for AI to substantiate its own work before human engineers review and approve it. ## Human Coordination as the Bottleneck - Traditional AI-assisted development speeds up individual tasks but leaves handoffs between requirements, implementation, verification, and review to humans. - Engineers still need to: - Write or refine specifications - Review AI-generated drafts - Transfer failed tests and feedback between steps - Inspect diffs - Prepare PR descriptions - Decide whether the result is trustworthy - The proposed solution is not to remove human judgment, but to automate repetitive coordination while preserving human ownership and final approval. ## Proposer–Challenger Collaboration - AI responsibilities are divided between two opposing groups: - **Proposers** develop specifications, implementations, and delivery materials. - **Challengers** validate them from specialized perspectives. - The separation prevents one general-purpose assistant from combining design, implementation, testing, and review into a single unchallenged response. - Specialized roles may include: - `requirements-synthesizer` - `security-analyst` - `test-coverage-reviewer` - `technical-writer` - `evidence-verifier` - An **orchestrator** mediates disagreements, redirects discussions, resolves deadlocks, and determines whether to revise, escalate, or advance. ## The Spec–Build–Deliver Pipeline ### Specification - The specification acts as a contract for all later stages. - It records: - Goals and constraints - Interpreted requirements - Explicit assumptions - Open questions - Proposed approach - Definition of done - Agents use evidence from the workspace and external sources such as Jira, Confluence, design documents, APIs, tests, dependencies, and existing conventions. - Ambiguous but low-risk and reversible issues can be documented as assumptions. - Unsafe, destructive, externally constrained, or hard-to-reverse uncertainties are escalated instead of guessed. ### Build - The approved specification is converted into a test-first verification plan before production code is changed. - The proposer identifies expected behavior, edge cases, required tests, and execution commands. - Challengers can dispute the verification design before or during implementation. - Proposers must support rejected objections with concrete evidence such as: - Execution paths - Compiler or linter output - Failing tests - Other workspace evidence - This prevents a simple green CI result from hiding missing or inadequate validation. ### Delivery - The final output is a review-ready PR package rather than merely a diff summary. - It explains: - What changed - Where reviewers should look first - Which checks passed - Remaining risks - Which challenges were already investigated - At this stage, the orchestrator acts more like a jury, judging whether sufficient evidence exists for release. ## Structured Debate Protocol - Each agent receives stage-specific context and returns structured JSON rather than a free-form essay. - Agents do not share one live context window. Shared state consists of: - Workspace files - Generated artifacts - The orchestrator’s accumulated transcript - Each round includes a proposer response, challenger response, and orchestrator decision. - The protocol distinguishes manageable uncertainty from blocking risk. - Consistent schemas make agent outputs easy to parse, compare, and feed into subsequent rounds. - For example, a challenger can identify an unclear scope boundary, explain why it matters, assign severity and confidence, and indicate whether user input is required. ## Overall Impact - Issues move through a continuous chain: debated specification, branch, tested implementation, and review-ready PR. - Humans intervene mainly to define intent, approve the final result, or resolve explicitly escalated decisions. - The central leverage comes not from generating code faster, but from requiring AI to explore, challenge, verify, and package its work before asking engineers to pay attention. The practical recommendation is to redesign AI development around explicit artifacts, specialized adversarial roles, evidence-based decisions, and automated handoffs. Human engineers should remain the final decision-makers, while AI handles the intermediate coordination and proof-building work.

Read original(opens in new tab)

AI AgentsCI/CDCode ReviewJson+3

line·Jun 25, 2026·4 min readCurated summary

In the AI Era, Development Ability Is Determined by Verification Skills: Strategies for Rapid Validation and Local Environment Setup Learned While Developing the Flava API Gateway

AI coding agents iterate quickly, but their output can be inconsistent, make incorrect design decisions, or generate code that does not compile. Because CI runs, environment provisioning, and human review are slower, the article argues that reliable agent-assisted development requires three practices: spec-driven development, automated verification, and fast, self-contained local environments. ## Flava API Gateway and the Development Challenge - Flava API Gateway is part of LY Corporation’s private Flava cloud. - It provides a multi-tenant RESTful control-plane API for creating, deploying, and monitoring web APIs. - Kong serves as the data plane. - The team adopted agent-based coding while building the product and focused on preserving software reliability without sacrificing AI-driven speed. ## Spec-Driven Development The team found that agents became more unpredictable when implementation began before the design was settled. They use explicit specifications to reduce ambiguity and constrain implementation decisions. - OpenAPI is written before code to define the control-plane API. - Features are divided into smaller units and implemented with OpenSpec. - Specifications serve both as implementation guidance and as a standard for detecting deviations. ### Managing OpenAPI with Nickel - Raw OpenAPI YAML is repetitive and difficult to maintain manually. - Nickel is used to describe API resources declaratively and generate complete CRUD specifications. - A resource definition can specify: - Description and parent resource - Whether updates are allowed - Automatic timestamps - Property schemas - Required fields - Sorting and filtering behavior - The generator produces consistent endpoints such as `listPaths`, `createPath`, `getPath`, and `deletePath`. - Generated endpoints include pagination, sorting, filtering, ETags for optimistic locking, and standardized error responses. ### OpenSpec Workflow OpenSpec structures each change into four artifacts: - **Proposal:** Why the change is needed and what will change - **Design:** Technical decisions and trade-offs - **Delta specifications:** Behavioral requirements written as Given-When-Then scenarios - **Task list:** A step-by-step implementation checklist The developer and agent review the feature together, the agent creates these artifacts, and then implements the checklist incrementally. Once complete, the delta specification is archived into the main specification library, creating a versioned, evolving record of the system’s behavior. ## Automated Verification The team initially tried adding lists of pitfalls to prompts, but found this ineffective and potentially harmful. Instead, they made tests and tools reveal errors progressively so the agent could diagnose and correct them. - Automated tests, linters, and formatters provide precise feedback. - Failed tests identify what went wrong, allowing the agent to fix one issue before moving to the next. - Project-specific skills bundle these checks together. - `AGENTS.md` tells the agent when to load the relevant skills, avoiding unnecessary instructions on every turn. - Testing and linting are treated as essential infrastructure rather than optional activities, since agents frequently make errors during implementation. ## Fast, Independent Local Environments Relying on CI and shared test environments is too slow for agent-driven iteration. Long waits can disrupt the agent’s context and make repeated experimentation impractical. - A complete local environment provides immediate feedback. - Local dependencies make logs and state easier to inspect. - Developers avoid sending every failed attempt through a remote pipeline. - The local test suite contains 2,754 tests across three layers: - **Unit tests:** Isolated business logic - **Integration tests:** Real PostgreSQL, database constraints, triggers, soft-delete cascades, transactions, in-process HTTP, and OpenAPI compliance - **End-to-end tests:** Athenz authentication, Kong, API keys, and multi-tenant isolation - The full suite completes in roughly 15 seconds on a developer machine. - Parallel execution and strong test isolation are critical to achieving this speed. ## Practical Recommendation Agent-assisted development works best when agents are given clear behavioral contracts, immediate automated feedback, and a fast local loop. Teams should invest in specifications, comprehensive tests and linting, and realistic local dependencies so agents can correct mistakes continuously without waiting for CI.

Read original(opens in new tab)

PostgreSQLCoding AgentsAutomated TestingOpenapi+4

line·Jun 24, 2026·4 min readCurated summary

Flava DBaaS Deep Dive: From Architecture to Migration and Beyond

LY Corporation’s Flava DBaaS is designed to unify the former Verda and YNW cloud platforms on a Kubernetes-based architecture. Its operator pattern separates database business logic from IaaS management, while API servers, managers, and agents divide responsibilities within each DBMS service. The platform expands database support, improves scalability, security, and usability, and treats migration from legacy platforms as part of the DBaaS responsibility. ## Kubernetes Operator-Based Design - Flava DBaaS uses the Kubernetes operator pattern. - Users declare the desired database state through custom resources rather than issuing procedural commands. - Controllers continuously reconcile the actual state with the declared specification. - This approach: - Simplifies troubleshooting through resource status and controller logs. - Handles large database infrastructures efficiently through event-driven processing. - Reuses Kubernetes capabilities for CI/CD and access control. ## Infrastructure Operator Layer - DBaaS must manage IaaS resources such as: - Virtual machines - Storage - Domains and networking - Flava isolates this infrastructure logic in a separate infrastructure operator. - IaaS resources are exposed as Kubernetes custom resources, allowing DBaaS to create infrastructure declaratively without directly calling IaaS APIs. - The resulting layers are: - **DBaaS:** Database-specific business logic - **Infrastructure operator:** Abstraction of IaaS as Kubernetes resources - **IaaS:** Compute, network, and storage services - This separation allows multiple DBMS products to use infrastructure consistently while their developers focus on database operations. ## Custom Resources and DBaaS Components - Each database cluster is represented by a Kubernetes custom resource containing settings such as: - DBMS version - VM size - Storage type and capacity - Replication configuration - These resources are stored in Kubernetes etcd and managed through the Kubernetes API. - Each DBMS implementation consists of three components: - **API server:** Provides REST APIs for creating, modifying, and deleting database resources. Flava UI and IaC tools use these APIs. - **Manager:** Watches resource changes and reconciles the database cluster toward the declared state. - **Agent:** Runs on database VMs and executes local operating-system and database commands. - For example, creating a MySQL cluster causes the API server to create a MySQL custom resource, the manager to provision the required VMs through the infrastructure operator, and the agent to configure replication and database processes inside those VMs. ## Improvements in Flava DBaaS - Flava preserves core DBaaS capabilities such as provisioning, high availability, backup and recovery, scalability, and monitoring. - It combines the DBMS offerings of Verda and YNW, expanding the range of supported database systems. ### Flexible Storage and Scaling - Storage can be configured in 100 GiB increments. - Block-storage-based databases can use up to 5 TiB of storage. - Unlike the legacy platforms, storage is no longer tightly limited by a VM’s local disk capacity. - Custom instance types and separate block storage reduce the need to consider alternatives such as sharding for larger databases. - The 5 TiB limit was selected to cover most analyzed use cases while reducing infrastructure fragmentation. ### Consistent User Experience - All Flava DBaaS products share a common architecture and UI. - Skills learned while changing MySQL server specifications or configuring Cassandra alerts can be applied to other DBMS products. - Users do not need to learn separate operational workflows for each database system. ### Security and Convenience - TDE and TLS are provided as platform-level security features. - Additional features include: - **Custom DB Role:** Reusable database users with configurable permissions. - **Database Parameter Group:** Reusable groups of database configuration parameters. - **Restore backup:** Creation of a new cluster from a selected backup for disaster recovery or realistic performance testing. - Features not yet available for every DBaaS product are planned for broader support. - These improvements reportedly resulted in high internal user-satisfaction scores. ## Migration Responsibilities - A new DBaaS platform is expected to provide migration paths from existing platforms, not merely offer new database clusters. - For migrations between the same DBMS type, the article identifies three general approaches. ### Dump and Restore - Data is backed up from the source database and restored into the destination. - It is the simplest method. - To guarantee consistency, the application generally must be stopped during the migration. ### Replication-Based Migration - The source database is continuously replicated to the destination. - Once replication is caught up, the destination is promoted through failover. - The source database can then be removed. - Data consistency depends on the DBMS’s replication mechanism. - A short application interruption may still occur during primary-node failover. The overall recommendation is to use Flava’s layered, declarative architecture to standardize database operations while continuing to provide practical migration mechanisms from Verda and YNW.

Read original(opens in new tab)

KubernetesInfrastructure As CodeMysqlDatabase Migration+4

line·Jun 24, 2026·4 min readCurated summary

Designing a Semantic Context OS: Beyond Token Stuffing in Agent Systems

The article argues that larger LLM context windows do not automatically produce better software-engineering agents. In long-running workflows, indiscriminately filling the context window can cause attention dilution, context rot, reasoning failures, and potential data exposure. It proposes a “Semantic Context OS,” a local runtime layer that actively governs context as a finite, structured system resource rather than treating it as an unmanaged text stream. ## The Context Window Is Not RAM - The article uses the “Karpathy metaphor”: - The LLM acts like a CPU: a largely stateless inference engine driven by pretrained parameters. - The context window acts like RAM: volatile working memory containing current state, instructions, telemetry, and runtime data. - Unlike physical RAM, LLM context is probabilistic rather than deterministic: - Traditional RAM provides precise address-based retrieval with predictable performance. - LLM retrieval depends on attention weights across Q, K, and V matrices. - Increasing capacity from 32K tokens to 1M or 2M tokens therefore does not guarantee proportionally better retrieval. Larger sequences also increase computational cost and structural noise. ## Attention Dilution and Long-Context Failure - Large codebases and logs contain substantial irrelevant material, including: - Boilerplate definitions - Unused imports - Duplicate syntax - Repeated utilities and naming patterns - As sequence length grows, the attention calculation `QKᵀ` accumulates entropy and background noise. - Softmax then spreads attention energy across more tokens, weakening the sharp attention peaks needed to retrieve important facts. - This contributes to the “lost in the middle” effect: - Information near the beginning and end of a prompt is often retrieved more reliably. - Retrieval accuracy can fall sharply across the middle portion of the context. - The article considers relying on massive, unmanaged contexts an architectural anti-pattern for tasks such as large-scale code review, dependency tracing, and automated refactoring. ## Context Rot in Long-Running Agents The article defines “context rot” as the degradation of an agent’s working context during extended autonomous tasks. - **Context poisoning** - Raw logs, obsolete errors, and previous execution data accumulate over multiple turns. - The model may treat temporary historical failures as current architectural constraints. - **Context distraction** - Monorepos often contain similar names, overloaded methods, and duplicated helper code. - Broad retrieval can overwhelm the model with structurally similar but logically irrelevant code. - **Context clash** - Old instructions may remain after the plan has evolved. - Contradictory directives can cause indecision, infinite reasoning loops, timeouts, or hallucinations. - The article claims that, without active management, failure rates increase nonlinearly with context depth and may reach roughly 40% in deeply nested codebases. ## Semantic Context OS as an AI Kernel The proposed Semantic Context OS sits between agent application logic and external foundation-model APIs, operating as a localhost loopback proxy at `localhost:8080`. Its responsibilities include: - Treating context as a finite hardware-like resource. - Tracking token lifecycles and state access. - Filtering and isolating data before it reaches the model. - Separating physical token limits from semantic governance. - Protecting downstream inference engines from structural noise and helping prevent intellectual-property leakage. The architecture includes: - A POSIX-like virtual file system for managing state topology. - A proprietary “PathAlign” stage for AST-based code-tree pruning. - An asynchronous “sawtooth” memory model for runtime token optimization. ## MVC: Minimum Viable Context The core MVC pipeline—described as “minimum viable context”—aims to provide only the smallest dense set of information required for the agent’s current reasoning step. Its processing stages include: - **Collection and token mapping** - Gather source files, dependency graphs, and runtime logs. - Map them using the target model’s tokenizer, such as `cl100k_base` or `o200k_base`. - **Structural pruning** - Use static analysis and structural rules to remove compiler comments, unused imports, boilerplate, and unrelated utilities. - The broader design replaces passive string concatenation with active context selection, lifecycle management, and bounded transmission policies. The article concludes that reliable enterprise agents require active context orchestration rather than larger prompts alone. A dedicated governance layer should prune, isolate, and refresh context throughout execution so that models receive minimal, relevant, and internally consistent information.

Read original(opens in new tab)

Large Language ModelsAI AgentsTransformerAbstract Syntax Tree+4

line·Jun 23, 2026·4 min readCurated summary

From Manual to AI Prompt Tuning: Genetic Algorithm–Based Automated Optimization and Acceleration

LY Corporation automated LLM prompt tuning with the GEPA genetic algorithm, reducing a process that previously took days or weeks to roughly one hour. GEPA evolves prompt candidates using evaluation scores and natural-language feedback, allowing it to improve prompts without manually inspecting every output. The approach was applied to Yahoo! JAPAN Search’s AI responses for health and medical queries, balancing policy compliance with improved readability. ## Challenges of Manual Prompt Tuning - Each prompt change requires repeated output generation and human review. - Practical tuning knowledge often remains with individual engineers and is difficult to document or explain. - The cycle of editing, generating, and evaluating responses can take days or weeks. - Model changes and version updates can alter output quality, requiring repeated retuning. - Manual effort leaves less time for defining evaluation criteria, judging quality, and verifying policy compliance. ## Automated Prompt Optimization Approaches - **Reinforcement learning:** Learns prompt-generation policies from scalar rewards, such as with GRPO. - **Bayesian optimization:** Efficiently searches candidate instructions and few-shot examples, as in MIPROv2. - **Genetic algorithms:** Iteratively evolve a population of prompt candidates, as in GEPA. - Genetic methods are well suited to discrete, natural-language prompts because they can use natural-language reflection to identify problems and propose improvements rather than relying only on numerical rewards. ## How GEPA Works - Generates and evaluates multiple prompt candidates. - Uses **Reflective Prompt Mutation** to analyze execution results and create improved instructions. - Applies Pareto-frontier selection to preserve candidates that perform well across multiple evaluation dimensions. - Repeats the process over several to dozens of generations until prompts converge toward the evaluation objectives. - The article notes that GEPA has reportedly outperformed previous optimization methods, including results presented at ICLR 2026. ## Implementation with DSPy - DSPy allows prompt optimization to be controlled programmatically. - A task is defined as a DSPy module with a signature containing input and output fields. - The signature’s docstring becomes an instruction for the LLM. - GEPA rewrites this instruction during optimization. - Separate models can be assigned for: - Task inference - Output evaluation - Reflection and prompt improvement ## Designing the Evaluation Function - GEPA requires an overall scalar score, even when quality is judged across multiple criteria. - Individual scores can be assigned to dimensions such as accuracy, completeness, and style, then normalized and averaged. - The evaluator can also return natural-language feedback through `dspy.Prediction(score=..., feedback=...)`. - Feedback explains why a candidate was penalized, giving GEPA a clearer direction for improvement than a score alone. - Evaluation can use: - LLM-as-a-Judge - Gold answers or labels - Rule-based correctness checks - In the example, an evaluator scores three criteria from 0 to 10, averages them into a single score, and passes the explanation to GEPA for reflection. ## Yahoo! JAPAN Search Health and Medical Queries - Health-related answers must follow stricter policies than general search responses. - Requirements include: - Avoiding definitive medical diagnoses or severity judgments - Matching wording to the strength of available evidence - Recommending medical consultation appropriately - Limiting responses to general explanations where necessary - The project pursued two goals simultaneously: - Satisfy medical and health-policy requirements. - Apply readable Markdown formatting, including headings, lists, and emphasis. - Improving one goal manually could easily damage the other, making automated optimization attractive. ## Applying GEPA to the Production Task - The system takes a search query as input and generates an AI answer. - The initial prompt combined an existing general-purpose prompt with additional health and medical policy instructions. - GEPA rewrote and optimized the instruction section rather than requiring engineers to manually redesign the entire prompt. - The optimization aimed to preserve policy compliance while improving structure and readability. Overall, GEPA with DSPy provides a practical way to shorten prompt-tuning cycles and make the improvement process more reproducible. Its effectiveness depends heavily on carefully designed evaluation criteria and meaningful natural-language feedback, especially for high-risk domains such as medical information.

Read original(opens in new tab)

Large Language ModelsLlm As A JudgePrompt OptimizationDspy+3

line·Jun 23, 2026·4 min readCurated summary

Total Capacity Exceeds 1 EB! How Do You Connect Two HDFS Systems with Different Histories? Challenges and Design Decisions in Data Platform Integration

LY Corporation’s Tech-Verse 2026 article examines how its former LINE and Yahoo Japan organizations operated HDFS platforms exceeding one exabyte in total capacity. Although both platforms used Hadoop at scale, their access models, namespace architectures, permission systems, and operational practices differed substantially. The article argues that large-scale data platforms must be designed around actual usage patterns, not just storage capacity, and previews how the two environments were later connected after organizational integration. ## Different Operating Models - Former LINE built a unified analytics environment for broad, cross-departmental data use. - Users accessed data through a web portal that managed catalogs, permissions, and role-based approval workflows rather than interacting directly with HDFS or Apache Ranger. - BI tools, reporting systems, and ETL pipelines supported diverse use cases, but integrating multiple existing clusters made operations complex. - Former Yahoo Japan evolved from a limited-purpose Hadoop deployment into a company-wide platform. - Its user interfaces and access methods were intentionally restricted, making the system easier to stabilize and support. - Yahoo Japan retained HDFS-style POSIX permissions, which limited flexibility compared with newer data-governance models. ## Different HDFS Architectures - Both platforms split their storage across multiple namespaces to overcome NameNode scaling limitations. - Each namespace used two to four NameNodes for redundancy, but the way namespaces were exposed differed: - LINE used **ViewFS**, requiring clients to maintain mount-table configurations. - Yahoo Japan used **Router-Based Federation (RBF)**, allowing routers to direct requests to the correct NameNode. - LINE shared DataNodes across namespaces, improving resource efficiency but increasing operational complexity. - LINE also had mixed NameNode and DataNode versions because several legacy platforms had been consolidated. - Yahoo Japan’s RBF design included Observer NameNodes to distribute read load. - These architectural differences affected later federation work, including connection endpoints, client configuration, network reachability, and permission management. ## Capacity and Network Challenges at LINE - Data growth exceeded forecasts, causing HDFS capacity shortages before new servers could be delivered. - Older servers were temporarily reused, leading to frequent node additions and removals. - Large changes in node count triggered HDFS Balancer activity and block redistribution, generating substantial network traffic. - Network engineers therefore had to coordinate closely with the Hadoop operations team during infrastructure changes. ## NameNode Metadata and Small-File Problems - As file and block counts increased, NameNode heap usage and processing load grew. - Larger heaps also increased garbage-collection times, making NameNodes slower and less stable. - The team analyzed regularly dumped FSImage data stored in Hive tables to identify users, paths, file counts, block counts, and data volumes. - They prioritized tables containing many small files where file compaction could significantly reduce block counts without requiring data deletion or schema changes. - File merging reduced both NameNode metadata pressure and the number of HDFS operations, improving response times for jobs. ## Namespace-Specific Load Patterns - Different namespaces experienced different types of pressure. - Temporary-file namespaces saw frequent Spark staging-file creation and deletion, producing repeated metadata updates requiring NameNode write locks. - When HDFS Balancer moved blocks, read-lock activity increased and could delay file creation and deletion. - Increasing Balancer parallelism initially worsened contention. - The team reduced parallelism to a level compatible with available DataNode disk capacity, balancing migration speed against cluster impact. ## Connecting the Two Platforms - Organizational integration introduced additional challenges beyond storage: - Determining which platform and entry point users should access - Reconciling different permission-management models - Establishing data-transfer paths between platforms - LINE’s ViewFS model depends on correctly distributed client mount tables. - Yahoo Japan’s RBF model depends on reliable, scalable, and reachable router infrastructure. - These differences directly influence cross-platform data movement, including transfers using DistCP. Large HDFS environments should be managed according to real workload behavior, namespace characteristics, and operational dependencies. Capacity planning alone is insufficient; teams should monitor metadata growth, small-file patterns, lock contention, balancing traffic, network effects, and the distinct access models of each platform.

Read original(opens in new tab)

Apache SparkData PlatformHdfsApache Hadoop+3

line·Jun 22, 2026·4 min readCurated summary

Unifying Analysis Through the Power of Analytics Agents: Work Innovation and Role Transformation in the Generative AI Era at a Professional Organization

PJ One Piece is LY Corporation’s initiative to connect business questions, data analysis, insight generation, and next-action planning through generative AI. Its analysis agent reduced typical turnaround times from about two weeks to roughly 10 minutes, enabling hundreds of analyses each month and adoption by more than half of an early-adopter business unit. The project treats AI not as a chat interface, but as an analysis platform that connects data, knowledge, people, and organizational processes. ## Three Disconnects Behind the Project - **Business and data:** Even with a data warehouse and BI tools, business users still needed to understand SQL, tables, column definitions, KPI rules, and result interpretation. - **Within the analysis process:** Task definition, analysis design, execution, review, and action planning were often handled by different people or tools, causing context loss, rework, delays, and inconsistent quality. - **Across domains:** Useful analysis patterns and domain knowledge remained isolated because services used different KPIs, table structures, business assumptions, and review criteria. ## The Analysis Agent as a Connector - Users ask questions in natural language without needing to know SQL or database structures. - The agent: - Clarifies the business objective and missing assumptions. - Finds relevant data and creates an analysis plan. - Executes queries and specialized analyses. - Interprets results and produces visualizations or reports. - Suggests further analysis and possible next actions. - The platform consists of: - A user-facing application. - An LLM-based agent for reasoning and tool use. - Tools for SQL, Python, document search, and visualization. - A knowledge base containing domain information, skills, and table metadata. - Logging, feedback, monitoring, and evaluation systems. - Domain knowledge is added through a plugin-like structure, while logs and feedback continuously improve the system. ## Turning Business Questions into Analysis Requirements - Natural-language questions often leave important assumptions unspecified, such as: - Target population or campaign definition. - Analysis period and comparison group. - KPI definitions. - Aggregation level. - Exclusion conditions. - Rather than requiring users to write detailed prompts, the agent uses domain knowledge to determine what can be inferred and asks only about unresolved points. - Knowledge bases document service context, KPI definitions, aggregation cautions, policy information, and review requirements. - Table metadata explains available tables, columns, appropriate use cases, samples, partition requirements, and usage restrictions. ## Reaching Data Safely and Reliably - Table metadata is revealed progressively: - The agent first narrows down relevant tables. - It then retrieves detailed definitions and usage rules only for those tables. - Analysis-oriented wide tables or logical views combine transaction data with commonly needed attributes, reducing complicated joins and SQL-generation errors. - SQL is checked before and after execution to enforce: - `SELECT`-only access. - Approved tables and usage rules. - Required partition conditions. - Restrictions on sensitive or personal data. - Result-size limits. - These guardrails allow the agent to perform analysis flexibly without exposing data or infrastructure to unnecessary risks. ## Preserving Context Across the Analysis Process - PJ One Piece uses a supervisor-style multi-agent architecture. - A main agent maintains: - The user’s request and business objective. - The current analysis plan. - Findings and constraints discovered so far. - Remaining questions and decision points. - Specialized sub-agents handle tasks such as statistical testing, time-series analysis, clustering, and independent review. - This separates complex or specialized work from the main context while preserving overall continuity. - Progress updates expose discoveries, design decisions, data limitations, and constraints so users can adjust direction during longer analyses. ## Building Reusable Organizational Capability - Logs record agent actions, assumption checks, analysis designs, generated SQL, errors, and outputs. - User and analyst feedback helps identify whether improvements are needed in prompts, tools, data, or reusable skills. - Repeated workflows are formalized as skills, including: - General-purpose methods such as time-series and clustering analysis. - Domain-specific workflows such as monthly reporting or policy monitoring. - Skills document required assumptions, comparison axes, cautions, and interpretation methods. - Over time, isolated domain knowledge becomes reusable organizational analysis capability. ## Business Impact - In early deployment, the platform expanded data use beyond data scientists to product owners and frontline employees. - More than half of the participating business unit’s members use it. - Analysis turnaround fell from an average of approximately two weeks to about 10 minutes. - The platform now supports hundreds of analyses per month and serves as a daily starting point for business questions. PJ One Piece’s main recommendation is to design AI analysis as an end-to-end operating platform—not merely an automated SQL or chatbot tool. Combining structured domain knowledge, safe data access, contextual multi-agent workflows, reusable skills, and continuous evaluation can make analysis faster while steadily improving its quality and organizational reach.

Read original(opens in new tab)

Large Language ModelsAI AgentsGenerative AIPython+4