line·Jun 26, 2026·3 min readCurated summary

From Prompting to Workflows: Boosting Frontend Development Productivity with AI

Frontend development is increasingly shifting from a coding problem to an orchestration problem. Requirements, designs, documentation, discussions, and existing code are scattered across tools, while LLMs can now connect these sources through repeatable workflows. The article argues that structured, reviewable workflows—rather than clever one-off prompts—are the key to scaling AI-assisted development and improving implementation quality. ## From Prompting to Repeatable Workflows - A prompt may produce a useful result once, but it does not create a reusable process. - A workflow defines a repeatable path from inputs to outputs: - Collect context from Jira, Confluence, Slack, and the codebase. - Summarize the actual requirements. - Identify ambiguities and unresolved decisions. - Propose an implementation plan. - Wait for human review before modifying code. - The LLM acts as the engine executing the workflow. - LY Corporation’s Noah MCP connects systems such as Jira, Confluence, Slack, and GitHub, allowing AI agents to access real organizational context instead of relying on manually copied prompts. - Once established, the same workflow pattern can be applied across many tickets, even when the specific inputs differ. ## Example: Planning a List Page The example Jira ticket requests a list page with search, filtering, sorting, and role-based filter visibility. - In the traditional process, a developer manually: - Reads the Jira ticket and identifies missing details. - Searches Figma for loading, empty, and no-results states. - Finds role-based filter rules in Confluence. - Searches Slack for prior decisions. - Inspects the codebase for reusable hooks and components. - Copies findings into notes and assembles an implementation plan. - Implements the feature, resolves bugs and edge cases, and submits a PR. - An AI workflow performs these steps systematically before coding. - The generated plan identifies: - A new `FeatureListPage` route and `FeatureList` component. - Reuse of `useTableFilters` and `useUrlState`. - Existing API support through `GET /api/<feature>`. - URL synchronization for filters, sorting, and pagination. - Role-based visibility using `useCurrentUserRole()`. - Required loading, empty, and no-results states. ## Surfacing Hidden Requirements The workflow improves quality by exposing information that might otherwise appear late in development. - A Slack decision establishes that filter and sort state should use URL parameters rather than `localStorage`, enabling shareable and reloadable views. - Existing hooks such as `useTableFilters` and `useUrlState` are discovered before new code is written, preventing unnecessary duplication. - Unresolved questions are explicitly listed for human review, including: - Whether filter and sort state belongs in URL parameters or `localStorage`. - Which empty-state design should be used when Figma contains multiple variants. - Resolving these questions early reduces rework during implementation or PR review. ## Closed-Loop Verification The workflow should continue after coding rather than stopping when the first implementation is complete. - The agent compares the implementation with the original plan. - It runs: - Type checks. - Linting. - Related unit tests. - Relevant smoke tests or local verification flows. - It reports: - Successful checks. - Failures that were fixed. - Items that could not be verified automatically. - UI screenshots or state notes. - Remaining risks before opening a PR. - This creates a closed-loop development cycle in which AI not only writes code but also validates its work against the intended requirements. Teams should treat AI as a workflow and context-orchestration layer, not merely a code generator. The most effective process gathers information across systems, obtains human approval for the plan, implements with existing project patterns, and automatically verifies the result before review.

Read original(opens in new tab)

Large Language ModelsModel Context ProtocolReactTypeScript+4

line·Jun 24, 2026·4 min readCurated summary

Designing a Semantic Context OS: Beyond Token Stuffing in Agent Systems

The article argues that larger LLM context windows do not automatically produce better software-engineering agents. In long-running workflows, indiscriminately filling the context window can cause attention dilution, context rot, reasoning failures, and potential data exposure. It proposes a “Semantic Context OS,” a local runtime layer that actively governs context as a finite, structured system resource rather than treating it as an unmanaged text stream. ## The Context Window Is Not RAM - The article uses the “Karpathy metaphor”: - The LLM acts like a CPU: a largely stateless inference engine driven by pretrained parameters. - The context window acts like RAM: volatile working memory containing current state, instructions, telemetry, and runtime data. - Unlike physical RAM, LLM context is probabilistic rather than deterministic: - Traditional RAM provides precise address-based retrieval with predictable performance. - LLM retrieval depends on attention weights across Q, K, and V matrices. - Increasing capacity from 32K tokens to 1M or 2M tokens therefore does not guarantee proportionally better retrieval. Larger sequences also increase computational cost and structural noise. ## Attention Dilution and Long-Context Failure - Large codebases and logs contain substantial irrelevant material, including: - Boilerplate definitions - Unused imports - Duplicate syntax - Repeated utilities and naming patterns - As sequence length grows, the attention calculation `QKᵀ` accumulates entropy and background noise. - Softmax then spreads attention energy across more tokens, weakening the sharp attention peaks needed to retrieve important facts. - This contributes to the “lost in the middle” effect: - Information near the beginning and end of a prompt is often retrieved more reliably. - Retrieval accuracy can fall sharply across the middle portion of the context. - The article considers relying on massive, unmanaged contexts an architectural anti-pattern for tasks such as large-scale code review, dependency tracing, and automated refactoring. ## Context Rot in Long-Running Agents The article defines “context rot” as the degradation of an agent’s working context during extended autonomous tasks. - **Context poisoning** - Raw logs, obsolete errors, and previous execution data accumulate over multiple turns. - The model may treat temporary historical failures as current architectural constraints. - **Context distraction** - Monorepos often contain similar names, overloaded methods, and duplicated helper code. - Broad retrieval can overwhelm the model with structurally similar but logically irrelevant code. - **Context clash** - Old instructions may remain after the plan has evolved. - Contradictory directives can cause indecision, infinite reasoning loops, timeouts, or hallucinations. - The article claims that, without active management, failure rates increase nonlinearly with context depth and may reach roughly 40% in deeply nested codebases. ## Semantic Context OS as an AI Kernel The proposed Semantic Context OS sits between agent application logic and external foundation-model APIs, operating as a localhost loopback proxy at `localhost:8080`. Its responsibilities include: - Treating context as a finite hardware-like resource. - Tracking token lifecycles and state access. - Filtering and isolating data before it reaches the model. - Separating physical token limits from semantic governance. - Protecting downstream inference engines from structural noise and helping prevent intellectual-property leakage. The architecture includes: - A POSIX-like virtual file system for managing state topology. - A proprietary “PathAlign” stage for AST-based code-tree pruning. - An asynchronous “sawtooth” memory model for runtime token optimization. ## MVC: Minimum Viable Context The core MVC pipeline—described as “minimum viable context”—aims to provide only the smallest dense set of information required for the agent’s current reasoning step. Its processing stages include: - **Collection and token mapping** - Gather source files, dependency graphs, and runtime logs. - Map them using the target model’s tokenizer, such as `cl100k_base` or `o200k_base`. - **Structural pruning** - Use static analysis and structural rules to remove compiler comments, unused imports, boilerplate, and unrelated utilities. - The broader design replaces passive string concatenation with active context selection, lifecycle management, and bounded transmission policies. The article concludes that reliable enterprise agents require active context orchestration rather than larger prompts alone. A dedicated governance layer should prune, isolate, and refresh context throughout execution so that models receive minimal, relevant, and internally consistent information.

Read original(opens in new tab)

Large Language ModelsAI AgentsTransformerAbstract Syntax Tree+4

line·Jun 23, 2026·4 min readCurated summary

From Manual to AI Prompt Tuning: Genetic Algorithm–Based Automated Optimization and Acceleration

LY Corporation automated LLM prompt tuning with the GEPA genetic algorithm, reducing a process that previously took days or weeks to roughly one hour. GEPA evolves prompt candidates using evaluation scores and natural-language feedback, allowing it to improve prompts without manually inspecting every output. The approach was applied to Yahoo! JAPAN Search’s AI responses for health and medical queries, balancing policy compliance with improved readability. ## Challenges of Manual Prompt Tuning - Each prompt change requires repeated output generation and human review. - Practical tuning knowledge often remains with individual engineers and is difficult to document or explain. - The cycle of editing, generating, and evaluating responses can take days or weeks. - Model changes and version updates can alter output quality, requiring repeated retuning. - Manual effort leaves less time for defining evaluation criteria, judging quality, and verifying policy compliance. ## Automated Prompt Optimization Approaches - **Reinforcement learning:** Learns prompt-generation policies from scalar rewards, such as with GRPO. - **Bayesian optimization:** Efficiently searches candidate instructions and few-shot examples, as in MIPROv2. - **Genetic algorithms:** Iteratively evolve a population of prompt candidates, as in GEPA. - Genetic methods are well suited to discrete, natural-language prompts because they can use natural-language reflection to identify problems and propose improvements rather than relying only on numerical rewards. ## How GEPA Works - Generates and evaluates multiple prompt candidates. - Uses **Reflective Prompt Mutation** to analyze execution results and create improved instructions. - Applies Pareto-frontier selection to preserve candidates that perform well across multiple evaluation dimensions. - Repeats the process over several to dozens of generations until prompts converge toward the evaluation objectives. - The article notes that GEPA has reportedly outperformed previous optimization methods, including results presented at ICLR 2026. ## Implementation with DSPy - DSPy allows prompt optimization to be controlled programmatically. - A task is defined as a DSPy module with a signature containing input and output fields. - The signature’s docstring becomes an instruction for the LLM. - GEPA rewrites this instruction during optimization. - Separate models can be assigned for: - Task inference - Output evaluation - Reflection and prompt improvement ## Designing the Evaluation Function - GEPA requires an overall scalar score, even when quality is judged across multiple criteria. - Individual scores can be assigned to dimensions such as accuracy, completeness, and style, then normalized and averaged. - The evaluator can also return natural-language feedback through `dspy.Prediction(score=..., feedback=...)`. - Feedback explains why a candidate was penalized, giving GEPA a clearer direction for improvement than a score alone. - Evaluation can use: - LLM-as-a-Judge - Gold answers or labels - Rule-based correctness checks - In the example, an evaluator scores three criteria from 0 to 10, averages them into a single score, and passes the explanation to GEPA for reflection. ## Yahoo! JAPAN Search Health and Medical Queries - Health-related answers must follow stricter policies than general search responses. - Requirements include: - Avoiding definitive medical diagnoses or severity judgments - Matching wording to the strength of available evidence - Recommending medical consultation appropriately - Limiting responses to general explanations where necessary - The project pursued two goals simultaneously: - Satisfy medical and health-policy requirements. - Apply readable Markdown formatting, including headings, lists, and emphasis. - Improving one goal manually could easily damage the other, making automated optimization attractive. ## Applying GEPA to the Production Task - The system takes a search query as input and generates an AI answer. - The initial prompt combined an existing general-purpose prompt with additional health and medical policy instructions. - GEPA rewrote and optimized the instruction section rather than requiring engineers to manually redesign the entire prompt. - The optimization aimed to preserve policy compliance while improving structure and readability. Overall, GEPA with DSPy provides a practical way to shorten prompt-tuning cycles and make the improvement process more reproducible. Its effectiveness depends heavily on carefully designed evaluation criteria and meaningful natural-language feedback, especially for high-risk domains such as medical information.

Read original(opens in new tab)

Large Language ModelsLlm As A JudgePrompt OptimizationDspy+3

line·Jun 22, 2026·4 min readCurated summary

Unifying Analysis Through the Power of Analytics Agents: Work Innovation and Role Transformation in the Generative AI Era at a Professional Organization

PJ One Piece is LY Corporation’s initiative to connect business questions, data analysis, insight generation, and next-action planning through generative AI. Its analysis agent reduced typical turnaround times from about two weeks to roughly 10 minutes, enabling hundreds of analyses each month and adoption by more than half of an early-adopter business unit. The project treats AI not as a chat interface, but as an analysis platform that connects data, knowledge, people, and organizational processes. ## Three Disconnects Behind the Project - **Business and data:** Even with a data warehouse and BI tools, business users still needed to understand SQL, tables, column definitions, KPI rules, and result interpretation. - **Within the analysis process:** Task definition, analysis design, execution, review, and action planning were often handled by different people or tools, causing context loss, rework, delays, and inconsistent quality. - **Across domains:** Useful analysis patterns and domain knowledge remained isolated because services used different KPIs, table structures, business assumptions, and review criteria. ## The Analysis Agent as a Connector - Users ask questions in natural language without needing to know SQL or database structures. - The agent: - Clarifies the business objective and missing assumptions. - Finds relevant data and creates an analysis plan. - Executes queries and specialized analyses. - Interprets results and produces visualizations or reports. - Suggests further analysis and possible next actions. - The platform consists of: - A user-facing application. - An LLM-based agent for reasoning and tool use. - Tools for SQL, Python, document search, and visualization. - A knowledge base containing domain information, skills, and table metadata. - Logging, feedback, monitoring, and evaluation systems. - Domain knowledge is added through a plugin-like structure, while logs and feedback continuously improve the system. ## Turning Business Questions into Analysis Requirements - Natural-language questions often leave important assumptions unspecified, such as: - Target population or campaign definition. - Analysis period and comparison group. - KPI definitions. - Aggregation level. - Exclusion conditions. - Rather than requiring users to write detailed prompts, the agent uses domain knowledge to determine what can be inferred and asks only about unresolved points. - Knowledge bases document service context, KPI definitions, aggregation cautions, policy information, and review requirements. - Table metadata explains available tables, columns, appropriate use cases, samples, partition requirements, and usage restrictions. ## Reaching Data Safely and Reliably - Table metadata is revealed progressively: - The agent first narrows down relevant tables. - It then retrieves detailed definitions and usage rules only for those tables. - Analysis-oriented wide tables or logical views combine transaction data with commonly needed attributes, reducing complicated joins and SQL-generation errors. - SQL is checked before and after execution to enforce: - `SELECT`-only access. - Approved tables and usage rules. - Required partition conditions. - Restrictions on sensitive or personal data. - Result-size limits. - These guardrails allow the agent to perform analysis flexibly without exposing data or infrastructure to unnecessary risks. ## Preserving Context Across the Analysis Process - PJ One Piece uses a supervisor-style multi-agent architecture. - A main agent maintains: - The user’s request and business objective. - The current analysis plan. - Findings and constraints discovered so far. - Remaining questions and decision points. - Specialized sub-agents handle tasks such as statistical testing, time-series analysis, clustering, and independent review. - This separates complex or specialized work from the main context while preserving overall continuity. - Progress updates expose discoveries, design decisions, data limitations, and constraints so users can adjust direction during longer analyses. ## Building Reusable Organizational Capability - Logs record agent actions, assumption checks, analysis designs, generated SQL, errors, and outputs. - User and analyst feedback helps identify whether improvements are needed in prompts, tools, data, or reusable skills. - Repeated workflows are formalized as skills, including: - General-purpose methods such as time-series and clustering analysis. - Domain-specific workflows such as monthly reporting or policy monitoring. - Skills document required assumptions, comparison axes, cautions, and interpretation methods. - Over time, isolated domain knowledge becomes reusable organizational analysis capability. ## Business Impact - In early deployment, the platform expanded data use beyond data scientists to product owners and frontline employees. - More than half of the participating business unit’s members use it. - Analysis turnaround fell from an average of approximately two weeks to about 10 minutes. - The platform now supports hundreds of analyses per month and serves as a daily starting point for business questions. PJ One Piece’s main recommendation is to design AI analysis as an end-to-end operating platform—not merely an automated SQL or chatbot tool. Combining structured domain knowledge, safe data access, contextual multi-agent workflows, reusable skills, and continuous evaluation can make analysis faster while steadily improving its quality and organizational reach.

Read original(opens in new tab)

Large Language ModelsAI AgentsGenerative AIPython+4

line·May 22, 2026·4 min readCurated summary

ODW #7: Reduce Token Consumption by 40% in Three Ways! Context Engineering with ADK

The post explains how LY Corporation’s Orchestration Development Workshop uses context engineering to reduce AI-agent costs and improve accuracy. As internal adoption of tools such as Claude Code, Cline, and ADK grows, excessive token usage, missed instructions, and declining performance in long conversations have become common. The recommended solution is to deliberately select and manage the context sent to an LLM, demonstrated through an ADK-based Jira weekly-report agent. ## Problems Caused by Expanding AI Use - Increased AI adoption has led to unexpectedly high token consumption. - Users often receive incomplete or incorrect results despite providing detailed prompts. - Long-running conversations can cause the model to produce irrelevant answers. - Major causes include: - Trial-and-error prompting - More complex and long-running agents - Expansion from single-agent to multi-agent systems - Tool integrations such as MCP, whose definitions also consume context - Limited awareness of context optimization techniques ## Context Rot and Context Engineering - **Context rot** occurs when long-running agents accumulate conversation history, intermediate results, and irrelevant information. - As the context grows: - The context window becomes pressured. - Relevant information becomes harder to identify. - Noise overwhelms important signals, reducing accuracy. - Context engineering is the deliberate design and management of all information provided during inference, including: - **Static context:** System prompts and tool definitions - **Dynamic context:** User messages, conversation history, and retrieved external data - **Long-term context:** Persistent session state and accumulated information - The core principles are: - Treat tokens as a limited resource and retain the smallest set of high-signal information. - Provide neither too little information, which forces guesswork, nor too much, which wastes tokens and reduces clarity. ## Why Use ADK Google’s open-source Agent Development Kit (ADK) is presented as a practical platform for applying context engineering. - Agents can be designed and shared using team knowledge rather than relying on individual CLI expertise. - ADK includes UI, API-server, evaluation, and multi-agent capabilities. - Its multi-agent architecture naturally supports separating and controlling context. ## ADK Context-Engineering Components The workshop introduces nine key components, including: - **Structured input and output:** JSON or schema-based formats reduce unnecessary text and make agent processing more reliable. - **AgentTool:** Embeds one agent inside another as a tool. The calling agent receives only the final result, preventing internal tools and intermediate context from accumulating. - **MCP Toolset filtering:** The `tool_filter` parameter exposes only required MCP tools, reducing tool-definition tokens and improving model decisions. - The remaining components can be combined with these techniques to control context throughout an agent workflow. ## Jira Weekly Report Example The workshop builds `jira_weekly_report`, an agent that analyzes team Jira tickets and generates a weekly Markdown report. ### Version 1: Single Agent Without Context Engineering - A single agent retrieves the ticket list, fetches each ticket, analyzes it, and builds the report. - All Jira tools are exposed through one MCP toolset. - As the number of tickets increases, detailed ticket contents accumulate in the agent’s context. - This leads to context rot, higher token usage, and declining reliability. ### Version 2: Context-Aware Multi-Agent Design - The workflow is split into: - A root agent that searches Jira tickets and aggregates the final report. - A sub-agent dedicated to analyzing one ticket at a time. - `input_schema` requires a structured `issue_key`. - `output_schema` requires a structured report containing ticket content and progress, including comments. - The sub-agent receives only the `jira_get_issue` MCP tool. - The root agent receives only the `jira_search` tool. - `AgentTool` hides the sub-agent’s internal context and returns only its final report. - The sub-agent is instructed to include facts only and avoid speculation. This design limits each agent’s responsibilities, removes unnecessary tool definitions, and prevents individual ticket details from polluting the root agent’s context. ## Practical Recommendation For production AI agents, treat context as a constrained resource. Use structured schemas, narrowly filtered tools, and specialized sub-agents to pass only the information needed for each step.

Read original(opens in new tab)

Large Language ModelsAI AgentsKubernetesModel Context Protocol+3

line·Mar 10, 2026·4 min readCurated summary

Journey Toward Perfect AI Guardrails

NeurIPS 2025 research shows that AI safety is moving beyond simple post-training alignment and output filtering toward system-level, modular defenses. New approaches intervene in reasoning, multimodal interpretation, policy enforcement, and continuous evaluation to balance safety with latency and usefulness. The central conclusion is that deployable AI requires adaptable guardrails designed for real-world systems, not isolated attack benchmarks. ## The Shift Toward Practical AI Safety - Guardrails protect AI services from harmful instructions, privacy leaks, confidential-data exposure, bias, prompt injection, and other failures. - NeurIPS 2025 reflects a broader shift: - From post-training safety tuning to intervention in reasoning mechanisms. - From text-only LLMs to VLMs, RAG systems, and reasoning models. - From laboratory attack scenarios to the practical balance between utility and safety. - The article focuses on guardrail frameworks, multimodal moderation, prompt injection and jailbreaks, hallucinations, and over-refusal. ## Modular Guardrail Frameworks **PRIME Guardrails: A General, Low-Latency Safety Framework for Generative AI** addresses the trade-off between rigorous safety checks and response latency through a modular architecture: - **Policy specification:** Declarative, human-readable rules separate policies from model parameters, allowing legal or policy teams to control behavior. - **Risk sensing and scoring:** Asynchronous detectors combine lexical rules, semantic similarity, and lightweight classifiers. Early exit blocks obvious attacks quickly while allowing domain-specific calibration. - **Intervention router:** A deterministic controller chooses whether to allow, rewrite, or reject an interaction based on policies and risk scores. - **Monitoring and memory:** Lightweight records preserve decisions and rejection reasons for predictability and auditing. - **Evaluation and evolution:** Red-team recipes and automated vulnerability testing help the system adapt to new attack methods. The framework supports defense in depth without running every expensive safety mechanism sequentially. Its modularity, auditing capabilities, and continuous-evaluation loop make it suitable for production environments. ## Turning Governance Policies into Code **Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agents** converts informal organizational materials into runtime-enforceable controls. - The framework analyzes sources such as PRDs, technical design documents, regulations, and source code. - It builds a **source-linked policy tree** connecting individual rules to their original documents. - The policies are compiled into lightweight prompt-based classifiers. - When an agent rejects a request, the system can trace the decision back to its legal or organizational basis. - The approach helps enforce: - Least-privilege access. - Data minimization. - Restrictions on out-of-scope tasks. - Protection against prompt injection. - It may be especially valuable in regulated industries such as finance and healthcare, where frequently changing policies create substantial technical debt. ## Multimodal Safety and VLM Reasoning Vision-language models create new safety challenges because harmful meaning can emerge from interactions between images and text. **GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning** trains models to reason about combined modalities rather than classifying each input independently. - It addresses cases where harmless text obscures harmful visual content, such as an image of a bloodied knife paired with “cooking.” - Its GRPO-based training process includes: - **Safety-aware data concatenation** to create difficult examples containing hidden or mixed harmful content. - **Dynamic clipping** that encourages exploration early in training and tighter refinement later. - **Length-aware safety rewards** that reward concise conclusions supported by reasoning. - The method aims to detect subtle harms such as hate speech hidden in memes and visual metaphors. ## Hidden Vulnerabilities in Multimodal Training Data **VLMs can Aggregate Scattered Training Patches** demonstrates that filtering training images may not be sufficient. - A harmful image can be divided into individually innocuous patches and included in training. - A VLM may reconstruct the harmful concept by associating patches that share the same text label. - The paper calls this behavior **visual stitching**, related to cross-sample reasoning and inductive out-of-context reasoning. - Text labels such as “safe” or “unsafe” can help the model connect fragmented visual information and infer the original image-level meaning. - This suggests that safety evaluations must inspect not only final outputs but also: - Input-processing pipelines. - Cross-sample interactions. - Internal or latent representations. The available article ends while introducing research on distorted safety perception, so that section cannot be summarized further from the provided text. In practice, organizations should combine modular, low-latency enforcement with traceable policy management and multimodal evaluations that test hidden interactions—not just obvious harmful prompts or images.

Read original(opens in new tab)

Large Language ModelsDatabase DesignMachine LearningGenerative AI+4

line·Mar 6, 2026·3 min readCurated summary

On-Device Image Model Training for Mess

This post describes an on-device image captioning system for mobile messenger apps. Because autoregressive vision-language models took more than five seconds to generate captions, the team replaced them with a non-autoregressive decoder, reducing latency to roughly 200–400 ms. They then used LLM-based acceptance evaluation, caption re-generation, and multi-stage knowledge distillation to improve quality while keeping the model at 172 MB. ## Why Conventional Captioning Was Unsuitable - Models such as BLIP-2, MobileVLM, PaliGemma, and MiniCPM were too large or slow for mobile deployment. - BLIP-1 was selected as a practical baseline because of its smaller size and clear licensing, but still required more than five seconds after quantization. - Autoregressive decoding generates tokens sequentially, requiring one decoder pass per token. - On a Samsung Galaxy Fold 4, the initial model required about 142 ms per token, or approximately 2.8 seconds for 20 tokens. - Mobile UX required stable latency in the hundreds of milliseconds, including cold-start and variable-device conditions, so simple model compression was insufficient. ## Non-Autoregressive Caption Generation - The system predicts all caption tokens in parallel using a fixed set of learnable query tokens. - This changes the decoding cost from roughly O(T) for autoregressive generation to near O(1) through parallel processing. - The architecture consists of: - An image encoder reused from the previous system - Image embeddings injected as a prefix, following the ClipCap approach - A 66.4-million-parameter Transformer-based text decoder - Twenty learnable query tokens for short captions - Query-CTC loss addresses the alignment problem caused by predicting tokens simultaneously. - The resulting model generated captions in about 200 ms, achieving the required speed improvement. ## Speed Improved, but Caption Quality Declined - Standard metrics such as CIDEr and CLIPScore appeared acceptable. - Manual inspection revealed frequent: - Repeated words, such as “a desk with a computer on a desk” - Spelling errors, such as “a people ons” - Grammatical problems - Incomplete captions, such as “a” - These defects made the model unsuitable for direct use in a messaging product. ## LLM-Based Acceptance Evaluation - The team introduced an “accept ratio” based on GPT-4o mini judgments. - Captions were classified as either `accept` or `non-accept`. - The evaluation checked for: - Duplicate content - Errors - Clarity and grammatical correctness - This better reflected production usability than conventional image-captioning benchmarks. - The low acceptance rate confirmed that CIDEr and CLIPScore alone could not measure whether captions were appropriate for users. ## Data Quality and Knowledge Distillation - Analysis showed that the training data contained inconsistent and noisy captions: - A mixture of very short and overly long descriptions - Unnecessary OCR-like attempts to describe text in images - Uneven language quality - The smaller 66.4-million-parameter model also had less representational capacity than BLIP-1’s 110 million parameters. - Generating an entire sentence in one pass was especially difficult for the compact non-autoregressive model. - The training pipeline was redesigned as an iterative quality-improvement loop: - Train a baseline using the original data - Identify failures with LLM-based acceptance evaluation - Re-caption poor-quality training examples - Distill knowledge from a larger teacher model into the student model - Replace or refine rejected samples and repeat - Architectural scaling and metric optimization did not consistently improve acceptance rates, while re-captioning and knowledge distillation produced more meaningful gains. The practical recommendation is to design on-device captioning around the actual product experience: prioritize parallel decoding for latency, measure quality with production-oriented acceptance criteria, and use carefully curated data plus knowledge distillation to make compact models reliable.

Read original(opens in new tab)

Large Language ModelsDatabase DesignKnowledge DistillationNon Autoregressive+3

line·Jan 14, 2026Original article

Building an Enterprise LLM (opens in new tab)

LY Corporation’s engineering team developed an AI assistant for their private cloud platform, Flava, by prioritizing "context engineering" over traditional prompt engineering. To manage a complex environment of 260 APIs and hundreds of technical documents, they implemented a strategy of progressive disclosure to ensure the LLM receives only the most relevant information for any given query. This approach allows the assistant to move beyond simple RAG-based document summarization to perform active diagnostics and resource management based on real-time API data. ### Performance Limitations of Long Contexts * Research indicates that LLM performance can drop by 13.9% to 85% as context length increases, even if the model technically supports a large token window. * The phenomenon of "context rot" occurs when low-quality or irrelevant information is mixed into the input, causing the model to generate confident but incorrect answers. * Because LLMs are stateless, maintaining conversation history and processing dense JSON responses from multiple APIs quickly exhausts context windows and degrades reasoning quality. ### Progressive Disclosure and Tool Selection * The system avoids loading all 260+ API definitions at once; instead, it analyzes the user's intent to select only the necessary tools, such as loading only Redis-related APIs when a user asks about a cluster. * Specific product usage hints, such as the distinction between private and CDN settings for Object Storage, are injected only when those specific services are invoked. * This phased approach significantly reduces token consumption and prevents the model from being overwhelmed by irrelevant technical specifications. ### Response Guidelines and the "Mock Tool Message" Strategy * The team distinguished between "System Prompts" (global rules) and "Response Guidelines" (situational instructions), such as directing users to a console UI before suggesting CLI commands. * Injecting specific guidelines into the system prompt often caused "instruction conflict," where the LLM might hallucinate information to satisfy a guideline while ignoring core requirements like using search tools. * To resolve these conflicts, the team utilized "ToolMessages" to inject guidelines; by formatting instructions as if they were results from a tool execution, the LLM treats the information as factual context rather than a command that might override the system prompt. To build a robust enterprise LLM service, developers should focus on dynamic context management rather than static prompt optimization. Treating operational guidelines as external data via mock tool messages, rather than system instructions, provides a scalable way to reduce hallucinations and maintain high performance across hundreds of integrated services.

Large Language ModelsAIPrompt EngineeringRAG+4

line·Dec 17, 2025Original article

Safety is a Given, Cost (opens in new tab)

AI developers often rely on system prompts to enforce safety rules, but this integrated approach frequently leads to "over-refusal" and unpredictable shifts in model performance. To ensure both security and operational efficiency, it is increasingly necessary to decouple safety mechanisms into separate guardrail systems that operate independently of the primary model's logic. ## Negative Impact on Model Utility * Integrating safety instructions directly into system prompts often leads to a high False Positive Rate (FPR), where the model rejects harmless requests alongside harmful ones. * Technical analysis using Principal Component Analysis (PCA) reveals that guardrail prompts shift the model's embedding results in a consistent direction toward refusal, regardless of the input's actual intent. * Studies show that aggressive safety prompting can cause models to refuse benign technical queries—such as "how to kill a Python process"—because the model adopts an overly conservative decision boundary. ## Positional Bias and Context Neglect * Research on the "Lost in the Middle" phenomenon indicates that LLMs are most sensitive to information at the beginning and end of a prompt, while accuracy drops significantly for information placed in the center. * The "Constraint Difficulty Distribution Index" (CDDI) demonstrates that the order of instructions matters; models generally follow instructions better when difficult constraints are placed at the beginning of the prompt. * In complex system prompts where safety rules are buried in the middle, the model may fail to prioritize these guardrails, leading to inconsistent safety enforcement depending on the prompt's structure. ## The Butterfly Effect of Prompt Alterations * Small, seemingly insignificant changes to a system prompt—such as adding a single whitespace, a "Thank you" note, or changing the output format to JSON—can alter more than 10% of a model's predictions. * Modifying safety-related lines within a unified system prompt can cause "catastrophic performance collapse," where the model's internal reasoning path is diverted, affecting unrelated tasks. * Because LLMs treat every part of the prompt as a signal that moves their decision boundaries, managing safety and task logic in a single string makes the system brittle and difficult to iterate upon. To build robust and high-performing AI applications, developers should move away from bloated system prompts and instead implement external guardrails. This modular approach allows for precise security filtering without compromising the model's creative or logical capabilities.

Large Language ModelsAIPrompt EngineeringPrompt Injection+4

line·Nov 4, 2025Original article

Security Threat Cases and Countermeasures (opens in new tab)

Developing AI products introduces unique security vulnerabilities that extend beyond traditional software risks, ranging from package hallucinations to sophisticated indirect prompt injections. To mitigate these threats, organizations must move away from trusting LLM-generated content and instead implement rigorous validation, automated threat modeling, and input/output guardrails. The following summary details the specific risks and mitigation strategies identified by LY Corporation’s security engineering team. ## Slopsquatting and Package Hallucinations - AI models frequently hallucinate non-existent library or package names when providing coding instructions (e.g., suggesting `huggingface-cli` instead of the correct `huggingface_hub[cli]`). - Attackers exploit this by registering these hallucinated names on public registries to distribute malware to unsuspecting developers. - Mitigation requires developers to manually verify all AI-suggested commands and dependencies before execution in any environment. ## Prompt Injection and Arbitrary Code Execution - As seen in CVE-2024-5565 (Vanna AI), attackers can inject malicious instructions into prompts to force the application to execute arbitrary code. - This vulnerability arises when developers grant LLMs the autonomy to generate and run logic within the application context without sufficient isolation. - Mitigation involves treating LLM outputs as untrusted data, sanitizing user inputs, and strictly limiting the LLM's ability to execute system-level commands. ## Indirect Prompt Injection in Integrated AI - AI assistants integrated into office environments (like Gemini for Workspace) are susceptible to indirect prompt injections hidden within emails or documents. - A malicious email can contain "system-like" instructions that trick the AI into hiding content, redirecting users to phishing sites, or leaking data from other files. - Mitigation requires the implementation of robust guardrails that scan both the input data (the content being processed) and the generated output for instructional anomalies. ## Permission Risks in AI Agents and MCP - The use of Model Context Protocol (MCP) and coding agents creates risks where an agent might overstep its intended scope. - If an agent has broad access to a developer's environment, a malicious prompt in a public repository could trick the agent into accessing or leaking sensitive data (such as salary info or private keys) from a private repository. - Mitigation centers on the principle of least privilege, ensuring AI agents are restricted to specific, scoped directories and repositories. ## Embedding Inversion and Vector Store Vulnerabilities - Attacks targeting the retrieval phase of RAG (Retrieval-Augmented Generation) systems can lead to data leaks. - Embedding Inversion techniques may allow attackers to reconstruct original sensitive text from the vector embeddings stored in a database. - Securing AI products requires protecting the integrity of the vector store and ensuring that retrieved context does not bypass security filters. ## Automated Security Assessment Tools - To scale security, LY Corporation is developing internal tools like "ConA" for automated threat modeling and "LAVA" for automated vulnerability assessment. - These tools aim to identify AI-specific risks during the design and development phases rather than relying solely on manual reviews. Effective AI security requires a shift in mindset: treat every LLM response as a potential security risk. Developers should adopt automated threat modeling and implement strict input/output validation layers to protect both the application infrastructure and user data from evolving AI-based exploits.

Large Language ModelsAIModel Context ProtocolAi Security+5

line·Oct 3, 2025Original article

A month-long project in (opens in new tab)

This blog post explores how LY Corporation reduced a month-long development task to just five days by leveraging "vibe coding" with Generative AI tools like ChatGPT and Cursor. By shifting from traditional, rigid documentation to an iterative, demo-first approach, developers can rapidly validate multiple UI/UX solutions for complex problems like restaurant menu registration. The author concludes that AI's ability to handle frequent re-work makes it more efficient to "build fast and iterate" than to aim for perfection through long-form specifications. ### Strategic Shift to Rapid Prototyping * Traditional development cycles (spec → design → dev → fix) are often too slow to keep up with market trends due to heavy documentation and impact analysis. * The "vibe coding" approach prioritizes creating "working demos" over perfect specifications to find "good enough" answers through rapid feedback loops. * AI reduces the psychological and logistical burden of "starting over," allowing developers to refine the context and quality of outputs through repeated interaction without the friction of manual re-documentation. ### Defining Requirements and Solution Ideation * Initial requirements are kept minimal, focusing only on the core mission, top priorities, and essential data structures (e.g., product name, image, description) to avoid limiting AI creativity. * ChatGPT is used to generate a wide range of solution candidates, which are then filtered into five distinct approaches: Stepper Wizards, Live Previews with Quick Add, Template/Cloning, Chat Input, and OCR-based photo scanning. * This stage emphasizes volume and variety, using AI-generated pros and cons to establish selection criteria and identify potential UX bottlenecks early in the process. ### Detailed Design and Multi-Solution Wireframing * Each of the five chosen solutions is expanded into detailed screen flows and UI elements, such as progress bars, bottom sheets, and validation logic. * Prompt engineering is used iteratively; if an AI-generated result lacks a specific feature like "temporary storage" or "mandatory field validation," the prompt is adjusted to regenerate the design instantly. * The focus remains on defining the "what" (UI elements) and "how" (user flow) through textual descriptions before moving to actual coding. ### Implementation with Cursor and Flutter * Cursor is utilized to generate functional code based on the refined wireframes, using Flutter as the framework to ensure rapid cross-platform development for both iOS and Android. * The development follows a "skeleton-first" approach: first creating a main navigation hub with five entry points, then populating each individual solution module one by one. * Technical architecture decisions, such as using Riverpod for state management or SQLite for data storage, are layered onto the demo post-hoc, reversing the traditional "stack-first" development order to prioritize functional validation. ### Recommendation To maximize efficiency, developers should treat AI as a partner for high-speed iteration rather than a one-shot tool. By focusing on creating functional demos quickly and refining them through direct feedback, teams can bypass the bottlenecks of traditional software requirements and deliver user-centric products in a fraction of the time.

Large Language ModelsAIGenerative AIPrompt Engineering+5

line·Oct 3, 2025Original article

IUI 202 (opens in new tab)

The IUI 2025 conference highlighted a significant shift in the AI landscape, moving away from a sole focus on model performance toward "human-centered AI" that prioritizes collaboration, ethics, and user agency. The prevailing consensus across key sessions suggests that for AI to be sustainable and trustworthy, it must transcend simple automation to become a tool that augments human perception and decision-making through transparent, interactive, and socially aware design. ## Reality Design and Human Augmentation The concept of "Reality Design" suggests that Human-Computer Interaction (HCI) research must expand beyond screen-based interfaces to design reality itself. As AI, sensors, and wearables become integrated into daily life, technology can be used to directly augment human perception, cognition, and memory. * Memory extension: Systems can record and reconstruct personal experiences, helping users recall details in educational or professional settings. * Sensory augmentation: Technologies like selective hearing or slow-motion visual playback can enhance a user's natural observational powers. * Cognitive balance: While AI can assist with task difficulty (e.g., collaborative Lego building), designers must ensure that automation does not erode the human will to learn or remember, echoing historical warnings about technology-induced "forgetfulness." ## Bridging the Socio-technical Gap in AI Transparency Transparency in AI, particularly for high-risk areas like finance or medicine, should not be limited to showing mathematical model weights. Instead, it must bridge the gap between technical complexity and human understanding by focusing on user goals and social contexts. * Multi-faceted communication: Effective transparency involves model reporting (Model Cards), sharing safety evaluation results, and providing linguistic or visual cues for uncertainty rather than just numerical scores. * Counterfactual explanations: Users gain better trust when they can see how a decision might have changed if specific input conditions were different. * Interaction-based transparency: Transparency must be coupled with control, allowing users to act as "adjusters" who provide feedback that the model then reflects in its future outputs. ## Interactive Machine Learning and Human-in-the-Loop The framework of Interactive Machine Learning (IML) challenges the traditional view of AI as a static black box trained on fixed data. Instead, it proposes an interactive loop where the user and the model grow together through continuous feedback. * User-driven training: Users should be able to inspect model classifications, correct errors, and have those corrections immediately influence the model's learning path. * Beyond automation: This approach reframes AI from a replacement for human labor into a collaborative partner that adapts to specific user behaviors and professional expertise. * Impact on specialized tools: Modern applications include educational platforms where students manipulate data directly and research tools that integrate human intuition into large-scale data analysis. ## Collaborative Systems in Specialized Professional Contexts Practical applications of human-centered AI are being realized in sensitive fields like child counseling, where AI assists experts without replacing the human element. * Counselor-AI transcription: Systems designed for counseling analysis allow AI to handle the heavy lifting of transcription while counselors manage the nuance and contextual editing. * Efficiency through partnership: By focusing on reducing administrative burdens, these systems enable professionals to spend more time on high-level cognitive tasks and emotional support, demonstrating the value of AI as a supportive infrastructure. The future of AI development requires moving beyond isolated technical optimization to embrace the complexity of the human experience. Organizations and developers should focus on creating systems where transparency is a tool for "appropriate trust" and where design is focused on empowering human capabilities rather than simply automating them.

Large Language ModelsAIHuman In The LoopHuman Centered Ai+5

line·Aug 12, 2025Original article

The Current State of LY Corporation (opens in new tab)

Tech-Verse 2025 showcased LY Corporation’s strategic shift toward an AI-integrated ecosystem following the merger of LINE and Yahoo Japan. The event focused on the practical hurdles of deploying generative AI, concluding that the transition from experimental models to production-ready services requires sophisticated evaluation frameworks and deep contextual integration into developer workflows. ## AI-Driven Engineering with Ark Developer LY Corporation’s internal "Ark Developer" solution demonstrates how AI can be embedded directly into the software development life cycle. * The system utilizes a Retrieval-Augmented Generation (RAG) based code assistant to handle tasks such as code completion, security reviews, and automated test generation. * Rather than treating codebases as simple text documents, the tool performs graph analysis on directory structures to maintain structural context during code synthesis. * Real-world application includes a seamless integration with GitHub for automated Pull Request (PR) creation, with internal users reporting higher satisfaction compared to off-the-shelf tools like GitHub Copilot. ## Quantifying Quality in Generative AI A significant portion of the technical discussion centered on moving away from subjective "vibes-based" assessments toward rigorous, multi-faceted evaluation of AI outputs. * To measure the quality of generated images, developers utilized traditional metrics like Fréchet Inception Distance (FID) and Inception Score (IS) alongside LAION’s Aesthetic Score. * Advanced evaluation techniques were introduced, including CLIP-IQA, Q-Align, and Visual Question Answering (VQA) based on video-language models to analyze image accuracy. * Technical challenges in image translation and inpainting were highlighted, specifically the difficulty of restoring layout and text structures naturally after optical character recognition (OCR) and translation. ## Global Technical Exchange and Implementation The conference served as a collaborative hub for engineers across Japan, Taiwan, and Korea to discuss the implementation of emerging standards like the Model Context Protocol (MCP). * Sessions emphasized the "how-to" of overcoming deployment hurdles rather than just following technical trends. * Poster sessions (Product Street) and interactive Q&A segments allowed developers to share localized insights on LLM agent performance and agentic workflows. * The recurring theme across diverse teams was that the "evaluation and verification" stage is now the primary driver of quality in generative AI services. For organizations looking to scale AI, the key recommendation is to move beyond simple implementation and invest in "evaluation-driven development." By building internal tools that leverage graph-based context and quantitative metrics like Aesthetic Scores and VQA, teams can ensure that generative outputs meet professional service standards.

Large Language ModelsAIModel Context ProtocolRAG+5

line·Jun 13, 2025Original article

AI and Writer's Partnership (opens in new tab)

LY Corporation is addressing the chronic shortage of high-quality technical documentation by treating the problem as an engineering challenge rather than a training issue. By utilizing Generative AI to automate the creation of API references, the Document Engineering team has transitioned from a "manual craftsmanship" approach to an "industrialized production" model. While the system significantly improves efficiency and maintains internal context better than generic tools, the team concludes that human verification remains essential due to the high stakes of API accuracy. ### Contextual Challenges with Generic AI Standard coding assistants like GitHub Copilot often fail to meet the specific documentation needs of a large organization. * Generic tools do not adhere to internal company style guides or maintain consistent terminology across projects. * Standard AI lacks awareness of internal technical contexts; for example, generic AI might mistake a company-specific identifier like "MID" for "Member ID," whereas the internal tool understands its specific function within the LY ecosystem. * Fragmented deployment processes across different teams make it difficult for developers to find a single source of truth for API documentation. ### Multi-Stage Prompt Engineering To ensure high-quality output without overwhelming the LLM's "memory," the team refined a complex set of instructions into a streamlined three-stage workflow. * **Language Recognition:** The system first identifies the programming language and specific framework being used. * **Contextual Analysis:** It analyzes the API's logic to generate relevant usage examples and supplemental technical information. * **Detail Generation:** Finally, it writes the core API descriptions, parameter definitions, and response value explanations based on the internal style guide. ### Transitioning to Model Context Protocol (MCP) While the prototype began as a VS Code extension, the team shifted to using the Model Context Protocol (MCP) to ensure the tool was accessible across various development environments. * Moving to MCP allows the tool to support multiple IDEs, including IntelliJ, which was a high-priority request from the developer community. * The MCP architecture decouples the user interface from the core logic, allowing the "host" (like the IDE) to handle UI interactions and parameter inputs. * This transition reduced the maintenance burden on the Document Engineering team by removing the need to build and update custom UI components for every IDE. ### Performance and the Accuracy Gap Evaluation of the AI-generated documentation showed strong results, though it highlighted the unique risks of documenting APIs compared to other forms of writing. * Approximately 88% of the AI-generated comments met the team's internal evaluation criteria. * The specialized generator outperformed GitHub Copilot in 78% of cases regarding style and contextual relevance. * The team noted that while a 99% accuracy rate is excellent for a blog post, a single error in a short API reference can render the entire document useless for a developer. To successfully implement AI-driven documentation, organizations should focus on building tools that understand internal business logic while maintaining a strict "human-in-the-loop" workflow. Developers should use these tools to generate the bulk of the content but must perform a final technical audit to ensure the precision that only a human author can currently guarantee.

Large Language ModelsAIGenerative AIModel Context Protocol+4