toss·Aug 11, 2026·5 min readCurated summary

Toss’s Speed and Quality: Are Commercial Tools Enough? — Tossion

Toss’s QA Platform team built Tossion to replace a rigid commercial test case management system with a platform they could continuously adapt. It unifies test cases, manual and automated results, evidence, history, and release snapshots while preserving a clear record of what was tested at each point in time. The platform later expanded into AI-assisted PR analysis, test case generation, and real-device regression testing, enabling QA processes to evolve at Toss’s speed. ## Why Tossion Was Needed - Previous QA information was scattered across automation results, manual test results, test cases, and decision evidence. - Answering questions such as “How did this test perform last time?” required checking multiple systems. - The commercial TCM could not be modified quickly, and requests for new features often stalled. - Toss needed a platform that could be changed directly as new QA requirements emerged. ## Preserving Test History - Tossion organizes work as: - Project - Suite - Section - Test Case - Test cases continue to change as products evolve, but test runs must preserve historical reality. - When a test run is created, it copies the test case’s assignee, steps, and description instead of merely referencing the current test case. - Status changes create history entries showing who made the decision, when, and against which version. - Closed test runs store snapshots of test cases, comments, and automation results, so reports remain available even if the original test cases are later deleted. - Manually selected test cases override Type or Platform filters. ## Collaborative Test Execution - Test runs progress through `Active → Completed → Closed`. - Assignee-based charts show each person’s remaining work at a glance. - Fields such as Status, Type, Assignee, Version, Platform, RNR, and History are added or removed based on actual usage. - Multiple users can work in the same test run simultaneously: - Active viewers appear as avatars. - Editing locks prevent conflicts. - Locks are automatically released when users leave or disconnect. - Status changes are synchronized without requiring a refresh. - The main benefit is short feedback loops: requested improvements can be built and deployed immediately. ## AI-Assisted Release and PR Analysis - Toss planned to use AI for test case creation, PR analysis, regression automation, and execution. - Tossion analyzes every PR included in a release, separating those with QA labels from those marked as not requiring QA. - The goal is not merely to summarize changes, but to audit whether “no QA needed” classifications are correct. - An agent running on a QA server: - Registers with Tossion. - Polls for work. - Executes the AI already authenticated on the server. - Hundreds of PRs are divided into smaller batches and analyzed in parallel for deeper review. - Results are checked for vague or unusable content, such as: - Missing screens or conditions. - Repeated PR titles. - Raw function names. - Missing reproduction steps, expected results, failure symptoms, or reasoning. - Invalid analyses are retried, and the number of merged PRs is compared with the number of analyzed PRs to detect omissions. - Historical incident data increases risk when a new PR touches files associated with past outages. - Results are uploaded after each batch, allowing interrupted jobs to resume without repeating completed analysis. - The resulting “must-test” list defines the scope of the Sprint test run. ## AI-Generated Test Cases - AI generates test cases for new features, but Tossion controls their placement in the test case tree. - AI returns paths such as `Assets > Account Connection > Select Bank`; Tossion converts them into actual sections, reusing existing sections or creating missing ones. - Reliability is checked in three layers: - AI reviews its own output for missing branches, error cases, and boundary values. - Scripts validate naming, numbering, screen coverage, and requirements coverage. - A separate AI creates an independent test plan based on ISTQB and ISO/IEC 29119 practices. - The plan and generated cases are compared: - Planned but missing cases indicate omissions. - Cases outside the plan indicate unplanned scope. - This catches state-based scenarios that screen-oriented test generation might miss. - Generated cases are uploaded through a portable CLI rather than direct UI integration, reducing dependency on local packages, runtimes, and paths. - Stable cases can later become regression tests. ## Running Regression Tests on Real Devices - A Tossion execution modal specifies: - Device - Build - Test scope - Target test run - Runners connected to Android and iOS physical devices register themselves with Tossion but require administrator approval before receiving work. - Runners report device status every 30 seconds. - Tossion supplies the correct build for installation, ensuring results are tied to a known version. - Users can run the full regression suite or a selected section. - Progress is streamed as scenarios finish, including duration and failure messages. - Results are stored at the step level: - Status - Duration - Error message - Screenshot - Scenario-level video - Automation results can be attached directly to the relevant Sprint test run, making them part of the same record used for manual testing. ## Linking Automation Results to Test Cases - Aggregate reports such as “200 regression tests, 3 failures” do not identify which manual test case rows were covered. - Tossion aims to connect test cases and automation in both directions: - Generate automation code from test cases. - Write automation results back into individual test cases. - This removes the need for QA engineers to manually reconcile separate reports with test case lists. Tossion’s central value is not just test management, but ownership and adaptability. By combining immutable test history, collaborative execution, AI validation, and real-device automation in one extensible platform, Toss’s QA team can adjust its tools and processes as quickly as product requirements change.

Read original(opens in new tab)

CliAIAndroidiOS+3

cloudflare·Jul 27, 2026·3 min readCurated summary

We’re open sourcing our privacy proxy CLI

Oblivious HTTP (OHTTP) is difficult to debug because requests pass through multiple parties, use binary HTTP encoding, and depend on several RFC-defined cryptographic steps. Cloudflare created and open-sourced `pvcli`, a CLI that simplifies testing and troubleshooting privacy protocols by exposing each stage of the process. Released under Apache-2.0, the tool is designed for production-scale debugging and community contributions. ## Why OHTTP Debugging Is Complex - OHTTP ensures that no single party can know both the client’s identity and the requested content. - It relies on two non-colluding servers: - A **relay** that sees the client but not the request contents. - A **gateway** that decrypts the request but does not see the client’s identity. - A typical request proceeds through several stages: - The client retrieves the gateway’s public key. - The client encrypts the request and sends it to the relay. - The relay forwards the anonymized request to the gateway. - The gateway decrypts it and contacts the target. - The response travels back through the gateway and relay before being decrypted by the client. - Every stage can introduce failures, making it difficult to identify whether the problem lies with the client, relay, gateway, or target. ## Why Cloudflare Built `pvcli` - Privacy products such as Privacy Proxy and Privacy Gateway introduced increasing operational complexity and customer-specific requirements. - Engineers frequently had to create one-off clients to test customer deployments. - Diagnosing failures required determining which protocol step had failed and which party was responsible. - OHTTP’s binary formats made manual inspection especially error-prone. - `pvcli` consolidates privacy-protocol functionality into one familiar CLI with support for different protocols and architectures. ## Debugging OHTTP Manually - Engineers first fetch the gateway’s public key, receiving a long hexadecimal binary payload. - They must manually parse fields according to RFC 9458, including: - The key entry length. - The public key identifier. - The asymmetric encryption method, such as DHKEM with X25519 and HKDF-SHA256. - The gateway’s public key. - Supported symmetric encryption algorithms, such as HKDF-SHA256 and AES-128-GCM. - The original HTTP request must then be converted into binary HTTP according to RFC 9292. - Engineers manually verify encoded fields such as: - The request method (`POST`). - The HTTPS scheme. - The target hostname. - The request path and headers. - The JSON body. - Finally, they need custom scripts to encrypt the binary request and construct the OHTTP wrapper request. ## Using `pvcli` - A complete OHTTP request can be issued with a single command: ```bash pvcli --ohttp \ --first-hop https://relay-cloudflare.ohttp.info \ --proxy https://gateway.ohttp.info \ -X POST \ --header "content-type: application/json" \ --data '{"test":1}' \ https://target.ohttp.info/anything ``` - The tool handles the relay, gateway, encryption, binary HTTP encoding, and target request flow. - It provides a clearer view of each protocol step, replacing manual hexadecimal parsing and bespoke scripts with a repeatable debugging workflow. `pvcli` is a practical way to test live OHTTP deployments, isolate failures across the relay and gateway chain, and reduce the risk of mistakes when inspecting binary protocol data.

Read original(opens in new tab)

CliCryptographyOblivious HttpBinary Http+3

toss·Jun 23, 2026·4 min readCurated summary

5. Technical Writer, A Decision to Disappear

Toss’s technical writing team argues that documentation is essential context for AI, but manually maintaining thousands of documents is impossible with only three technical writers serving roughly 4,000 people. Their solution is to automate the technical writer’s work by teaching AI the team’s implicit standards and embedding those standards into reusable Skills. The initial system supported document creation and review, but adoption remained low because users still had to install, invoke, and supply information to the AI manually. ## Why Toss Wanted to Automate Technical Writing - Documentation gives AI the organizational context it needs to work effectively. - Toss has approximately 4,000 employees but only three technical writers. - Reviewing documents individually does not scale, especially in a fast-moving organization where features change or disappear before documentation is complete. - The team’s goal to “eliminate technical writers” means transferring routine writing and editing work to AI, not abandoning documentation quality. ## Teaching AI Technical Writing Principles - The team analyzed existing technical writing review comments to identify how writers evaluate documents. - Existing writing guidelines were converted into explicit principles, such as: - Focus each page on one subject. - Present value before implementation details. - Each principle was supplemented with incorrect and correct examples so AI would understand the intent rather than apply rules mechanically. - Common document types were converted into templates. - Templates include: - Instructions explaining what each section should contain. - `(required)` markers for information that must not be omitted. - For example, an ADR template requires an overview, context, considered alternatives, decision, and rationale, while also allowing optional sections such as expected outcomes and related references. ## Skill for Writing New Documents The document-writing Skill reproduces the four stages a technical writer typically follows: - **Clarify the purpose:** Ask about the project, document goal, audience, level of detail, source materials, and expected structure. - **Design the structure:** Use a standard structure or select a relevant template, such as onboarding guides, meeting notes, or PRDs. - **Write the content:** Apply technical writing and MDX rules while using templates as structural guidance. - **Review the draft:** Check for awkward wording, missing information, and other quality issues. The Skill also distinguishes between required and optional template sections: - Required sections remain in the draft even when source information is incomplete. - Missing information is represented with questions or comments rather than guesses. - Optional sections are omitted when there is not enough source material to complete them. ## Skill for Reviewing and Improving Documents - The team initially converted past review comments into a checklist. - This produced poor results: AI overlooked important issues while generating unnecessary comments. - The problem was that good writing follows relatively stable principles, whereas bad writing can fail in many different ways. - The revised workflow lets AI independently: - Read the technical writing principles. - Analyze the document. - Identify violations. - Explain the issue and suggest revised wording. - Perform a final checklist-based review. - Previous review comments are now used as examples of how principles apply, rather than as a rigid list of required findings. - One example principle requires descriptions of parameters or properties to include their meaning, accepted format, and usage example—not merely a type such as `date: string`. ## Low Adoption Revealed a Usability Problem - Despite creating both Skills, the team found that few employees used them. - Users still had to: - Download and install the Skill manually. - Understand CLI-based setup, which was unfamiliar to non-developers. - Remember to invoke the Skill whenever they began writing documentation. - Find and provide all relevant source materials themselves. - The team concluded that improving the AI’s capabilities was not enough; the workflow also had to reduce the effort required from users. The main lesson is that AI-based documentation succeeds only when organizational knowledge, writing principles, and templates are encoded clearly—and when the system is integrated into everyday work so employees do not have to remember to use it or prepare everything manually.

Read original(opens in new tab)

CliAIPrompt EngineeringTechnical Writing+3

cloudflare·Apr 13, 2026·3 min readCurated summary

Building a CLI for all of Cloudflare

Cloudflare is rebuilding Wrangler into a unified CLI for its entire platform, motivated by the growing role of coding agents in configuring and deploying Cloudflare applications. The technical preview, available as `npx cf` or the globally installed `cf` package, currently covers only a subset of products but is intended to support the full API surface. The effort depends on a new TypeScript-based schema system that can generate consistent commands, configuration, bindings, documentation, and agent-oriented interfaces. ## A CLI for all of Cloudflare - Cloudflare offers more than 100 products and nearly 3,000 HTTP API operations. - Agents increasingly use Cloudflare APIs to: - Build and deploy applications - Configure accounts - Query analytics and logs - Create agents and platforms - Cloudflare aims to expose its products consistently through: - CLI commands - Workers Bindings - SDKs - Configuration files - Terraform - Documentation and OpenAPI schemas - MCP servers and Agent Skills - The new Wrangler technical preview can be tried with: - `npx cf` - `npm install -g cf` - A broader internal version already supports the full Cloudflare API, with ongoing work to make command output useful for both humans and agents. ## A new schema and code-generation pipeline - Existing OpenAPI schemas already generate: - Cloudflare SDKs - The Terraform provider - The Code Mode MCP server - Other interfaces, including Wrangler commands, Workers Bindings, configuration, documentation, and Agent Skills, were previously maintained manually. - Manual synchronization was error-prone and could not scale to Cloudflare’s full product range. - OpenAPI alone is insufficient because it primarily describes REST APIs, while Cloudflare also needs to represent: - Interactive CLI workflows - Multiple local and remote actions - RPC-style Workers Bindings - Agent Skills and related documentation - Cloudflare therefore created a TypeScript schema format containing: - API definitions - CLI commands and arguments - Context required to generate different interfaces - Conventions, linting, and guardrails enforce consistency while allowing the schema to generate OpenAPI and future interfaces. ## Consistency for agents and humans - Agents depend on predictable command names and flags. Inconsistent syntax can cause them to call commands that do not exist. - Cloudflare is enforcing conventions at the schema layer, including: - `get`, never `info` - `--force`, never `--skip-confirmations` - `--json`, never `--format` - Applying these rules across interfaces avoids discrepancies between the CLI, REST APIs, and SDKs. - Wrangler must also clearly distinguish local and remote resources. - This is especially important for D1, R2, and KV, where local simulation and remote bindings can coexist. - Clear defaults and output indicating whether an operation targets local or remote resources help agents avoid modifying the wrong environment. ## Local Explorer for simulated resources - Local Explorer is available in open beta through Wrangler and the Cloudflare Vite plugin. - It lets developers inspect locally simulated: - KV - R2 - D1 - Durable Objects - Workflows - Local resources use the same underlying API structure as Cloudflare’s remote APIs and Dashboard. - Cloudflare’s local development environment runs Workers APIs locally, including D1 backed by SQLite through Miniflare. - Previously, developers had to inspect `.wrangler/state` or use third-party tools to understand local data. - Local Explorer provides an interface showing: - Which bindings are attached to a Worker - What data those bindings contain - It can be opened with the `e` keyboard shortcut and helps developers or agents verify schemas, seed test data, and reset local databases. Cloudflare’s direction is to make Wrangler a consistent, machine-readable interface to the entire platform. The technical preview is early, but the new schema-driven system and Local Explorer establish the foundation for a CLI that is easier for both developers and coding agents to use safely.

Read original(opens in new tab)

CliDatabase DesignCloudflareTypeScript+4

aws·Mar 26, 2026·2 min readCurated summary

Customize your AWS Management Console experience with visual settings including account color, region and service visibility | Amazon Web Services

AWS has expanded User Experience Customization (UXC) to let administrators tailor the Management Console by account color, visible Regions, and visible services. These settings help teams distinguish accounts and reduce clutter by showing only relevant resources. They affect console appearance only and do not restrict access through the CLI, SDKs, APIs, or Amazon Q Developer. ## Account Color Customization - Administrators can assign a color to an AWS account through **Account display settings**. - The color appears in the console navigation bar to make account purpose easier to recognize. - Teams can use colors such as: - Orange for development - Light blue for testing - Red for production ## Region Visibility - Administrators can configure which AWS Regions appear in the console’s Region selector. - They can either show all available Regions or select a specific list. - After saving, only the chosen Regions appear in the navigation bar. - This reduces unnecessary scrolling and helps users focus on approved or relevant Regions. ## Service Visibility - Administrators can select which AWS services appear in the **All services** menu and console search results. - Services can be searched for or selected by category, such as Popular services. - Hidden services are removed from the console interface but remain accessible through programmatic tools. ## Programmatic Configuration - Account customization can be managed through the `AWS::UXC::AccountCustomization` CloudFormation resource. - The resource supports: - `AccountColor` - `VisibleServices` - `VisibleRegions` - Example configurations can expose services such as `s3`, `ec2`, and `lambda`, while limiting Regions to `us-east-1` and `us-west-2`. - Templates can be deployed with the AWS CLI using `aws cloudformation deploy`. AWS administrators can use these settings to create a clearer, more focused console experience without changing permissions or underlying account access.

Read original(opens in new tab)

CliAWSAPICloudformation+3