Airbnb/graphql

4 posts

airbnb

My Journey to Airbnb — Anna Sulkina (opens in new tab)

Anna Sulkina’s career journey moved from hardware diagnostics and frontend development into backend infrastructure and engineering leadership. Her experiences at Twitter taught her to design distributed systems for failure and to build consensus around transformative technologies like GraphQL. She joined Airbnb in 2022 because it aligned her passion for travel with an opportunity to strengthen developer infrastructure, organizational strategy, and engineering collaboration. ## Discovering Technology in Post-Soviet Ukraine - Sulkina grew up in Eastern Ukraine as the Soviet Union collapsed. - Her older brother introduced her to computers by bringing home hardware components and assembling a machine that loaded programs from a cassette player. - Seeing how individual components formed a working system inspired her to pursue technology. ## Learning English While Building Technical Skills - She studied programming at a Ukrainian university before immigrating to the United States. - Although she understood written English and knew how to program, communicating in English was initially more difficult than learning programming languages. - She took ESL classes while studying C++ and Java through Berkeley Extension. - Her first job was in hardware diagnostics at a five-person company. - A language barrier caused her to run out of time on a technical interview, but an interviewer familiar with her Berkeley class gave her another opportunity. - She eventually transitioned from C++ to Java, which became her primary language for many years. ## Moving Down the Stack and Into Leadership - Sulkina’s career progressed from hardware diagnostics to frontend, backend, and infrastructure engineering. - At the same time, she increasingly took on leadership responsibilities. - At Caymas Systems, her manager recognized her leadership potential and showed her the difference effective leadership makes. - At Comcast, she moved from individual contributor to engineering manager. - Coaching engineers, building software collaboratively, and developing high-performing teams convinced her that leadership was the right path. ## Lessons from Twitter’s Distributed Systems - During nearly nine years at Twitter, Sulkina advanced from first-line manager to director. - She worked through major operational events, including the “fail whale” period and the tweetstorm surrounding Ellen DeGeneres’s viral selfie. - Twitter’s transition from a monolith to microservices taught her that failure is inevitable in complex systems. - Resilient distributed systems must be designed to handle failures rather than assuming failures can be prevented. - Her cultural lesson involved turning promising ideas into adopted technologies. - She helped bootstrap Twitter’s GraphQL API, replacing legacy REST services. - The effort required leadership support, cross-team consensus, and stakeholder alignment, but ultimately improved product teams’ development velocity. ## Choosing Airbnb - Airbnb contacted Sulkina in 2022, when she felt ready to move beyond a well-established organization at Twitter. - The company appealed to her because it combined her professional interests with her personal passion for travel; she had been an Airbnb guest since 2013. - Airbnb’s Developer Platform organization had strong work happening in separate silos but needed clearer strategy, direction, and trust across engineering. - Sulkina began by clarifying the organization’s purpose and future direction. - Her early priorities included strengthening the organization, coaching leaders, and creating alignment within the team and with the teams it supported. - Over the following years, this work produced a high-performing organization with clearer strategy, stronger execution, and a focus on delivering business value. Sulkina’s story emphasizes that technical growth, organizational leadership, and personal motivation can reinforce one another. Her experience suggests that successful engineering leaders design for failure, invest in alignment, and use clear strategy to turn fragmented efforts into meaningful platform-wide impact.

airbnb

GraphQL Data Mocking at Scale with LLMs and @generateMock (opens in new tab)

Airbnb’s `@generateMock` directive combines GraphQL schemas, product context, design references, and LLMs to generate realistic, type-safe mock data automatically. Integrated into the existing Niobe code-generation workflow, it reduces manual mock maintenance and helps client engineers prototype and test features before backend implementation is complete. ## Challenges with GraphQL Mocking - Manually creating large JSON responses or schema-generated objects is tedious and error-prone. - Client engineers often hardcode data or modify networking logic when the server is not yet ready, slowing frontend development. - Handwritten mocks drift out of sync as queries and schemas evolve. - Random generators and field-level resolvers lack the domain knowledge needed for convincing, meaningful data. ## Airbnb’s Goals - Eliminate hand-written mock data and ongoing maintenance. - Generate realistic data suitable for demos, snapshots, and tests. - Keep engineers in their normal local development workflow without requiring separate tools or repositories. ## The `@generateMock` Directive - Engineers can add `@generateMock` to GraphQL operations, fragments, or fields. - Optional arguments customize the generated data: - `id` identifies a mock and names generated helper functions. - `hints` provide instructions such as destinations, content, or desired density. - `designURL` links to a design mockup so generated names, addresses, and other values better match the intended UI. - The directive can be repeated with different arguments to create multiple mock variations. ## Integration with Niobe - After adding or changing `@generateMock` in a `.graphql` file, engineers run Niobe just as they would for ordinary GraphQL code generation. - Niobe generates: - JSON files containing the mock responses. - TypeScript, Kotlin, or Swift helpers for consuming the mocks. - Generated functions return instantiated, type-safe model objects for demo apps, snapshot tests, and unit tests. - Engineers can edit the generated JSON manually; Niobe preserves those changes during later generation runs. ## Context Used by the LLM Niobe supplies the LLM with information needed to create realistic results: - The mocked operations, fragments, fields, and their dependencies. - The relevant subset of the GraphQL schema and inline documentation. - Only schema types and fields needed to resolve the query, avoiding unnecessary context-window usage. - A snapshot image of the design referenced by `designURL`, generated through Airbnb’s internal design-document API.

airbnb

Viaduct, Five Years On: Modernizing the Data-Oriented Service Mesh (opens in new tab)

Viaduct, Airbnb’s data-oriented service mesh, has evolved substantially over five years while retaining its core model: a central schema, hosted business logic, and re-entrant composition through GraphQL. Its usage has grown eightfold, supporting more than 130 teams and over 1.5 million lines of production code, without increasing operational overhead. Viaduct Modern now aims to simplify its developer API and establish stronger architectural boundaries, alongside the project’s release as open source. ## Adoption and Evolution - Viaduct traffic has increased by a factor of eight since 2020. - More than 130 teams now host code in Viaduct, supported by hundreds of weekly active developers. - The hosted codebase has grown to over 1.5 million lines, with roughly the same amount of test code. - Operational overhead has remained constant, incident-minutes have been cut in half, and costs have grown linearly with QPS. - Viaduct is now available as open-source software. ## Core Principles That Remain - **Central schema:** Viaduct provides one integrated schema connecting domains across Airbnb. - More than 75% of requests are internal. - The schema is developed by many teams but exposed as a connected graph. - **Hosted business logic:** Teams run business logic directly in Viaduct rather than maintaining separate microservices. - This reduces operational overhead and can allow standalone services to be retired. - Viaduct provides a serverless environment so developers can focus on application logic. - **Re-entrancy:** Hosted logic composes with other hosted logic through GraphQL fragments and queries. - This supports modularity. - It helps avoid the tightly coupled structure and maintenance problems associated with traditional monoliths. ## Problems with the Earlier Design - Viaduct’s APIs evolved reactively in response to individual use cases. - Multiple mechanisms emerged for accomplishing similar tasks, creating confusion for developers. - Some capabilities were well supported while others were not. - The framework’s layers had loose, inconsistent interfaces. - The boundary between Viaduct and hosted application code was weak. - These issues made framework improvements increasingly risky because changes could disrupt existing users. ## Simplifying the Tenant API - Viaduct Modern overhauls the developer-facing API and execution engine. - The new Tenant API reduces the implementation choices to two mechanisms: - **Node resolvers** - **Field resolvers** - The choice is determined by the schema rather than by ad hoc behavioral distinctions. - Resolver APIs have been unified wherever possible. - The goal is a smaller, more consistent surface that preserves successful ideas from the old API while removing unnecessary alternatives. ## Tenant Modules and Re-Entrant Composition - Viaduct uses modules and re-entrancy to provide boundaries similar to service definitions and RPC APIs in microservice architectures. - A tenant module combines: - Schema owned by a team - The code implementing that schema - Modules can create rich connections in the shared graph, but direct code dependencies between teams are discouraged. - Instead, teams declare their data requirements through GraphQL fragments and queries. ### Example: Extending the `User` Type - A Core User team owns the base `User` type and resolves fields such as `firstName` and `lastName`. - A Messaging team can extend `User` with a `displayName` field. - Its resolver declares that it needs `firstName` and `lastName`. - Messaging does not depend directly on Core User’s implementation or need to know where those fields originate. - This declarative model lets teams collaborate through the schema while preserving ownership and modularity. ## Framework Modularity - Viaduct Modern also restructures the framework itself. - The system consists of: - The GraphQL execution engine - The Tenant API - Hosted application code - Historically, the interfaces between these layers were weak, making performance and reliability improvements difficult to introduce safely. - The redesign focuses on stronger abstraction boundaries so the framework can evolve independently of application code. Viaduct’s modernization is intended to preserve its centralized, data-oriented model while making development simpler and framework evolution safer. The open-source release provides an opportunity for other organizations to evaluate or adopt this approach to schema-driven, modular service composition.

airbnb

Taming Service-Oriented Architecture Using A Data-Oriented Service Mesh (opens in new tab)

Airbnb’s Viaduct rethinks the service mesh as a data-oriented layer rather than a network for routing procedural service calls. Built on GraphQL, it presents a unified data graph that hides microservice dependencies from consumers and improves modularity in large SOAs. The central schema can also coordinate service APIs, database models, and serverless data transformations, making system-wide changes more agile. ## The Problem with Large SOAs - Modern organizations may operate thousands of microservices connected through highly tangled dependency graphs. - These graphs resemble “spaghetti code” at the service level: - Changes become difficult to plan. - Teams must coordinate across many service boundaries. - Consumers often depend directly on multiple underlying services. - Airbnb argues that microservice architectures need stronger organizing principles and technical mechanisms for enforcing modularity. ## From Procedure-Oriented to Data-Oriented Design - Traditional procedural design groups procedures into modules with public APIs and hidden implementation details. - Data-oriented design instead organizes software around encapsulated data objects and the methods that operate on them. - Microservices have largely returned SOA to a procedural model: - Each service exposes collections of remote procedural endpoints. - Consumers must know which services provide the data they need. - Viaduct applies data-oriented principles to the service mesh itself. ## Viaduct’s GraphQL Data Mesh - Viaduct defines the mesh through a GraphQL schema containing: - Types and interfaces representing managed data. - Queries and subscriptions for reading data. - Mutations for updating data. - The schema forms a single graph spanning data owned by many microservices. - A consumer can navigate related data through one query, such as: - `productById { manufacturer }` - `productById { reviews }` - `productById { reviews { author } }` - Viaduct determines which services provide each requested field. - This hides service dependencies from consumers and prevents every client from building its own cross-service orchestration logic. ## The Central Schema - Unlike distributed GraphQL approaches that split schemas across modules or federated services, Viaduct treats the schema as one central artifact. - Airbnb uses schema-management primitives to let multiple teams collaborate while preserving a unified model. - Portions of the central schema can define individual microservice APIs. - Airbnb ultimately aims to use the same schema to define database structures. - This could improve “data agility”: - Database changes would no longer need manual translation through several API layers. - A single schema update could propagate changes from storage through services to clients. - Cross-team coordination and delivery times could be reduced. ## Serverless Derived Fields - Many SOAs contain stateless services that transform backend data for particular clients or presentation layers. - Viaduct supports derived fields computed by serverless cloud functions. - These functions operate on the graph without needing direct knowledge of the underlying microservices. - Moving transformation logic into stateless containers can: - Reduce the number of services. - Lower operational overhead. - Keep the core service graph simpler. ## Implementation and Operational Features - Viaduct is built on `graphql-java`. - It supports fine-grained field selection through GraphQL selection sets. - It uses data-loading techniques and an intra-request cache. - Reliability features include short-circuiting and soft dependencies. - Field-level observability shows which services consume particular data. - Its GraphQL interface enables use of established open-source tooling and interactive development tools. Viaduct’s practical recommendation is to place a unified data schema at the center of the architecture, allowing the mesh—not individual consumers—to manage service composition. This can make large SOAs more modular, easier to evolve, and better suited to serverless execution.