meta·Jul 15, 2026·3 min readCurated summary

Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization

Hierarchical Interest Representation is an upstream embedding layer for Meta Ads that connects users’ inferred interests with advertisers, products, and services. It combines engagement data, multimodal content, graph learning, and hierarchical abstractions to address sparse deep-funnel signals and rare or unseen entities. The resulting universal embeddings and “Bag-of-Meaning” interest tokens could support retrieval, personalization, supervision, and ranking across Meta’s advertising systems. ## Purpose and Role in Deep-Funnel Optimization - The system aims to identify people with genuine latent interest in an advertiser’s offerings. - It is intended to complement systems such as Meta’s Generative Ads Model (GEM), Andromeda, and the Adaptive Ranking Model. - It uses user behavior—including scrolling, engagement, and explicit “Interested/Not interested” feedback—to infer preferences. - The broader goal is to improve discovery-oriented ad experiences and downstream conversion performance. ## Technical Challenges ### Sparse Engagement and Large-Scale Graphs - Users, advertisers, products, services, and campaigns are modeled as graph nodes. - Interactions and activities form edges in a graph spanning millions of advertisers, millions of ads, and billions of users. - Deep-funnel feedback is relatively scarce, leaving many entities with limited direct evidence. ### Long-Range Relationships - Useful signals may come from indirectly connected users and entities rather than direct interactions. - Capturing these relationships at Meta’s scale requires memory-efficient sparse attention and high-performance graph-learning algorithms. ### Dynamic and Unseen Entities - The ads ecosystem changes rapidly, while individual entities may have little historical engagement. - Representations must generalize to rare and previously unseen businesses and products. ## Core Design Properties ### Dimension Reduction - The raw graph is projected into a configurable “super-graph.” - Learned latent interest primitives act as super-nodes. - Sparse user-ad relationships become denser connections at the interest level. - The primitive graph is more stable and stationary than the constantly changing ads vocabulary. ### Knowledge Enrichment - Advertiser and product representations incorporate text, images, video, metadata, and catalog attributes. - Vision and language models process this multimodal content. - Content helps the system understand what a product or business is, not merely how users interacted with it. - This enables better generalization to new or sparsely observed entities. ### Unified Relational Representation - Users, advertisers, products, and latent interest primitives are embedded in a shared metric space. - The system can estimate: - Relationships between interest primitives - Similarity between users, ads, and products - A user’s proximity to particular interests - Which interests an advertiser or product serves - Embedding operations support both primitive-to-primitive and cluster-to-cluster relationship modeling. ### Multiple Hierarchical Granularities - Coarse representations capture dense, stable, high-level interests. - Finer representations capture sparse and specific deep-funnel intent. - Cascading hierarchical layers allow the embeddings to serve different needs across retrieval, personalization, ranking, and supervision. ## Architecture and Training - The architecture combines: - An in-house transformer-based graph learner - Bias-aware attention - Self-supervised cross-view distillation - Sparse attention for long-range graph relationships - It combines real-world semantic knowledge with users’ temporal engagement histories. - The model learns multi-hierarchical interest representations across a large graph. - Training is performed end-to-end on real Meta Ads data involving billions of interactions. ## Outputs and Potential Applications - Universal embeddings for users and ads entities. - “Bag-of-Meaning” interest tokens representing latent interests at different granularities. - Potential uses include: - Ad retrieval - Personalization - Ranking - Specialized ranking architectures - Training supervision - Cross-entity similarity and discovery Hierarchical Interest Representation is best understood as shared infrastructure for Meta’s ads recommendation stack. By combining sparse behavioral evidence with multimodal world knowledge and hierarchical graph abstractions, it could make deep-funnel optimization more robust, especially for specialized, rare, or newly introduced products and advertisers.

Read original(opens in new tab)

Graph LearningLarge Language ModelsMultimodal AITransformer+4

google·Jul 9, 2025Original article

Graph foundation models for relational data (opens in new tab)

Google researchers have introduced Graph Foundation Models (GFMs) as a solution to the limitations of traditional tabular machine learning, which often ignores the rich connectivity of relational databases. By representing tables as interconnected graphs where rows are nodes and foreign keys are edges, this approach enables a single model to generalize across entirely different schemas and feature sets. This shift allows for transferable graph representations that can perform inference on unseen tasks without the costly need for domain-specific retraining. ### Transforming Relational Schemas into Graphs The core methodology involves a scalable data preparation step that converts standard relational database structures into a single heterogeneous graph. This process preserves the underlying logic of the data while making it compatible with graph-based learning: * **Node Mapping:** Each unique table is treated as a node type, and every individual row within that table is converted into a specific node. * **Edge Creation:** Foreign key relationships are transformed into typed edges that connect nodes across different tables. * **Feature Integration:** Standard columns containing numerical or categorical data are converted into node features, while temporal data can be preserved as features on either nodes or edges. ### Overcoming the Generalization Gap A primary hurdle in developing GFMs is the lack of a universal tokenization method, unlike the word pieces used in language models or patches used in vision models. Traditional Graph Neural Networks (GNNs) are typically locked to the specific graph they were trained on, but GFMs solve this through several technical innovations: * **Schema Agnosticism:** The model avoids hard-coded embedding tables for specific node types, allowing it to interpret database schemas it has never encountered during training. * **Feature Interaction Learning:** Instead of training on "absolute" features (like specific price distributions), the model captures how different features interact with one another across diverse tasks. * **Generalizable Encoders:** The architecture uses transferable methods to derive fixed-size representations for nodes, whether they contain three continuous float features or dozens of categorical values. ### Scaling and Real-World Application To handle the requirements of enterprise-level data, the GFM framework is built to operate on a massive scale using Google’s specialized infrastructure: * **Massive Throughput:** The system utilizes JAX and TPU infrastructure to process graphs containing billions of nodes and edges. * **Internal Validation:** The model has been tested on complex internal Google tasks, such as spam detection in advertisements, which requires analyzing dozens of interconnected relational tables simultaneously. * **Performance Benefits:** By considering the connections between rows—a factor traditional tabular baselines like decision trees often ignore—the GFM provides superior downstream performance in high-stakes prediction services. Transitioning from domain-specific models to Graph Foundation Models allows organizations to leverage relational data more holistically. By focusing on the connectivity of data rather than just isolated table features, GFMs provide a path toward a single, generalist model capable of handling diverse enterprise tasks.

Graph LearningAITransformerJax+5

google·Mar 30, 2025Original article

The evolution of graph learning (opens in new tab)

The evolution of graph learning has transformed from classical mathematical puzzles into a cornerstone of modern machine learning, enabling the modeling of complex relational data. By bridging the gap between discrete graph algorithms and neural networks, researchers have unlocked the ability to generate powerful embeddings that capture structural similarities. This progression, spearheaded by milestones like PageRank and DeepWalk, has established graph-based models as essential tools for solving real-world challenges ranging from traffic prediction to molecular analysis. **Foundations of Graph Theory and Classical Algorithms** * Graph theory originated in 1736 with Leonhard Euler’s analysis of the Seven Bridges of Königsberg, which established the mathematical framework for representing connections between entities. * Pre-deep learning efforts focused on structural properties, such as community detection and centrality, or solving discrete problems like shortest paths and maximum flow. * The 1996 development of PageRank by Google’s founders applied these principles at scale, treating the internet as a massive graph of nodes (pages) and edges (hyperlinks) to revolutionize information retrieval. **Bridging Graph Data and Neural Networks via DeepWalk** * A primary challenge in the field was the difficulty of integrating discrete graph structures into neural network architectures, which typically favor feature-based embeddings over relational ones. * Developed in 2014, DeepWalk became the first practical method to bridge this gap by utilizing a neural network encoder to create graph embeddings. * These embeddings convert complex relational data into numeric representations that preserve the structural similarity between objects, allowing graph data to be processed by modern machine learning pipelines. **The Rise of Graph Convolutional Networks and Message Passing** * Following the success of graph embeddings, the field moved toward Graph Convolutional Networks (GCNs) in 2016 to better handle non-Euclidean data. * Modern frameworks now utilize Message Passing Neural Networks (MPNNs), which allow nodes to aggregate information from their neighbors to learn more nuanced representations. * These advancements are supported by specialized libraries in TensorFlow and JAX, enabling the application of graph learning to diverse fields such as physics simulations, disease spread modeling, and fake news detection. To effectively model complex systems where relationships are as important as the entities themselves, practitioners should transition from traditional feature-based models to graph-aware architectures. Utilizing contemporary libraries like those available for JAX and TensorFlow allows for the integration of relational structure directly into the learning process, providing more robust insights into interconnected data.

Graph LearningAIMachine LearningJax+4