Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization

Hierarchical Interest Representation is an upstream embedding layer for Meta Ads that connects users’ inferred interests with advertisers, products, and services. It combines engagement data, multimodal content, graph learning, and hierarchical abstractions to address sparse deep-funnel signals and rare or unseen entities. The resulting universal embeddings and “Bag-of-Meaning” interest tokens could support retrieval, personalization, supervision, and ranking across Meta’s advertising systems.

Purpose and Role in Deep-Funnel Optimization

• The system aims to identify people with genuine latent interest in an advertiser’s offerings.
• It is intended to complement systems such as Meta’s Generative Ads Model (GEM), Andromeda, and the Adaptive Ranking Model.
• It uses user behavior—including scrolling, engagement, and explicit “Interested/Not interested” feedback—to infer preferences.
• The broader goal is to improve discovery-oriented ad experiences and downstream conversion performance.

Technical Challenges

Sparse Engagement and Large-Scale Graphs

• Users, advertisers, products, services, and campaigns are modeled as graph nodes.
• Interactions and activities form edges in a graph spanning millions of advertisers, millions of ads, and billions of users.
• Deep-funnel feedback is relatively scarce, leaving many entities with limited direct evidence.

Long-Range Relationships

• Useful signals may come from indirectly connected users and entities rather than direct interactions.
• Capturing these relationships at Meta’s scale requires memory-efficient sparse attention and high-performance graph-learning algorithms.

Dynamic and Unseen Entities

• The ads ecosystem changes rapidly, while individual entities may have little historical engagement.
• Representations must generalize to rare and previously unseen businesses and products.

Core Design Properties

Dimension Reduction

• The raw graph is projected into a configurable “super-graph.”
• Learned latent interest primitives act as super-nodes.
• Sparse user-ad relationships become denser connections at the interest level.
• The primitive graph is more stable and stationary than the constantly changing ads vocabulary.

Knowledge Enrichment

• Advertiser and product representations incorporate text, images, video, metadata, and catalog attributes.
• Vision and language models process this multimodal content.
• Content helps the system understand what a product or business is, not merely how users interacted with it.
• This enables better generalization to new or sparsely observed entities.

Unified Relational Representation

• Users, advertisers, products, and latent interest primitives are embedded in a shared metric space.
• The system can estimate:
  • Relationships between interest primitives
  • Similarity between users, ads, and products
  • A user’s proximity to particular interests
  • Which interests an advertiser or product serves
• Embedding operations support both primitive-to-primitive and cluster-to-cluster relationship modeling.

Multiple Hierarchical Granularities

• Coarse representations capture dense, stable, high-level interests.
• Finer representations capture sparse and specific deep-funnel intent.
• Cascading hierarchical layers allow the embeddings to serve different needs across retrieval, personalization, ranking, and supervision.

Architecture and Training

• The architecture combines:
  • An in-house transformer-based graph learner
  • Bias-aware attention
  • Self-supervised cross-view distillation
  • Sparse attention for long-range graph relationships
• It combines real-world semantic knowledge with users’ temporal engagement histories.
• The model learns multi-hierarchical interest representations across a large graph.
• Training is performed end-to-end on real Meta Ads data involving billions of interactions.

Outputs and Potential Applications

• Universal embeddings for users and ads entities.
• “Bag-of-Meaning” interest tokens representing latent interests at different granularities.
• Potential uses include:
  • Ad retrieval
  • Personalization
  • Ranking
  • Specialized ranking architectures
  • Training supervision
  • Cross-entity similarity and discovery

Hierarchical Interest Representation is best understood as shared infrastructure for Meta’s ads recommendation stack. By combining sparse behavioral evidence with multimodal world knowledge and hierarchical graph abstractions, it could make deep-funnel optimization more robust, especially for specialized, rare, or newly introduced products and advertisers.