Recommender Systems | Techlist.io

metaJul 15, 20263 min readCurated summary

Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization

Hierarchical Interest Representation is an upstream embedding layer for Meta Ads that connects users’ inferred interests with advertisers, products, and services. It combines engagement data, multimodal content, graph learning, and hierarchical abstractions to address sparse deep-funnel signals and rare or unseen entities. The resulting universal embeddings and “Bag-of-Meaning” interest tokens could support retrieval, personalization, supervision, and ranking across Meta’s advertising systems. ## Purpose and Role in Deep-Funnel Optimization - The system aims to identify people with genuine latent interest in an advertiser’s offerings. - It is intended to complement systems such as Meta’s Generative Ads Model (GEM), Andromeda, and the Adaptive Ranking Model. - It uses user behavior—including scrolling, engagement, and explicit “Interested/Not interested” feedback—to infer preferences. - The broader goal is to improve discovery-oriented ad experiences and downstream conversion performance. ## Technical Challenges ### Sparse Engagement and Large-Scale Graphs - Users, advertisers, products, services, and campaigns are modeled as graph nodes. - Interactions and activities form edges in a graph spanning millions of advertisers, millions of ads, and billions of users. - Deep-funnel feedback is relatively scarce, leaving many entities with limited direct evidence. ### Long-Range Relationships - Useful signals may come from indirectly connected users and entities rather than direct interactions. - Capturing these relationships at Meta’s scale requires memory-efficient sparse attention and high-performance graph-learning algorithms. ### Dynamic and Unseen Entities - The ads ecosystem changes rapidly, while individual entities may have little historical engagement. - Representations must generalize to rare and previously unseen businesses and products. ## Core Design Properties ### Dimension Reduction - The raw graph is projected into a configurable “super-graph.” - Learned latent interest primitives act as super-nodes. - Sparse user-ad relationships become denser connections at the interest level. - The primitive graph is more stable and stationary than the constantly changing ads vocabulary. ### Knowledge Enrichment - Advertiser and product representations incorporate text, images, video, metadata, and catalog attributes. - Vision and language models process this multimodal content. - Content helps the system understand what a product or business is, not merely how users interacted with it. - This enables better generalization to new or sparsely observed entities. ### Unified Relational Representation - Users, advertisers, products, and latent interest primitives are embedded in a shared metric space. - The system can estimate: - Relationships between interest primitives - Similarity between users, ads, and products - A user’s proximity to particular interests - Which interests an advertiser or product serves - Embedding operations support both primitive-to-primitive and cluster-to-cluster relationship modeling. ### Multiple Hierarchical Granularities - Coarse representations capture dense, stable, high-level interests. - Finer representations capture sparse and specific deep-funnel intent. - Cascading hierarchical layers allow the embeddings to serve different needs across retrieval, personalization, ranking, and supervision. ## Architecture and Training - The architecture combines: - An in-house transformer-based graph learner - Bias-aware attention - Self-supervised cross-view distillation - Sparse attention for long-range graph relationships - It combines real-world semantic knowledge with users’ temporal engagement histories. - The model learns multi-hierarchical interest representations across a large graph. - Training is performed end-to-end on real Meta Ads data involving billions of interactions. ## Outputs and Potential Applications - Universal embeddings for users and ads entities. - “Bag-of-Meaning” interest tokens representing latent interests at different granularities. - Potential uses include: - Ad retrieval - Personalization - Ranking - Specialized ranking architectures - Training supervision - Cross-entity similarity and discovery Hierarchical Interest Representation is best understood as shared infrastructure for Meta’s ads recommendation stack. By combining sparse behavioral evidence with multimodal world knowledge and hierarchical graph abstractions, it could make deep-funnel optimization more robust, especially for specialized, rare, or newly introduced products and advertisers.

Read original

Recommender Systems Large Language Models Multimodal AI Transformer+4

netflixJun 29, 20263 min readCurated summary

GenPage: Towards End-to-End Generative Homepage Construction at Netflix

GenPage is Netflix’s end-to-end generative approach to building personalized homepages. Instead of separately ranking rows and items, one transformer autoregressively generates the entire page—including rows, entities, and layout—from user and request context. In production, it outperformed Netflix’s mature multi-stage recommender on a core engagement metric while reducing serving latency by 20%. ## Reframing Homepage Recommendation - Netflix’s homepage is a personalized two-dimensional structure, not a single ranked list. - Traditional systems use separate candidate-generation and ranking stages for rows and entities. - GenPage treats homepage construction as a prompt-response task: - The prompt contains user history, profile information, and request context. - The response is the complete homepage generated autoregressively. - The approach aims to: - Replace complex multi-stage pipelines with one end-to-end model. - Optimize the whole page using reinforcement learning. - Capture interactions such as diversity and the trade-off between high-value rows and continued browsing. - Scale more predictably with additional data, compute, and model capacity. - Support new content types, layouts, UI components, and personalized artwork with fewer architectural changes. ## Production Challenges and Results - Real-time generation makes serving latency a major constraint. - The system must address: - Cold-start entities in a constantly changing catalog. - Shifting user interests and cultural trends. - Product and business rules that constrain generated pages. - An online A/B test against Netflix’s optimized production recommender produced: - Statistically significant improvement on Netflix’s primary launch engagement metric. - A 20% reduction in end-to-end serving latency. - Offline experiments found that: - Improving the prompt helped more than increasing model capacity in the tested regime. - Reinforcement-learning post-training improved homepage diversity, even though diversity was not an explicit objective. ## Tokenizing Context and Pages - Each training example contains: - **Context:** user history, profile attributes, and request information. - **Page:** displayed rows and entities in layout order. - **Feedback:** interactions such as plays, thumbs-up, and abandonment. - Context and page are tokenized as model inputs and outputs. - Feedback is used to derive reward and supervision signals rather than being directly generated. ## Domain-Specific Tokenization - GenPage uses a custom recommender-system tokenizer instead of a general-purpose text tokenizer. - This reduces sequence length and improves inference cost and latency. - For example, an action such as watching *Orange Is the New Black* can be represented with four tokens: - Entity ID - Action type - Time bucket - Duration bucket - Direct token mappings to product concepts, such as rows and entities, also make it easier to enforce generation rules and business constraints. ## Context Representation - User-history tokens encode: - Action type - Entity ID - Timestamp - Duration - The history includes explicit signals, such as playback, adding titles to My List, and thumbs-up, as well as implicit signals such as trailer views and detail-page visits. - Profile tokens represent attributes including language and profile type. - Request-context tokens include time of day, day of week, and device. - Long data sources, such as complete impression histories, are summarized to control sequence length and cost. - These summaries improve practicality but introduce handcrafted prompt engineering; learning to compress such information end to end remains a future direction. - Special segment markers help the model distinguish between different context sources.

Read original

Recommender Systems Generative AI Transformer Reinforcement Learning+2

googleJun 26, 2025Original article

REGEN: Empowering personalized recommendations with natural language (opens in new tab)

Google Research has introduced REGEN, a benchmark dataset designed to evolve recommender systems from simple item predictors into conversational agents capable of natural language interaction. By augmenting the Amazon Product Reviews dataset with synthetic critiques and narratives using Gemini 1.5 Flash, the researchers provide a framework for training models to understand user feedback and explain their suggestions. The study demonstrates that integrating natural language critiques significantly improves recommendation accuracy while enabling models to generate personalized, context-aware content. ### Composition of the REGEN Dataset * The dataset enriches the existing Amazon Product Reviews archive by adding synthetic conversational elements, specifically targeting the gap in datasets that support natural language feedback. * **Critiques** are generated for similar item pairs within hierarchical categories, allowing users to guide the system by requesting specific changes, such as a different color or increased storage. * **Narratives** provide contextual depth through purchase reasons, product endorsements, and concise user summaries, helping the system justify its recommendations to the end-user. ### Unified Generative Modeling Approaches * The researchers framed a "jointly generative" task where models must process a purchase history and optional critique to output both a recommended item ID and a supporting narrative. * The **FLARE (Hybrid)** architecture uses a sequential recommender for item prediction based on collaborative filtering, which then feeds into a Gemma 2B LLM to generate the final text narrative. * The **LUMEN (Unified)** model functions as an end-to-end system where item IDs and text tokens are integrated into a single vocabulary, allowing one LLM to handle critiques, recommendations, and narratives simultaneously. ### Performance and Impact of User Feedback * Incorporating natural language critiques consistently improved recommendation metrics across different architectures, demonstrating that language-guided refinement is a powerful tool for accuracy. * In the Office domain, the FLARE hybrid model's Recall@10—a measure of how often the desired item appears in the top 10 results—increased from 0.124 to 0.1402 when critiques were included. * Results indicate that models trained on REGEN can achieve performance comparable to state-of-the-art specialized recommenders while maintaining high-quality natural language generation. The REGEN dataset and the accompanying LUMEN architecture provide a path forward for building more transparent and interactive AI assistants. For developers and researchers, utilizing these conversational benchmarks is essential for moving beyond "black box" recommendations toward systems that can explain their logic and adapt to specific user preferences in real time.

Recommender Systems AI Large Language Models Gemini+4