pinterestJan 28, 2026

Ads Candidate Generation using Behavioral Sequence Modeling (opens in new tab)

Pinterest’s Ads team uses behavioral sequence modeling to improve ad candidate generation by predicting what users are likely to convert on next. Transformer-based two-tower models first predict relevant advertisers and then specific products, using offsite activity such as views, purchases, and add-to-cart events. The advertiser model is already in production, while item-level modeling addresses Pinterest’s rapidly growing catalog and enables more precise, scalable personalization. ## Predicting Advertiser Interaction - A bidirectional Transformer encodes each user’s behavioral event sequence. - An MLP-based advertiser tower represents candidate advertisers. - Training uses: - In-batch negative samples - Sampled softmax loss - Positive events consisting of checkout, add-to-cart, or signup conversions within a future K-day window - Log-Q bias correction to avoid excessively penalizing popular advertisers - The model is evaluated with Recall@K by comparing user and advertiser embedding similarity against an indexed set of roughly 2 million advertisers. - An offline batch job generates each user’s top 100 advertisers and publishes them to the online feature store. - During ad serving, eligible ads from those advertisers are passed to the L1 ranker, blended with other candidate sources, and scored by heavier downstream models and the marketplace auction. - Online experiments produced higher conversion volume and lower cost per action. - The advertiser-level model has served production traffic for Standard ads since Spring 2024. ## Moving from Advertisers to Products - Pinterest next sought to predict the specific products a user would interact with, rather than only the likely advertiser. - Item-level prediction better matches the item-based ad delivery funnel and avoids forcing downstream models to score an impractically large set of products from selected advertisers. - The approach aims to capture both immediate intent and longer-term interests. ## Item-Level Model Architecture - The model retains the two-tower design: - A user tower encodes behavioral sequences. - An item tower represents individual shopping product Pins. - Item representations combine: - Internal Pin embeddings learned from Pinterest’s engagement graph - Product metadata from the merchant catalog - Because the catalog exceeds 1 billion items, training uses both in-batch negatives and a randomly sampled negative set of 20 million Pins. - The model uses the same conversion labels as the advertiser model. - Label weights and log-Q parameters are tuned to balance retrieval quality with diversity across both products and advertisers. - Daily inference updates user embeddings only for users with new activity, appending them to a previous feature-store snapshot to reduce computation. - The trained item tower indexes hundreds of millions of ad items. ## Evaluation and Diversity - Item retrieval is evaluated using cosine similarity and hit rates at different K values. - Final model selection considers both: - Item-level Recall@K - Advertiser-level Recall@K - Qualitative review is also important because offsite activity is sparse and noisy. - The model is compared with max-pooling and mean-pooling baselines that use aggregated embeddings without Transformer-based sequence modeling. - The evaluation emphasizes that strong retrieval must also produce semantically relevant and sufficiently diverse recommendations. Pinterest’s progression from advertiser prediction to item prediction shows how behavioral sequence models can make ad retrieval more personalized while remaining scalable. A practical system should combine sequence-aware user representations, large-scale approximate retrieval, and explicit controls for popularity, diversity, and computational efficiency.

contrastive-learningtransformerembeddingsfeature-store+4

pinterestJan 13, 2026

PinLanding: Turn Billions of Products into Instant Shopping Collections with Multimodal AI (opens in new tab)

PinLanding is a production pipeline for turning billions of products into searchable shopping collections using multimodal AI. Rather than relying mainly on historical queries or manual curation, it derives structured product attributes from images and metadata, then aligns those attributes with real user search behavior. The system combines multimodal LLMs, embedding-based consolidation, a CLIP-style classifier, and distributed infrastructure to produce scalable, precise shopping feeds. ## Understanding Shopping Intent - Pinterest analyzes search history, autocomplete use, filters, and browsing paths to estimate shopping demand. - Existing systems handle high-volume queries such as “black cocktail dress” well, but provide weaker coverage for: - Long-tail queries - Conversational requests - Contextual intents such as “what to wear for an Italian summer vacation” - The analysis identifies: - Product areas with strong demand but poor collection coverage - Important attribute dimensions, including color, occasion, style, fit, price, and brand - The goal is to expand and improve collection coverage, not replace query understanding. ## Generating and Curating Shopping Topics - Each product is represented by an image plus metadata such as title, description, merchant tags, and price. - A vision-language model generates normalized key-value attributes rather than free-form descriptions. - Raw model output has high recall but produces: - Excessively specific attributes - Near-duplicates such as “boho,” “bohemian,” and “boho-chic” - Sparse attributes that apply to very few products - PinLanding builds a compact vocabulary through: - Frequency filtering to remove rarely useful attributes - Embedding-based clustering to merge semantically similar terms - Manual and LLM-assisted review - An LLM judge evaluates generated topics for semantic coherence, realistic shopping intent, and alignment with natural search phrasing. ## Scalable Attribute Assignment - Running the vision-language model over every product is too expensive and operationally fragile. - PinLanding trains a CLIP-inspired dual encoder: - One encoder embeds product images and text - Another embeds attribute phrases - Matching product-attribute pairs are trained as positives, while mismatches are negatives - A bidirectional contrastive loss aligns related products and attributes. - At inference, products and attributes are embedded once, and attributes are assigned when similarity exceeds a calibrated threshold. - This produces fewer distinct attributes while increasing the average number assigned to each product, creating a denser and more consistent attribute graph. ## Distributed Feed Construction - Ray handles large-scale batch inference across millions of products and topics. - The pipeline separates: - CPU-based image and metadata loading, tokenization, and serialization - GPU-based classifier inference - Streaming allows preprocessing and inference to overlap, while heterogeneous CPU and GPU clusters can scale independently. - The classifier pipeline reportedly completes in about 12 hours using eight NVIDIA A100 GPUs, at an estimated cost of roughly $500 per training run. - Feed construction uses approximate-nearest-neighbor techniques and strict attribute matching. - Topics are represented as attribute tuples, such as: - Category: dress - Color: yellow - Season: summer - Occasion: party - Apache Spark computes topic-product relevance using shared attributes and confidence weights, with partitioning and overlap filters reducing unnecessary candidate comparisons. The core recommendation is to combine user-behavior signals with content-first multimodal modeling. This approach can expand shopping coverage into conversational and long-tail intents while remaining practical through attribute consolidation, contrastive retrieval, and distributed inference.

contrastive-learningmultimodal-aiapache-sparkclip+4