Neural Networks | Techlist.io

googleJul 15, 20262 min readCurated summary

Towards demystifying the creativity of diffusion models

Diffusion models generate novel data because neural networks learn a smoothed approximation of the score function rather than perfectly memorizing it. This smoothing prevents denoising trajectories from collapsing directly onto training examples, allowing samples to interpolate between them. In high-dimensional data spaces, it helps recover the underlying data manifold while preserving realism and novelty. ## How Diffusion Models Denoise - Training corrupts real samples with noise, then teaches a model to reverse the corruption step by step. - The score function acts like a force field, directing noisy points toward meaningful data. - A perfectly learned score function would pull every generated sample onto one of the training examples, resulting in memorization. - In practice, neural networks learn an approximation of this function and therefore can generalize beyond the training set. ## Score Smoothing Creates Interpolation - Neural network regularization, including weight decay, makes sharp changes in the ideal score function difficult to represent. - In a one-dimensional example with training points at -1 and +1: - The perfect score sharply switches direction at zero. - Generated points eventually collapse onto either -1 or +1. - A smoothed score creates a gentler transition near zero. - Points in this transition region move more slowly and can settle between the training points. - This interpolation produces novel but plausible samples. - Smoothing can arise from explicit regularization or implicit regularization caused by gradient-based optimization. ## Recovering the Hidden Data Manifold - Real images occupy a small, structured manifold within a much larger high-dimensional pixel space. - Generating new images requires recovering this manifold from finite training data. - Score smoothing behaves differently depending on direction: - Along the manifold, it slows movement toward individual training examples. - Toward the manifold, the score is already relatively smooth, so smoothing has little effect. - This directional behavior prevents samples from becoming blurry in empty regions while reducing memorization along the manifold. - The result is a balance between fidelity and creativity: generated outputs remain realistic while differing from the training examples. ## Conclusion The paper argues that diffusion-model creativity is a predictable mathematical consequence of score smoothing. Neural networks’ regularized, approximate learning allows denoising trajectories to interpolate across the data manifold instead of merely retrieving memorized samples.

Read original

Neural Networks Generative AI Diffusion Models Score Function+4

metaMay 26, 20264 min readCurated summary

SilverTorch: Index as Model — A New Retrieval Paradigm for Recommendation Systems

SilverTorch is a unified, GPU-based recommendation retrieval system designed to replace fragmented microservices with one integrated neural network. Its “Index as Model” architecture represents retrieval components—including item indices, filtering, reranking, and user modeling—as PyTorch modules. The system reportedly delivers up to 23.7× higher throughput and 20.9× better compute-cost efficiency than comparable CPU-based or traditional multi-service systems, while improving recommendation quality. ## Limits of Microservice-Based Retrieval - Traditional retrieval pipelines use separate services for: - Computing user embeddings - Finding similar content - Applying eligibility rules - Scoring and reranking candidates - An orchestrator coordinates these services before passing thousands of candidates to downstream ranking, all within roughly 100 milliseconds. - This architecture creates several structural problems: - **Data movement:** Network calls, serialization, and service coordination consume latency that could otherwise support more computation. - **Version inconsistency:** User models, item indices, and filtering rules may be updated independently, causing mismatches between user and item representations. - **Siloed engineering:** ML teams typically work in PyTorch while infrastructure teams work in C++, making improvements difficult to translate, test, and deploy. - GPU optimizations such as Faiss-GPU can accelerate individual services but do not eliminate the architectural overhead or enable deep coordination between components. ## Index as Model - SilverTorch replaces the service mesh with a single neural network. - Its central design principle, **Index as Model**, turns traditional retrieval artifacts into model components: - Item indices become tensors. - Eligibility filters become operators. - User towers, scoring layers, and rerankers become modules. - A single request passes through the integrated model, which: - Finds content relevant to the user’s interests - Applies language, geography, and policy constraints - Predicts multiple engagement outcomes - Produces a combined score for the final candidate set - This integration enables more complex models and larger candidate evaluations without exceeding the sub-100-millisecond latency target. ## Unified Retrieval Components - SilverTorch incorporates multiple functional regions within one model: - Approximate nearest-neighbor search identifies relevant items efficiently. - Eligibility filtering removes content that cannot be shown to a user. - Multi-task reranking predicts actions such as likes, shares, and comments. - Composite scoring combines these predictions into a final ranking signal. - Some components are hand-engineered, while others can be trained end-to-end through backpropagation. - From the runtime’s perspective, every component is a standard PyTorch `nn.Module`, regardless of whether it performs search, filtering, or learned prediction. ## Pure PyTorch Implementation - SilverTorch reimplements ANN search, Bloom-filter indexing, eligibility checks, neural reranking, and composite scoring as pure PyTorch modules. - The unified design requires: - Tensor-based data representation - Tensor-in, tensor-out operations - A consistent `nn.Module` interface - This allows modules to share memory, execution graphs, and compilation steps. - Engineers can co-design stages—for example, selecting promising clusters, filtering within them, and scoring only surviving candidates—instead of treating each operation as an isolated service. - The approach reduces the separation between ML and infrastructure engineering, allowing both groups to work within the same programmable layer. ## Performance and Scale - In an 80-million-item end-to-end evaluation, SilverTorch achieved: - **23.7× higher requests per second** than a strong traditional multi-service baseline using the same model architecture. - **20.9× better estimated total-cost-of-ownership efficiency** than a CPU-based solution. - The system is intended to support retrieval across multiple applications and large-scale feeds and video products. - Its increased efficiency makes neural reranking and multi-task engagement scoring practical within strict production latency budgets. SilverTorch’s main recommendation is architectural: consolidate retrieval into a single, composable model rather than optimizing disconnected services. Representing every retrieval stage as a PyTorch module can reduce overhead, improve consistency, enable deeper cross-stage optimization, and make more sophisticated recommendations feasible at scale.

Read original

Neural Networks Microservices Recommendation Systems Pytorch+3

googleApr 29, 20262 min readCurated summary

Four ways Google Research scientists have been using Empirical Research Assistance

Empirical Research Assistance (ERA) is being used by Google researchers to tackle practical scientific problems rather than only benchmark exercises. Early applications span public-health forecasting, cosmology, and climate monitoring, showing that AI can improve prediction, solve difficult mathematical problems, and extract new value from existing data. The results suggest ERA could make advanced computational research more accessible while producing interpretable, scientifically grounded models. ## Public Health Forecasting - Google expanded ERA-based hospitalization forecasts from COVID-19 to influenza and RSV. - The team submits weekly forecasts for every U.S. state, covering horizons of up to four weeks. - Google forecasts have performed at or near the top of public CDC flu and COVID-19 leaderboards, with similarly strong internal results for RSV. - Forecast accuracy is evaluated using the Weighted Interval Score on log-transformed hospitalization data. - This approach could broaden access to epidemiological modeling and support forecasting for more diseases and regions. ## Cosmology: Cosmic Strings - Cosmic strings are theoretical spacetime defects that may emit gravitational radiation. - Calculating their radiation spectrum is difficult because the governing equations contain singularities. - Earlier work found only a partial solution for a square loop with an angle of 90 degrees. - By combining ERA with Gemini Deep Think, researchers derived six general solutions and a concise formula for the asymptotic limit. - The result demonstrates how AI systems can help explore advanced mathematical techniques and address previously unsolved cosmological problems. ## Climate Monitoring with Weather Satellites - Existing CO₂ satellites provide highly precise but infrequent and geographically limited measurements. - Geostationary satellites such as GOES East scan large areas every 10 minutes, but were not designed to measure CO₂. - Researchers used ERA to create a physics-guided neural network that combines: - 16 GOES East wavelength bands - Lower-troposphere meteorology - Solar angles - Time of year - Trained using sparse OCO-2 and OCO-3 observations, the model estimated column-averaged CO₂ continuously across the satellite’s coverage area. - Comparisons with independent satellite and ground-based observations showed that it captured real CO₂ variation. - The work illustrates how AI can repurpose existing instruments and improve the value of expensive scientific datasets. ERA’s early applications indicate that AI-assisted empirical software can support accurate forecasting, novel mathematical discovery, and higher-resolution environmental monitoring. Its greatest potential may lie in combining domain expertise with existing data and infrastructure to solve problems that would otherwise require substantial time and specialized resources.

Read original

Neural Networks AI Geospatial Analysis Data Collection+4

googleApr 16, 20263 min readCurated summary

AI-generated synthetic neurons speed up brain mapping

Google Research developed MoGen, an AI model that generates realistic synthetic neuron geometries to improve brain-mapping systems. Adding these synthetic examples to PATHFINDER reduced neuron reconstruction errors by 4.4%, primarily by preventing incorrect merges. At the scale of a complete mouse brain, the improvement could eliminate an estimated 157 person-years of manual proofreading. ## Connectomics and the Reconstruction Challenge - Connectomics creates brain wiring maps by imaging thin tissue slices, aligning them, and reconstructing 3D neurons. - AI assists with segmentation and reconstruction, but human experts must still correct errors. - Larger brains pose major scaling challenges: the fruit fly map contains about 166,000 neurons, while a mouse brain is roughly 1,000 times larger. ## Why Neuron Shape Matters - Neurons have complex structures, including long axons, branching dendrites, dendritic spines, and synapses. - PATHFINDER identifies neurite segments and combines them into complete neurons. - Poor or ambiguous microscopy data can cause: - **Split errors:** connected neurites are separated. - **Merge errors:** unrelated neurites are incorrectly joined. - Correcting these mistakes manually is one of the most time-consuming parts of brain mapping. ## MoGen’s Synthetic Neurons - MoGen uses point-cloud flow matching to transform random 3D point clouds into realistic neuronal shapes. - It was trained on surface samples from 1,795 human-verified mouse axons. - Experts could not reliably distinguish MoGen-generated neurite fragments from real ones. - The synthetic data reproduced features such as bending, twisting, thickening, and branching. ## Results with PATHFINDER - Training PATHFINDER with 10% MoGen-generated data reduced reconstruction errors by 4.4%. - The largest gains came from reducing merge errors. - Millions of synthetic neuron shapes were added to the training pipeline. - Although the percentage improvement is modest, it could save the equivalent of 157 years of expert proofreading for a full mouse-brain map. ## Future Applications - MoGen could be tuned to generate neuron types with specific lengths, branching patterns, or spatial ranges. - Future versions may focus on geometries that are particularly likely to cause reconstruction errors. - Google has also trained species-specific models for zebra finches and fruit flies. - The team is exploring synthetic electron-microscopy images to improve earlier stages of reconstruction. - MoGen and its species-specific models have been released as open source. Synthetic neuron generation is a practical way to expand training data without requiring additional manual annotation. Combined with targeted generation and synthetic microscopy, it could help make large-scale projects such as complete mouse-brain mapping more feasible.

Read original

Neural Networks AI Synthetic Data Connectomics+4

googleJan 11, 2026Original article

NeuralGCM harnesses AI to better simulate long-range global precipitation (opens in new tab)

NeuralGCM represents a significant evolution in atmospheric modeling by combining traditional fluid dynamics with neural networks to solve the long-standing challenge of simulating global precipitation. By training the AI component directly on high-quality NASA satellite observations rather than biased reanalysis data, the model achieves unprecedented accuracy in predicting daily weather cycles and extreme rainfall events. This hybrid approach offers a faster, more precise tool for both medium-range weather forecasting and multi-decadal climate projections. ## The Limitations of Cloud Parameterization * Precipitation is driven by cloud processes occurring at scales as small as 100 meters, which is far below the kilometer-scale resolution of global weather models. * Traditional models rely on "parameterizations," or mathematical approximations, to estimate how these small-scale events affect the larger atmosphere. * Because these approximations are often simplified, traditional models struggle to accurately capture the complexity of water droplet formation and ice crystal growth, leading to errors in long-term forecasts. ## Training on Direct Satellite Observations * Unlike previous AI models trained on "reanalyses"—which are essentially simulations used to fill observational gaps—NeuralGCM is trained on NASA satellite-based precipitation data spanning 2001 to 2018. * The model utilizes a differentiable dynamical core, an architecture that allows the neural network to learn the effects of small-scale events directly from physical observations. * By bypassing the weaknesses inherent in reanalysis data, the model effectively creates a machine-learned parameterization that is more faithful to real-world cloud physics. ## Performance in Weather and Climate Benchmarks * At a resolution of 280 km, NeuralGCM outperforms leading operational models in medium-range forecasts (up to 15 days) and matches the precision of sophisticated multi-decadal climate models. * The model shows a marked improvement in capturing precipitation extremes, particularly for the top 0.1% of rainfall events. * Evaluation through WeatherBench 2 demonstrates that NeuralGCM accurately reproduces the diurnal (daily) weather cycle, a metric where traditional physics-based models frequently fall short. NeuralGCM provides a highly efficient and accessible framework for researchers and city planners who need to simulate long-range climate scenarios, such as 100-year storms or seasonal agricultural cycles. Its ability to maintain physical consistency while leveraging the speed of AI makes it a powerful candidate for the next generation of global atmospheric modeling.

Neural Networks AI Machine Learning Climate Modeling+5

pinterestDec 5, 20253 min readCurated summary

Improving Quality of Recommended Content through Pinner Surveys

Pinterest uses Pinner surveys to measure visual quality and incorporate user preferences into recommendation systems, rather than optimizing solely for engagement. The company surveyed 5,000 Pins, trained a lightweight neural network to predict average perceived quality, and applied the resulting model across Homefeed, Related Pins, and Search. This approach aims to reduce clickbait and promote content that supports positive, long-term user experiences. ## Why Engagement Alone Is Insufficient - High engagement does not necessarily indicate high-quality content; optimizing for clicks can promote clickbait or harmful material. - Pinterest defines quality as content that feels good, inspires further exploration, and encourages fulfilling long-term engagement. - Direct user feedback helps recommendation systems prioritize content that Pinners actually value. - The work supports Pinterest’s Inspired Internet Pledge principles, especially listening to users and tuning the platform for wellbeing. ## Collecting Pinner Quality Ratings - Pinners rated images from 1 to 5 in response to: “How visually pleasing or displeasing is this Pin?” - Pinterest collected ratings for 5,000 Pins, sampling 1,000 from each of five major interest categories: - Art - Beauty - DIY & Crafts - Home Decor - Women’s Fashion - Pins were sampled based on impressions and were generally mid-to-high quality rather than deliberately exposing users to poor content. - Each image received at least 10 ratings, allowing Pinterest to average responses and reduce noise from subjectivity or accidental misclicks. - Surveys were considered appropriate for visual appeal, which is subjective but still measurable across many users. More objective issues should be evaluated by trained reviewers, while highly contextual judgments such as personal relevance are harder to capture with a single Pin-level score. - Highly rated content included makeup, grooming styles, maximalist interiors, landscapes, sunsets, and baby animals. - Home Decor images tended to receive higher ratings overall, while Art showed the greatest variation, reflecting its subjective nature. ## Training a Visual-Quality Model - Pinterest trained a model to estimate the average Pinner’s perception of visual quality from image embeddings. - Embeddings encode visual, textual, and behavioral information, including relationships between images and the boards where they are saved. - The model produces a score from 0 to 1, with higher values representing greater perceived quality. - Pinterest chose a small fully connected neural network with approximately 92,000 parameters: - The limited size helps prevent overfitting to the 5,000-image dataset. - It also makes large-scale inference faster and less expensive. - Instead of predicting an exact rating, the model uses pairwise ranking: - It learns which of two images Pinners would consider better. - The comparison is based on each image’s mean survey rating. - Training comparisons are restricted to images within the same top-level interest category, encouraging the model to learn visual quality rather than simply recognizing that one topic is more popular than another. Pinterest’s approach demonstrates how survey-based quality signals can complement engagement metrics. Training recommendation systems on what users perceive as appealing can help the platform promote more satisfying content while reducing incentives to favor attention-grabbing but low-quality material.

Read original

Neural Networks Machine Learning Recommendation Systems Embeddings+3

googleNov 20, 2025Original article

Reducing EV range anxiety: How a simple AI model predicts port availability (opens in new tab)

Google Research has developed a lightweight AI model designed to predict the probability of EV charging port availability at specific future intervals, directly addressing the "range anxiety" experienced by electric vehicle drivers. By co-designing the model with deployment infrastructure, researchers found that a simple linear regression approach outperformed more complex architectures like neural networks and decision trees. The resulting system effectively predicts availability changes during high-turnover periods, providing more reliable navigation and planning data than traditional "no-change" assumptions. ### Model Architecture and Feature Selection * The development team prioritized a minimal feature set to ensure low-latency deployment and high speed in real-world navigational applications. * After testing various architectures, a straightforward linear regression model was selected for its robustness and superior performance in this specific predictive task. * The model was trained using real-time availability data from diverse geographical regions, specifically California and Germany, with an emphasis on larger charging stations that reflect high-traffic usage patterns. ### Temporal Feature Weights and Occupancy Trends * The model uses the hour of the day as a primary feature, treating each hour as an independent variable to capture specific daily cycles. * Learned numerical "weights" dictate the predicted rate of occupancy change: positive weights indicate ports are becoming occupied (e.g., during morning rush), while negative weights indicate ports are being freed up (e.g., during evening hours). * The system is designed to only deviate from the current occupancy state when the change rate is statistically significant or when a station's large size amplifies the likelihood of a status change. ### Performance Benchmarking and Validation * The model was evaluated against a "Keep Current State" baseline, which assumes future availability will be identical to the present status—a difficult baseline to beat since port status remains unchanged roughly 90% of the time over 30-minute windows. * Accuracy was measured using Mean Squared Error (MSE) and Mean Absolute Error (MAE) over 30-minute and 60-minute time horizons across 100 randomly selected stations. * Testing confirmed that the linear regression model provides its greatest value during infrequent but critical moments of high turnover, successfully identifying when a station is likely to become full or available. The success of this model demonstrates that sophisticated deep learning is not always the optimal solution for infrastructure challenges. By combining intuitive real-world logic—such as driver schedules and station capacity—with simple machine learning techniques, developers can create highly efficient tools that significantly improve the EV user experience without requiring massive computational overhead.

Neural Networks AI Machine Learning Predictive Modeling+4

googleJun 23, 2025Original article

From research to climate resilience (opens in new tab)

Google Research is leveraging advanced artificial intelligence to transform climate science from theoretical exploration into scalable, real-world resilience tools. By developing sophisticated models for floods, cyclones, and hyper-local weather, the initiative provides critical lead times that empower communities to protect lives and livelihoods against increasingly frequent environmental threats. This transition from "impossible" research to global implementation highlights AI's capacity to bridge data gaps in the world's most vulnerable regions. ## AI-Powered Global Flood Forecasting * Google developed a global hydrological AI model, recently published in *Nature*, which enables riverine flood forecasts up to seven days in advance. * The system utilizes "virtual gauges" to analyze historical data and provide predictions in regions where physical water-monitoring infrastructure is non-existent. * The Flood Hub platform now covers over 100 countries and 700 million people, providing an expert data layer and API access for local governments and researchers. ## Cyclone Tracking and Intensity Prediction * Collaborative research between Google DeepMind and Google Research has produced models that predict storm existence, track, intensity, and size up to 15 days in advance. * The AI generates up to 50 different possible scenarios for each storm, providing a more nuanced view of potential impact than traditional physics-based supercomputer simulations. * Through the new Weather Lab website, these experimental models are being shared with the US National Hurricane Center to assist in forecasting during the Atlantic hurricane season. ## Global Nowcasting with MetNet-3 * The MetNet-3 state-of-the-art neural weather model provides hyper-local precipitation forecasts with a 5km resolution, updated every 15 minutes. * By utilizing satellite observations instead of traditional ground-based radar, the system delivers reliable weather data to regions like Africa that lack extensive physical infrastructure. * These 12-hour "nowcasting" windows are integrated directly into Google Search, specifically helping agricultural communities react to changing conditions to improve crop yields and reduce waste. These advancements demonstrate that the "art of the possible" is rapidly expanding, offering a future where data-scarce regions can access the same life-saving predictive capabilities as developed nations through global partnerships and satellite-based modeling.

Neural Networks AI Machine Learning Weather Forecasting+4