Recurrent Neural Networks

2 posts

google3 min readCurated summary

Protecting cities with AI-driven flash flood forecasting

Google Research is expanding Flood Hub with urban flash flood forecasts that can provide up to 24 hours’ warning. The system addresses the lack of historical flood observations by using Gemini to extract verified events from public news reports, creating the Groundsource dataset for model training. Its global, lower-resolution approach aims to extend useful warnings to regions that lack expensive sensors and forecasting infrastructure, particularly in the Global South. ## The Need for Earlier Flash Flood Warnings - Flash floods cause roughly 85% of flood-related deaths worldwide and kill more than 5,000 people annually. - They often develop within six hours of intense rainfall, making rapid warnings essential. - Even 12 hours of warning can reduce flood damage by about 60%. - Early warning coverage remains highly unequal: fewer than half of developing countries have access to multi-hazard warning systems. - Flood Hub previously focused mainly on slower-moving riverine floods, covering more than 2 billion people across 150 countries. ## The Data Problem: “Invisible” Floods - River flood models can rely on stream gauges that record water levels and flow. - Flash floods may occur far from gauges, especially in cities where rainfall, impermeable surfaces, drainage, and terrain interact unpredictably. - Building detailed physical simulations globally would be computationally expensive. - Historical, precisely located flash flood records are also scarce, preventing conventional supervised machine learning. - Google’s Groundsource method uses Gemini to analyze public news reports, verify flood locations and times, and assemble a historical flash flood dataset. ## Scaling from Local Systems to Global Coverage - Local flash flood systems can be highly accurate using rain sensors, radar, water-level monitors, and flow measurements. - These systems are expensive to deploy and require location-specific calibration and engineering expertise. - Broader systems such as WMO’s FFGS, ERIC, and the U.S. NWS warning system depend on high-resolution maps, radar forecasts, and skilled hydrologists. - Those resources are often unavailable in the Global South. - Google’s model instead uses globally available products, including NASA IMERG, NOAA CPC, ECMWF’s IFS HRES forecasts, and Google DeepMind’s medium-range weather model. - Forecasts currently operate at a 20-by-20-kilometer resolution, constrained by the resolution of global data sources. ## The Urban Flash Flood Model - The model estimates whether a flash flood is likely in a given area during the next 24 hours. - It uses a recurrent neural network with a long short-term memory (LSTM) component to process meteorological time series. - Inputs also include static geographic and human-environment factors: - Urbanization density - Topography - Soil absorption rates - The initial rollout targets urban regions, where news coverage is denser and most of the world’s population lives. - It currently predicts impacts in areas with population densities above 100 people per square kilometer. ## Evaluation and Reported Performance - Precision was measured against the Groundsource dataset, but raw precision likely understates actual performance because some genuine floods are never reported. - A manual review of 100 alerts per continent found that many apparent false positives were confirmed flood events. - Recall was also evaluated against major floods recorded by the Global Disaster Awareness and Coordination System (GDACS). - Results indicate comparable precision and recall in regions such as South America and Southeast Asia and in wealthier countries with better instrumentation. The approach demonstrates how AI and unstructured public information can help provide scalable flash flood warnings where conventional monitoring infrastructure is limited. Its current urban focus and 20-kilometer resolution make it a broad early-warning tool rather than a replacement for highly localized sensor networks.

Read original(opens in new tab)
googleOriginal article

Titans + MIRAS: Helping AI have long-term memory (opens in new tab)

Google Research has introduced Titans, a new architecture, and MIRAS, a theoretical framework, designed to overcome the computational limitations of Transformers while maintaining high-fidelity long-term memory. These innovations utilize "test-time memorization," allowing models to update their core parameters in real-time as they process data without requiring offline retraining. By combining the speed of linear recurrent neural networks (RNNs) with the accuracy of attention mechanisms, the system enables AI to handle massive contexts such as genomic analysis or full-document understanding. ## Titans and Neural Long-Term Memory * Unlike traditional RNNs that compress context into fixed-size vectors or matrices, Titans uses a multi-layer perceptron (MLP) as a dedicated long-term memory module. * This deep neural memory provides significantly higher expressive power, allowing the model to synthesize and understand entire narratives rather than just storing passive snapshots. * The architecture separates memory into two distinct modules: an attention mechanism for precise short-term context and the MLP for summarizing long-term information. ## The Gradient-Based Surprise Metric * Titans employs a "surprise metric" to decide which information is important enough to store, mirroring the human brain's tendency to remember unexpected events. * The model calculates an internal error signal (gradient); a high gradient indicates that the new input is anomalous or context-breaking, signaling it should be prioritized for long-term storage. * The system incorporates "Momentum" to track the flow of context over time, ensuring that subsequent relevant information is captured even if individual tokens are not surprising. * To manage memory capacity during extremely long sequences, an adaptive weight decay mechanism acts as a forgetting gate to discard information that is no longer useful. ## MIRAS: A Unified Framework for Sequence Modeling * MIRAS provides a theoretical blueprint that views all major sequence models—including Transformers and linear RNNs—as different forms of associative memory modules. * The framework defines sequence models through four key design choices: memory architecture (e.g., MLP vs. vector), attentional bias, and the internal learning objectives used to combine new and old data. * This approach shifts AI modeling toward real-time adaptation, where the model actively learns and incorporates specific new details into its core knowledge as data streams in. These advancements suggest a shift away from static context windows toward dynamic systems capable of lifelong learning. For developers working with large-scale data, the Titans architecture provides a practical tool for scaling performance, while the MIRAS framework offers a roadmap for designing next-generation models that adapt instantly to new information.