dropbox·Oct 23, 2025·3 min readCurated summary

With Mobius Labs' Aana models, we're bringing deeper multimodal understanding to Dropbox Dash

Dropbox is integrating Mobius Labs’ Aana multimodal models into Dropbox Dash to improve how it understands text, images, audio, and video together. Aana is designed for efficient, large-scale processing, enabling Dash to interpret scenes, actions, speech, and relationships across media rather than relying on isolated metadata. The goal is to make rich media searchable and support future AI agents that can analyze content and act on it. ## Dropbox Dash’s Multimodal Goal - Dash combines AI assistance and search to understand a team’s work, context, and tools. - Its multimodal capabilities are intended to cover formats such as: - Text - Photos - Videos - Audio - This is especially valuable for creative, technical, media, and marketing teams whose work is distributed across many applications and content repositories. ## Why Multimedia Understanding Is Difficult - A video may require analysis of: - Scene changes - Speaker shifts - On-screen text - Objects and actions - Audio cues and dialogue - Images require systems to understand people, locations, events, and visual context. - Each modality has different timelines and semantic boundaries; transcripts, shots, visuals, and audio do not automatically align. - Effective understanding requires connecting modalities—for example, linking what someone says to the diagram they are showing. - Processing this information across exabytes of content would be prohibitively expensive without highly efficient infrastructure. ## How Aana Combines Models and Context - Aana analyzes images, audio, video, and related text as a connected system instead of separate data streams. - It combines fine-tuned, open-source foundation models for: - Speech, including optimized Whisper-based models such as `faster-whisper-large-v3-turbo` - Vision - Language - Transformer and mixture-of-experts architectures support fast inference on standard GPUs. - Aana tracks how scenes evolve over time, including: - Object movement - Actions - Changing layouts - Cross-modal events - Insights are stored in a shared vector space, enabling searches such as “the part where the presenter explains the API flow” rather than requiring users to scrub through a video manually. ## Efficiency at Dropbox Scale - Aana uses low-bit 8-bit and 4-bit inference through its HQQ system to reduce memory and compute requirements. - Gemlite provides custom GPU kernels to accelerate matrix multiplication and attention operations. - The Aana SDK coordinates models, batching, GPU utilization, and production pipelines. - Teams can configure and deploy different combinations of models while experimenting with new workflows. - These optimizations are intended to make analysis of exabytes of media possible with a much smaller compute footprint than conventional approaches. ## Future Applications in Dash - Aana could help users find specific visual elements across large creative archives. - It could summarize years of client meetings into searchable highlights. - Its capabilities may support agentic workflows that: - Analyze multimedia - Surface relevant insights automatically - Generate ideas - Take actions for teams Dropbox’s integration of Aana is intended to make Dash a more context-aware search and AI assistant for rich media. The practical outcome is less time spent organizing and manually reviewing content, and more time spent creating and acting on information.

Read original(opens in new tab)

WhisperMultimodal AITransformerAgentic Workflows+4

google·Mar 20, 2025Original article

Deciphering language processing in the human brain through LLM representations (opens in new tab)

Recent research by Google Research and collaborating universities indicates that Large Language Models (LLMs) process natural language through internal representations that closely mirror neural activity in the human brain. By comparing intracranial recordings from spontaneous conversations with the internal embeddings of the Whisper speech-to-text model, the study found a high degree of linear alignment between artificial and biological language processing. These findings suggest that the statistical structures learned by LLMs via next-word prediction provide a viable computational framework for understanding how humans comprehend and produce speech. ## Mapping LLM Embeddings to Brain Activity * Researchers utilized intracranial electrodes to record neural signals during real-world, free-flowing conversations. * The study compared neural activity against two distinct types of embeddings from the Transformer-based Whisper model: "speech embeddings" from the model’s encoder and "language embeddings" from the decoder. * A linear transformation was used to predict brain signals based on these embeddings, revealing that LLMs and the human brain share similar multidimensional spaces for coding linguistic information. * The alignment suggests that human language processing may rely more on statistical structures and contextual embeddings rather than traditional symbolic rules or syntactic parts of speech. ## Neural Sequences in Speech Comprehension * When a subject listens to speech, the brain follows a specific chronological sequence that aligns with model representations. * Initially, speech embeddings predict cortical activity in the superior temporal gyrus (STG), which is responsible for processing auditory speech sounds. * A few hundred milliseconds later, language embeddings predict activity in Broca’s area (located in the inferior frontal gyrus), marking the transition from sound perception to decoding meaning. ## Reversed Dynamics in Speech Production * During speech production, the neural sequence is reversed, beginning approximately 500 milliseconds before a word is articulated. * Processing starts in Broca’s area, where language embeddings predict activity as the brain plans the semantic content of the utterance. * This is followed by activity in the motor cortex (MC), aligned with speech embeddings, as the brain prepares the physical articulatory movements. * Finally, after articulation, speech embeddings predict activity back in the STG, suggesting the brain is monitoring the sound of the speaker's own voice. This research validates the use of LLMs as powerful predictive tools for neuroscience, offering a new lens through which to study the temporal and spatial dynamics of human communication. By bridging the gap between artificial intelligence and cognitive biology, researchers can better model how the brain integrates sound and meaning in real-time.

WhisperAILarge Language ModelsNatural Language Processing+4