Synthetic Data | Techlist.io

googleJun 30, 20263 min readCurated summary

Introducing TabFM: A zero-shot foundation model for tabular data

TabFM is a foundation model for tabular classification and regression that uses in-context learning instead of dataset-specific training. It processes an entire table in one forward pass, eliminating much of the usual hyperparameter tuning and feature engineering. The authors report that it performs competitively with or better than tuned tree-based models, with planned integration into BigQuery. ## The Challenge of Traditional Tabular ML - Tabular prediction supports applications such as churn prediction and fraud detection. - Algorithms including XGBoost, AdaBoost, and random forests remain widely used. - Building reliable models typically requires: - Hyperparameter optimization - Cross-validation - Domain-specific feature engineering - Repeated model training for each dataset ## Tabular Prediction Through In-Context Learning - TabFM treats tabular prediction as an in-context learning problem. - Users provide historical training rows and target rows together as a single input. - The model learns relationships between rows and columns during inference without updating model weights. - This enables zero-shot predictions on previously unseen datasets. ## Hybrid Architecture TabFM combines ideas from TabPFN and TabICL to address the two-dimensional, order-independent nature of tables. - **Alternating row and column attention** - Attends across both features and examples. - Learns feature interactions and dependencies directly from the table. - Reduces the need for manually engineered features. - **Row compression** - Converts the contextualized information for each row into a dense vector. - **In-context Transformer** - Operates on compressed row representations rather than the full table grid. - Reduces computation and supports larger datasets efficiently. ## Training with Synthetic Data - Large, diverse real-world tables are difficult to obtain because they often contain proprietary or sensitive information. - TabFM is trained entirely on hundreds of millions of synthetic datasets. - Structural causal models generate tables using varied random functions and relationships. - The goal is to reproduce the broad range of distributions and feature interactions found in real-world data. ## Benchmark Results and Model Variants - Evaluation used TabArena, an Elo-based benchmark covering: - 38 classification datasets - 13 regression datasets - Dataset sizes from 700 to 150,000 samples - **TabFM** - Works out of the box. - Produces predictions in one forward pass without tuning or cross-validation. - **TabFM-Ensemble** - Adds cross features and SVD-derived features. - Uses a 32-model ensemble with non-negative least squares to optimize weights. - Applies Platt scaling for classification calibration. - The authors state that TabFM consistently outperforms heavily tuned conventional supervised algorithms. ## Availability and BigQuery Integration - TabFM is available through Google’s Hugging Face and GitHub repositories. - Google plans to integrate it into BigQuery. - Users will be able to run regression and classification through an `AI.PREDICT` SQL command without specialized machine-learning expertise. TabFM’s main promise is to make high-quality tabular prediction as simple as supplying a dataset and requesting predictions. Its zero-shot approach could reduce the engineering burden of traditional workflows, though practical adoption will depend on benchmark reproducibility, inference costs, and performance across specific production datasets.

Read original

Synthetic Data Transformer Hugging Face Xgboost+4

googleApr 16, 20263 min readCurated summary

AI-generated synthetic neurons speed up brain mapping

Google Research developed MoGen, an AI model that generates realistic synthetic neuron geometries to improve brain-mapping systems. Adding these synthetic examples to PATHFINDER reduced neuron reconstruction errors by 4.4%, primarily by preventing incorrect merges. At the scale of a complete mouse brain, the improvement could eliminate an estimated 157 person-years of manual proofreading. ## Connectomics and the Reconstruction Challenge - Connectomics creates brain wiring maps by imaging thin tissue slices, aligning them, and reconstructing 3D neurons. - AI assists with segmentation and reconstruction, but human experts must still correct errors. - Larger brains pose major scaling challenges: the fruit fly map contains about 166,000 neurons, while a mouse brain is roughly 1,000 times larger. ## Why Neuron Shape Matters - Neurons have complex structures, including long axons, branching dendrites, dendritic spines, and synapses. - PATHFINDER identifies neurite segments and combines them into complete neurons. - Poor or ambiguous microscopy data can cause: - **Split errors:** connected neurites are separated. - **Merge errors:** unrelated neurites are incorrectly joined. - Correcting these mistakes manually is one of the most time-consuming parts of brain mapping. ## MoGen’s Synthetic Neurons - MoGen uses point-cloud flow matching to transform random 3D point clouds into realistic neuronal shapes. - It was trained on surface samples from 1,795 human-verified mouse axons. - Experts could not reliably distinguish MoGen-generated neurite fragments from real ones. - The synthetic data reproduced features such as bending, twisting, thickening, and branching. ## Results with PATHFINDER - Training PATHFINDER with 10% MoGen-generated data reduced reconstruction errors by 4.4%. - The largest gains came from reducing merge errors. - Millions of synthetic neuron shapes were added to the training pipeline. - Although the percentage improvement is modest, it could save the equivalent of 157 years of expert proofreading for a full mouse-brain map. ## Future Applications - MoGen could be tuned to generate neuron types with specific lengths, branching patterns, or spatial ranges. - Future versions may focus on geometries that are particularly likely to cause reconstruction errors. - Google has also trained species-specific models for zebra finches and fruit flies. - The team is exploring synthetic electron-microscopy images to improve earlier stages of reconstruction. - MoGen and its species-specific models have been released as open source. Synthetic neuron generation is a practical way to expand training data without requiring additional manual annotation. Combined with targeted generation and synthetic microscopy, it could help make large-scale projects such as complete mouse-brain mapping more feasible.

Read original

Synthetic Data AI Neural Networks Connectomics+4

googleApr 16, 20264 min readCurated summary

Designing synthetic datasets for the real world: Mechanism design and reasoning from first principles

Synthetic data generation should be treated as dataset-level mechanism design rather than one-sample-at-a-time prompting. Google’s Simula framework uses reasoning models to control coverage, diversity, complexity, and quality independently, enabling seedless, reproducible datasets for scarce or privacy-sensitive domains. Experiments across five domains show that thoughtfully designed data can outperform larger datasets, but the best configuration depends on the target task and model. ## Why Real-World Data Is Insufficient - Specialized AI applications often lack accessible data because domains are uncommon, expensive to label, or privacy-sensitive. - Manually creating datasets is costly, slow, and error-prone. - Real-world datasets are static, limiting rapid iteration and making it difficult to proactively generate safety edge cases. - Synthetic-first workflows can make data programmable, versioned, reproducible, and inspectable. ## Limitations of Existing Synthetic Data Methods - Many approaches depend on manual prompts, evolutionary algorithms, or large amounts of seed data. - These dependencies reduce scalability and explainability. - Generation parameters are often entangled, making it difficult to independently control diversity, difficulty, and correctness. - Most methods optimize individual samples instead of designing the dataset’s overall distribution. ## Simula’s Reasoning-First Design - Simula constructs datasets from first principles using reasoning models rather than opaque generation processes. - The framework is seedless and agentic, allowing improvements as the underlying models become better at reasoning. - It separates generation into independently controllable axes. ### Global Diversification - Reasoning models map a domain into deep, hierarchical taxonomies that serve as sampling scaffolds. - A recursive propose-and-refine process generates candidate categories, then evaluates, merges, and filters them with a critic model. - These taxonomies help datasets cover long-tail concepts instead of concentrating on common examples. - The approach was demonstrated with structures such as a Cyber Threat Intelligence taxonomy. ### Local Diversification - Taxonomy nodes are converted into “meta-prompts” representing scenarios. - Multiple distinct instantiations are generated for each scenario. - This reduces mode collapse—for example, representing SQL injection through varied contexts rather than repeating nearly identical questions. ### Complexification - A configurable portion of scenarios is made more elaborate or difficult. - Complexity can therefore be adjusted without changing the dataset’s semantic coverage. - The appropriate difficulty level depends on the capabilities of the model that will consume the data. ### Quality Checks - A dual-critic loop independently assesses whether outputs and answers are correct. - Independent verification helps reduce sycophancy and improves label reliability. - This enables quality control without requiring human review of every example. ## Reasoning-Based Evaluation - Conventional metrics such as embedding cosine distance offer only broad signals and limited practical guidance. - Simula introduces: - **Taxonomic Coverage**, which measures how thoroughly the conceptual space is represented. - **Calibrated Complexity Scoring**, which uses LLM-based batch comparisons and chess-style Elo ratings to estimate the difficulty of individual examples. - These metrics aim to evaluate diversity and difficulty in ways that better reflect downstream usefulness. ## Results Across Domains - Simula was evaluated using Gemini 2.5 Flash as a teacher and Gemma 3 4B as a student. - The experiments covered cybersecurity, legal reasoning, grade-school mathematics, and multilingual academic knowledge, with datasets reaching 512,000 examples per domain. - The full combination of global coverage, local diversity, and quality critique consistently outperformed simpler baselines. - High complexity improved math reasoning accuracy by 10% in GSM8k but harmed legal reasoning performance, where the teacher model was weaker. - Simula often achieved stronger downstream results with fewer examples, suggesting that data properties matter as much as volume. - The findings show that there is no universally optimal synthetic-data recipe; effective dataset design must be tailored to the domain and target model. Synthetic data is most effective when treated as an engineered system rather than a source of additional volume. Practitioners should separately tune coverage, variation, difficulty, and verification, then validate those choices against the downstream task.

Read original

Synthetic Data PostgreSQL Data Generation Mechanism Design+4

googleOct 19, 2025Original article

A picture's worth a thousand (private) words: Hierarchical generation of coherent synthetic photo albums (opens in new tab)

Researchers at Google have developed a hierarchical method for generating differentially private (DP) synthetic photo albums, providing a way to share representative datasets while protecting sensitive individual information. By utilizing an intermediate text representation and a two-stage generation process, the approach maintains thematic coherence across multiple images in an album—a significant challenge for traditional synthetic data methods. This framework allows organizations to apply standard, non-private analytical techniques to safe synthetic substitutes rather than modifying every individual analysis method for differential privacy. ## The Hierarchical Generation Process * The workflow begins by converting original photo albums into structured text; an AI model generates detailed captions for each image and a summary for the entire album. * Two large language models (LLMs) are privately fine-tuned using DP-SGD: the first is trained to produce album summaries, and the second generates individual photo captions based on those summaries. * Synthetic data is then produced hierarchically, where the model first generates a global album summary to serve as context, followed by a series of individual photo captions that remain consistent with that context. * The final step uses a text-to-image AI model to transform the private, synthetic text captions back into a set of coherent images. ## Benefits of Intermediate Text Representations * Text summarization is inherently privacy-enhancing because it is a "lossy" operation, meaning the text description is unlikely to capture the exact unique details of an original photo. * Using text as a midpoint allows for more efficient resource management, as generated albums can be filtered and curated at the text level before undergoing the computationally expensive process of image generation. * The hierarchical approach ensures that photos within a synthetic album share the same characters and themes, as every caption in a set is derived from the same contextual summary. * Training two separate models with shorter context windows is significantly more efficient than training one large model, because the computational cost of self-attention scales quadratically with the length of the context. This hierarchical, text-mediated approach demonstrates that high-level semantic information and thematic coherence can be preserved in synthetic datasets without sacrificing individual privacy. Organizations should consider this workflow—translating complex multi-modal data into structured text before synthesis—to scale differentially private data generation for advanced modeling and analysis.

Synthetic Data AI Large Language Models Generative AI+4

dropboxOct 2, 20253 min readCurated summary

A practical blueprint for evaluating conversational AI at scale

Conversational AI systems depend on many probabilistic stages, so even small changes can cause unexpected regressions. Dropbox Dash’s experience shows that evaluation should be treated like production engineering: systematic, repeatable, and required before changes are shipped. The approach combines curated datasets, actionable metrics, LLM-based judging, and human review. ## Evaluation as a Development Discipline - AI pipelines include intent classification, retrieval, ranking, prompt construction, inference, and safety filtering. - Changes to any stage can affect final answer quality in unpredictable ways. - Dropbox initially used ad-hoc testing, but shifted to a standardized process in which every model, prompt, or retrieval change had to pass evaluation before merging. - The evaluation framework covers datasets, metrics, tools, and workflows. - Future-proof evaluation must extend beyond text to images, video, and audio. ## Curating Public and Internal Datasets - Public datasets established baseline retrieval and question-answering performance: - **Natural Questions** tested retrieval from very large documents. - **MS MARCO** emphasized queries requiring multiple document hits. - **MuSiQue** tested multi-hop reasoning. - Internal datasets captured real-world usage from anonymized Dropbox employee queries and content. - Representative query sets reflected actual user behavior, using proxy labels or internal annotators. - Representative content sets focused on shared files, documentation, and connected data sources. - LLM-generated synthetic questions and answers covered tables, images, tutorials, and factual lookups. - These datasets became the foundation for automated pass/fail checks and experiment monitoring. ## Why Traditional Metrics Fall Short - Metrics such as BLEU, ROUGE, METEOR, BERTScore, and embedding similarity are fast and useful for detecting major regressions. - They often fail to measure production requirements, including: - Correct source citations - Factual accuracy - Valid file names and references - Reliable table parsing - Clear formatting - High ROUGE or BERTScore can coexist with hallucinations, missing citations, or buried factual errors. - Production evaluation therefore requires metrics tied directly to task requirements. ## Using LLMs as Evaluation Judges - LLM judges can assess dimensions traditional metrics miss, including: - Whether an answer addresses the query - Whether claims are supported by context - Citation correctness - Clarity, formatting, and tone - Judges receive the query, model answer, source context, and sometimes a hidden reference answer. - They return structured scores—scalar or categorical—alongside explanations. - Rubrics and judge models must themselves be tested, calibrated, versioned, and improved. - Specialized judges may be needed for particular languages or technical domains. ## Calibration and Human Review - Periodic manual labeling of sampled outputs created calibration sets for comparing human and judge-model decisions. - The team tracked agreement and judge drift over time. - Engineers manually reviewed 5–10% of each release’s regression suite. - Disagreements were investigated as either judge-prompt defects or model errors. - Recurring problems led to prompt revisions or more detailed scoring criteria. Dropbox’s evaluation-first approach treats AI changes like code changes: test them against realistic datasets, score them with task-specific rubrics, validate the evaluators, and retain human oversight. This makes conversational AI development more reliable as systems and modalities grow more complex.

Read original

Synthetic Data Large Language Models Natural Language Processing Prompt Engineering+4

googleJul 23, 2025Original article

Synthetic and federated: Privacy-preserving domain adaptation with LLMs for mobile applications (opens in new tab)

Researchers at Google have developed a framework for improving both small and large language models (LMs) in mobile applications like Gboard by utilizing privacy-preserving synthetic data and federated learning. This approach combines differential privacy (DP) with large language model (LLM) generation to minimize data memorization risks while achieving significant gains in production metrics like next-word prediction and proofreading. The result is a robust pipeline that allows models to adapt to specific user domains without compromising individual privacy or requiring centralized data storage. ### Strengthening Privacy with DP-FL * Gboard has transitioned all production LMs trained on user data to a Federated Learning with Differential Privacy (DP-FL) framework, ensuring data remains on-device and is never memorized. * The deployment utilizes the **BLT-DP-FTRL** algorithm, which offers an optimized trade-off between privacy guarantees and model utility while being easier to deploy in production. * Engineers adopted the **SI-CIFG** model architecture to facilitate efficient on-device training, ensuring the hardware can handle local updates while maintaining compatibility with DP constraints. ### Synthetic Data Generation via Public LLMs * Powerful LLMs trained on public web data are prompted to synthesize high-quality text that mimics mobile user interactions without ever accessing actual private user data. * The process involves a two-step prompting strategy: first, filtering public datasets to identify topics common in mobile communication, and second, generating new, domain-specific text based on those patterns. * This synthetic data serves as a bridge for pre-training small LMs, which are then refined through private post-training on-device to capture the nuances of user behavior. ### Adapting LLMs for Mobile Proofreading * To support advanced features like Gboard's "Proofread," researchers developed a "Synthesize-then-Adapt" pipeline specifically for error correction. * LLMs generate synthetic "corrupted" text to simulate common mobile typing errors, providing the necessary training pairs (error/correction) that are difficult to find in public datasets. * Federated learning is then used to adapt these error-correction models to specific app domains (such as messaging or email) using on-device signals, ensuring the model understands the specific context of the user's typing. The success of these techniques in Gboard demonstrates that synthetic data can effectively replace or augment private data throughout the machine learning lifecycle. For developers working with sensitive user information, adopting a "synthetic-first" approach combined with federated learning provides a scalable path to model improvement that adheres to the core principles of data minimization and anonymization.

Synthetic Data Large Language Models Differential Privacy On Device Ai+3

googleApr 29, 2025Original article

Benchmarking LLMs for global health (opens in new tab)

Google Research has introduced a benchmarking pipeline and a dataset of over 11,000 synthetic personas to evaluate how Large Language Models (LLMs) handle tropical and infectious diseases (TRINDs). While LLMs excel at standard medical exams like the USMLE, this study reveals significant performance gaps when models encounter the regional context shifts and localized health data common in low-resource settings. The research concludes that integrating specific environmental context and advanced reasoning techniques is essential for making LLMs reliable decision-support tools for global health. ## Development of the TRINDs Synthetic Dataset * Researchers created a dataset of 11,000+ personas covering 50 tropical and infectious diseases to address the lack of rigorous evaluation data for out-of-distribution medical tasks. * The process began with "seed" templates based on factual data from the WHO, CDC, and PAHO, which were then reviewed by clinicians for clinical relevance. * The dataset was expanded using LLM prompting to include diverse demographic, clinical, and consumer-focused augmentations. * To test linguistic distribution shifts, the seed set was manually translated into French to evaluate how language changes impact diagnostic accuracy. ## Identifying Critical Performance Drivers * Evaluations of Gemini 1.5 models showed that accuracy on TRINDs is lower than reported performance on standard U.S. medical benchmarks, indicating a struggle with "out-of-distribution" disease types. * Contextual information is the primary driver of accuracy; the highest performance was achieved only when specific symptoms were combined with location and risk factors. * The study found that symptoms alone are often insufficient for an accurate diagnosis, emphasizing that LLMs require localized environmental data to differentiate between similar tropical conditions. * Linguistic shifts pose a significant challenge, as model performance dropped by approximately 10% when processing the French version of the dataset compared to the English version. ## Optimization and Reasoning Strategies * Implementing Chain-of-Thought (CoT) prompting—where the model is directed to explain its reasoning step-by-step—led to a significant 10% increase in diagnostic accuracy. * Researchers utilized an LLM-based "autorater" to scale the evaluation process, scoring answers as correct if the predicted diagnosis was meaningfully similar to the ground truth. * In tests regarding social biases, the study found no statistically significant difference in performance across race or gender identifiers within this specific TRINDs context. * Performance remained stable even when clinical language was swapped for consumer-style descriptions, suggesting the models are robust to variations in how patients describe their symptoms. To improve the utility of LLMs for global health, developers should prioritize the inclusion of regional risk factors and location-specific data in prompts. Utilizing reasoning-heavy strategies like Chain-of-Thought and expanding multilingual training sets are critical steps for bridging the performance gap in underserved regions.

Synthetic Data AI Large Language Models Gemini+5

googleMar 17, 2025Original article

Generating synthetic data with differentially private LLM inference (opens in new tab)

Researchers at Google have developed an inference-only method for generating differentially private (DP) synthetic data that avoids the high costs and data requirements associated with private fine-tuning. By prompting off-the-shelf large language models (LLMs) with sensitive examples in parallel and aggregating their outputs, the approach can generate thousands of high-quality synthetic data points while maintaining rigorous privacy guarantees. This method allows synthetic data to serve as a secure interface for model development, enabling teams to collaborate without requiring specialized knowledge of differential privacy. ## Differentially Private Prediction and Aggregation The core of this method relies on "private prediction," where privacy is applied to the model's output rather than the model itself. * Sensitive data points are distributed across multiple independent prompts, ensuring that no single individual's record can significantly influence the final output. * The LLM generates next-token predictions for each prompt in parallel, which are then aggregated to mask individual contributions. * The researchers designed a DP token sampling algorithm that treats the standard LLM "softmax" sampling process as a version of the exponential mechanism, a mathematical framework used to select the best option from a set while maintaining privacy. ## Enhancing Efficiency via KV Caching Previous attempts at private prediction were computationally expensive because they required a fresh batch of sensitive examples for every single token generated. * A new privacy analysis allows the system to reuse a fixed batch of sensitive examples across an entire generation sequence. * By maintaining the same context for each generation step, the system becomes compatible with standard inference optimization techniques like KV (Key-Value) caching. * This improvement enables the generation of synthetic data at a scale two to three orders of magnitude larger than prior methods. ## Optimizing Privacy Spend with Public Drafters To preserve the "privacy budget"—the limited amount of information that can be released before privacy is compromised—the method introduces a public drafter model. * The drafter model predicts the next token based solely on previously generated synthetic text, without ever seeing the sensitive data. * Using the sparse vector technique, the system only consumes the privacy budget when the public drafter’s suggestion disagrees with the private aggregate of the sensitive data. * This is particularly useful for structured data, where the drafter can handle formatting and syntax tokens, saving the privacy budget for the actual content. By leveraging off-the-shelf models like Gemma, this approach provides a scalable way to transform sensitive datasets into useful synthetic versions. These synthetic datasets are high-quality enough to replace real data in downstream machine learning tasks, such as in-context learning or fine-tuning models like BERT, without the risk of leaking individual user information.

Synthetic Data AI Large Language Models Machine Learning+5