Latest Synthetic Data Research Papers

The newest Synthetic Data papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Synthetic Data so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• Mitigating Model Collapse through Dreaming Learning
  Matteo Benati, Alessandro Londei, Denise Lanzieri, Raffaello Mastromarino et al. · arXiv · Oct 21, 2026

  Training generative models on synthetic data causes loss of diversity and a decrease in performance, a phenomenon known as model collapse. We study this degeneration spectrally in a controlled Markov setting. The eigenvalue spectrum of the …

• Kronecker Delta Rule for Long Context Linear Attention
  Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer et al. · arXiv · Oct 19, 2026

  We propose KronLSTM, a linear attention architecture based on the Kronecker Delta Rule, a novel update rule for associative matrix memories. KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associati…

• Decoupling Exploration from Optimization in RLVR
  Saif Punjwani, Micah Goldblum · arXiv · Oct 7, 2026

  Modern language models undergo reinforcement learning with verifiable rewards (RLVR) on top of already-trained checkpoints. A key promise of RLVR is the discovery of new reasoning strategies. In principle, a model can sample novel ideas abs…

• IdeaLens: Detecting AI Ideas in Long-form Writing
  Rishanth Rajendhran, Minjoon Choi, Jenna Russell, Ramya Namuduri et al. · arXiv · Oct 5, 2026

  While modern AI detectors identify who wrote the words, emerging policies on AI use increasingly hinge on a different question: who came up with the ideas? We introduce IdeaLens, a detector that identifies whether a document's ideas came fr…

• FALCON: A Model and Dataset Agnostic Framework for Synthetic Data Generation for NL2SQL Pairs
  Darian Lee, Shannon Rumsey, Jack St. Clair, Xinyi Tang et al. · arXiv · Oct 2, 2026

  Relational databases are among the most widely deployed forms of structured knowledge, and natural language access to them requires grounding language onto schema entities and relations while handling the ambiguity inherent in how people ph…

• How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text
  Jenna Russell, Ben Glickenhaus, Katherine Thai, John Wieting et al. · arXiv · Sep 30, 2026

  Web text makes up the majority of pretraining data and is increasingly AI-generated. After applying FineWeb quality filtering, we find that 27.5% of tokens from June 2026 web data are labeled as AI-generated by Pangram, rising to 31.1% by A…

• Let Training Guide Selection: Online Synthetic Data Filtering via Real-Anchored Utility
  Yanran Wu, Sana Lakdawala, Renzo Tassara Miller, Chongyang Bai et al. · arXiv · Sep 24, 2026

  Synthetic data can scale training supervision when real-world data are limited, but noise and distribution mismatch can reduce its value. Existing synthetic data selection methods often emphasize fidelity or diversity rather than the learne…

• Contrastive Learning for Authorship Verification
  Peter Kirby · arXiv · Sep 23, 2026

  Our results show that contrastive learning outperforms a classification-based approach to authorship verification under the tested settings. We identify loss function, batch size, training duration, pre-trained model, input context length, …

• hyperbolix: Hyperbolic Deep Learning in JAX
  Timo Klein, Thomas Lang, Yllka Velaj, Sebastian Tschiatschek · arXiv · Sep 23, 2026

  We present hyperbolix, an open-source library for hyperbolic deep learning in JAX, built on Flax NNX. To our knowledge, it is the first comprehensive, general-purpose hyperbolic deep learning library in JAX. It includes six manifolds with a…

• Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data
  Matteo Marchi, João Pedro Silvestre, Bahman Gharesifard, Paulo Tabuada · arXiv · Sep 16, 2026

  Large Language Models (LLMs) are now routinely trained using synthetic data, since high-quality human data has been exhausted by the ever increasing needs of larger and larger models. However, recursive training on synthetic data frequently…

• Physics-Informed Deep Learning for False Ventricular Tachycardia Alarm Reduction in the ICU
  Athanasios Papastathopoulos-Katsaros, Alexandra Stavrianidi, Zhandong Liu · arXiv · Sep 8, 2026

  False ventricular tachycardia (VT) alarms are a leading contributor to alarm fatigue in intensive care units. We propose a deep learning framework combining a 1D SE-ResNet with ICU-realistic data augmentations and a physics-informed auxilia…

• LoRA-TSD: Tangent-Space Spectral Descent for LoRA via Muon-Style Updates
  Dmitrii Andriianov, Andrey Veprikov, Aleksandr Beznosikov · arXiv · Sep 2, 2026

  Low-rank adaptation (LoRA) is the standard way to fine-tune large models, yet when its two factors are trained independently, the update ignores the geometry of the low-rank weight change it induces. We introduce LoRA-TSD, an optimizer that…

• Learning a Size-Weight Frontier for Synthetic-Augmented Inference
  Chengpiao Huang, Kaizheng Wang · arXiv · Aug 28, 2026

  Synthetic data can improve statistical inference when real data are scarce, but naively treating synthetic samples as real data can introduce bias and lead to unreliable inference. We develop a general framework for synthetic-augmented infe…

• CorporateBench: Large-Scale Q&A Benchmarking with Temporal Knowledge Bases
  Sil Hamilton, Albert Yu Sun, Oscar J. Romero, Carl-Leander Henneking et al. · arXiv · Aug 27, 2026

  LLMs are increasingly able to answer complex questions about enterprise-scale document collections. But evaluation is hard: companies don't want to share internal communications, and synthetic datasets have been overly simple. We present Co…

• SynQuE: Estimating Synthetic Dataset Quality Without Annotations
  Arthur Chen, Victor Zhong · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  We introduce and formalize the Synthetic Dataset Quality Estimation (SYNQUE) problem: ranking synthetic datasets by their expected real-world task performance using only limited unannotated real data. This addresses a critical and open chal…

• ESDAE: Evaluating Synthetic Data for Agent Evaluation
  Shuaiqi Wang, Aadyaa Maddi, Zinan Lin, Giulia Fanti · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  Agent evaluation is often performed on static datasets of execution trajectories, but real traces may be sensitive, proprietary, or too small to support comprehensive testing. Practitioners may therefore replace or augment real datasets wit…

• Learning from Synthetic Data Improves Multi-hop Reasoning
  Anmol Kabra, Yilun Yin, Albert Gong, Kamilė Stankevičiūtė et al. · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  Reinforcement Learning (RL) has been shown to significantly boost reasoning capabilities of large language models (LLMs) in math, coding, and multi-hop reasoning tasks. However, RL fine-tuning requires abundant high-quality verifiable data,…

• PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models
  Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik, Vijay Prakash Dwivedi et al. · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train such models are rarely public due to privacy constraints. W…

• Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence
  Bingji Yi, Qiyuan Liu, Yuwei Cheng, Haifeng Xu · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  Synthetic data has been increasingly used to train frontier generative models. However, recent study raises key concerns that iteratively retraining a generative model on its self-generated synthetic data may keep deteriorating model perfor…

• Less is More: Adaptive Coverage Sampling for Synthetic Training Data
  Sasan Tavakkol, Max Springer, Mohammadhossein Bateni, Vincent Cohen-Addad et al. · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  Large Language Models (LLMs) enable rapid generation of synthetic training data for downstream classifiers, offering a solution when human-labeled data is costly, scarce, or time-sensitive. However, synthetic datasets suffer from systematic…

• EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors
  Amin Banayeeanzade, Qingchuan Yang, Deqing Fu, Spencer Hong et al. · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  High-quality data is essential for modern machine learning, yet many valuable corpora are sensitive and cannot be freely shared. Synthetic data offers a practical substitute for downstream development, and large language models (LLMs) have …

• ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models
  Krishna Teja Chitty-Venkata, Murali Emani · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  We develop ImageNet-Think-250K, a multimodal reasoning dataset designed to aid the development of Vision Language Models (VLMs) with explicit reasoning capabilities. Our dataset is built on 250,000 images from ImageNet-21k dataset, providin…

• Motion Capture is Not the Target Domain: Scaling Synthetic Data for Learning Motion Representations
  Firas Darwish, George Nicholson, Aiden Doherty, Hang Yuan · ICLR 2026 Workshop DATA-FM · Mar 2, 2026

  Synthetic data offers a compelling path to scalable pretraining when real-world data is scarce, but models pretrained on synthetic data often fail to transfer reliably to deployment settings. We study this problem in full-body human motion,…

• InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policy
  Yang Tian, Yu-Yin Yang, Yi-Man Xie, Ze-Tao Cai et al. · arXiv.org · Nov 20, 2025

  Recent works explore how real and synthetic data contribute to Vision-Language-Action (VLA) models'generalization. While current VLA models have shown the strong effectiveness of large-scale real-robot pre-training, synthetic data has not p…

• Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  Fei-Yang Kang, Newsha Ardalani, Michael Kuchnik, Youssef Emad et al. · Conference on Empirical Methods in Natural Language Processing · Oct 2, 2025

  Training data plays a crucial role in Large Language Models (LLM) scaling, yet high quality data is of limited supply. Synthetic data techniques offer a potential path toward sidestepping these limitations. We conduct a large-scale empirica…

• WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning
  Kuan Li, Zhong-Wang Zhang, Hui-Feng Yin, Rui Ye et al. · arXiv.org · Sep 16, 2025

  Transcending human cognitive limitations represents a critical frontier in LLM training. Proprietary agentic systems like DeepResearch have demonstrated superhuman capabilities on extremely complex information-seeking benchmarks such as Bro…

• BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
  Pratyush Maini, Vineeth Dorna, Parth Doshi, A. Carranza et al. · arXiv.org · Aug 14, 2025

  Recent advances in large language model (LLM) pretraining have shown that simply scaling data quantity eventually leads to diminishing returns, hitting a data wall. In response, the use of synthetic data for pretraining has emerged as a pro…

• Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use
  Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai et al. · arXiv.org · Apr 7, 2025

  Reinforcement learning has been shown to improve the performance of large language models. However, traditional approaches like RLHF or RLAIF treat the problem as single-step. As focus shifts toward more complex reasoning and agentic tasks,…

• Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  M. Nadăş, Laura Dioşan, Andreea Tomescu · IEEE Access · Mar 18, 2025

  This survey reviews how large language models (LLMs) are transforming synthetic training data generation in both natural language and code domains. By producing artificial but task-relevant examples, these models can significantly augment o…

• Synthetic data generation: a privacy-preserving approach to accelerate rare disease research
  Jorge M. Mendes, Aziz Barbar, Marwa Refaie · Frontiers Digit. Health · Mar 18, 2025

  Rare disease research faces significant challenges due to limited patient data, strict privacy regulations, and the need for diverse datasets to develop accurate AI-driven diagnostics and treatments. Synthetic data—artificially generated da…