Latest Efficient ML Research Papers

The newest Efficient ML papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Efficient ML so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• The Role of Feedback Alignment in Self-Distillation
  Semih Kara, Oğuzhan Ersoy · arXiv · Jun 9, 2026

  Conditioning a language model on additional context, such as feedback on a previous attempt, typically improves its response. Self-distillation trains the model to retain this improvement when the context is not present. The method works by…

• Algorithmic and Minimax Complexities in Kernel Bandits
  Yunbei Xu · arXiv · Jun 9, 2026

  Gaussian-process upper confidence bound (GP-UCB) and decision-estimation-coefficient (DEC) methods may appear, at first sight, to belong to different theories. This paper places the two viewpoints in a common algorithmic-information languag…

• Itô maps for any-step SDEs
  Zhengkai Pan, Peter Potaptchik, Wenxi Yao, Michael S. Albergo et al. · arXiv · Jun 9, 2026

  Recent one-step generative models accelerate sampling by learning deterministic flow maps of the underlying dynamics. These methods rely on learning from ordinary differential equations, leaving open how to define an exact distillation proc…

• Efficiently Learning Drifting Halfspaces with Massart Noise
  Mingchen Ma, Guyang Cao, Jelena Diakonikolas, Ilias Diakonikolas · arXiv · Jun 9, 2026

  We study the problem of learning a drifting concept in the presence of Massart noise. In this framework, an online learner has access to a history of independent samples whose labels are noisy versions of a target concept that may change fr…

• OncoTraj: a public benchmark for longitudinal resistance prediction in EGFR-mutant non-small-cell lung cancer on osimertinib
  Abhijoy Sarkar, Aarchi Singh Thakur · arXiv · Jun 9, 2026

  Resistance to first-line osimertinib in EGFR-mutant non-small-cell lung cancer (NSCLC) is the canonical example of predictable clonal evolution under therapeutic pressure, yet no public benchmark exists for training or evaluating computatio…

• Data assimilation for subsurface flow using latent diffusion model parameterization: performance of ensemble-Kalman and Monte Carlo techniques
  Guido Di Federico, Wenchao Teng, Louis J. Durlofsky · arXiv · Jun 9, 2026

  Data assimilation (DA) in subsurface flow entails calibrating model parameters to match observed data, typically at wells, while preserving geological realism. Latent diffusion models (LDMs) provide efficient mappings from high-dimensional …

• Robust Regression of General ReLUs with Queries
  Ilias Diakonikolas, Daniel M. Kane, Mingchen Ma · arXiv · Jun 9, 2026

  We study the task of agnostically learning general (as opposed to homogeneous) ReLUs under the Gaussian distribution with respect to the squared loss. In the passive learning setting, recent work gave a computationally efficient algorithm t…

• TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning
  Heming Zou, Qi Wang, Yun Qu, Yuhang Jiang et al. · arXiv · Jun 9, 2026

  Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models. However, rollout-intensive policy optimization is often limited by insufficient reward cont…

• Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization
  Ming Sun, Kun Yuan · arXiv · Jun 5, 2026

  Decentralized stochastic optimization is a fundamental paradigm for large-scale learning over networks, where agents communicate only with their neighbors and no central coordinator is required. For strongly convex problems, communication e…

• CoMetaPNS: Continually Meta-learning Personalized Neural Surrogates for Cardiac Electrophysiology Simulations
  Ryan Missel, Xiajun Jiang, Linwei Wang · arXiv · Jun 5, 2026

  Personalized virtual heart simulations face challenges in model personalization and computational cost. While neural surrogates offer state-of-the-art solutions, they typically address either efficient personalization or training generaliza…

• Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation
  Mohammadreza Sadeghi, Sareh Soleimani, Zihan Wang, Narges Armanfard · arXiv · Jun 5, 2026

  Unsupervised Continual Learning (UCL) aims to enable neural networks to learn sequential tasks without labels or access to past data. A major challenge in this setting is Catastrophic Forgetting, where models forget previously learned tasks…

• Amortized Neural Optimization for Pre-Layout Signal Integrity Design Space Exploration using Differentiable Surrogates
  Julian Withöft, Werner John, Emre Ecik, Ralf Brüning et al. · arXiv · Jun 5, 2026

  Pre-layout design space exploration (DSE) for high-speed signal integrity (SI) analysis is often limited by the computational cost of simulations and iterative optimization algorithms within modern electronic design automation (EDA) workflo…

• TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
  Marius Dragoi, Ioana Pintilie, Alexandra Dragomir, Antonio Barbalau et al. · arXiv · Jun 4, 2026

  Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed referenc…

• MoNe: Modular Neural Memory for Efficient Long Context Inference
  Wonguk Cho, Kyubyung Chae, Tribhuvanesh Orekondy, Sunghyun Park et al. · AdaptFM Poster · Jun 1, 2026

  We present MoNe, a lightweight modular neural memory that attaches to any frozen pretrained Transformer to enable long-context inference without retraining. MoNe reads context in fixed-size segments via test-time learning of fast-weight neu…

• LExI: Layer-Adaptive Active Experts for Efficient MoE Inference
  Krishna Teja Chitty-Venkata, Murali Emani · AdaptFM Poster · Jun 1, 2026

  Mixture-of-Experts (MoE) models scale efficiently by activating only a subset of experts per token, offering a computationally sparse alternative to dense architectures. While prior post-training optimizations, such as inter- and intra-expe…

• Permissive Safety Through Trusted Inference: Verifiable Belief-Space Neural Safety Filters for Assured Interactive Robotics
  Haimin Hu · arXiv · Jun 1, 2026

  Autonomous robots that interact with people must make safe and efficient decisions under human-induced uncertainty, such as their preferences, goals, competency, and willingness to cooperate. Safety filters are a popular approach for ensuri…

• Speculative Sampling For Faster Molecular Dynamics
  Arthur Kosmala, Stephan Günnemann, Meng Gao, Brandon Wood · arXiv · Jun 1, 2026

  Molecular dynamics (MD) is a key tool for simulating the dynamical behavior of atomic systems. However, MD is inherently serial, which makes it difficult to increase single-system throughput with concurrent compute. To address this, we intr…

• On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
  Mind Lab, :, Song Cao, Vic Cao et al. · arXiv · Jun 1, 2026

  Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, …

• Efficient Test-Time Finetuning of LLMs via Convex Reconstruction and Gradient Caching
  Alaa Khamis, Alaa Maalouf · arXiv · May 28, 2026

  Test-time finetuning (TTFT) is a rapidly evolving paradigm that adapts a language model to each prompt by retrieving related sequences, updating the model on them, and then evaluating the prompt. However, TTFT is only practical if it is fas…

• SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?
  Sy-Tuyen Ho, Minghui Liu, Huy Nghiem, Furong Huang · arXiv · May 28, 2026

  Autonomous AI research agents aim to accelerate scientific discovery by automating the research pipeline, from hypothesis generation to peer review. However, existing benchmarks rarely test a fundamental bottleneck: whether Large Language M…

• Reasoning with Sampling: Cutting at Decision Points
  Felix Zhou, Anay Mehrotra, Quanquan C. Liu · arXiv · May 28, 2026

  Frontier reasoning models are produced by posttraining base language models with reinforcement learning. Recent work has challenged this by showing that sampling from a sharpened version of the base model's distribution, a so-called power d…

• Neural Operator-Based Surrogate Model for CFD:Helical Coil Steam Generator in Small Modular Reactor
  Minseo Lee, Seongmin Oh, Chaehyeon Song, Bumjin Cho et al. · arXiv · May 28, 2026

  Real-time thermal-hydraulic simulation is essential for digital twin (DT) technology that supports the safe and efficient operation of small modular reactors (SMRs). Computational fluid dynamics (CFD) provides high-fidelity flow analysis, b…

• PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
  Yangyi Huang, Ruotian Peng, Zeju Qiu, Jiale Kang et al. · arXiv · May 27, 2026

  Parameter-efficient finetuning (PEFT) has become the standard approach for adapting large language models, yet evaluations largely emphasize downstream accuracy while overlooking the retention of pretrained capabilities. We argue that PEFT …

• Ω-QVLA: Robust Quantization for Vision-Language-Action Models via Composite Rotation and Per-step Scaling
  Xinyu Wang, Mingze Li, Sicheng Lyu, Dongxiu Liu et al. · arXiv · May 27, 2026

  Vision-Language-Action (VLA) models unify perception, reasoning, and control within a single policy, yet their multi-billion-parameter backbones and diffusion-based action heads make on-device deployment prohibitively expensive. Prior quant…

• Stage-wise Distortion-Perception Traversal in Zero-shot Inverse Problems with Diffusion Models
  Jiawei Zhang, Ziyuan Liu, Leon Yan, Zhenyu Xiao et al. · arXiv · May 27, 2026

  The distortion-perception (D-P) tradeoff is a fundamental phenomenon of Bayesian inverse problems, which characterizes the inherent tension between distortion performance and perceptual quality. Enabling flexible traversal of the D-P tradeo…

• From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models
  Yuchen Liang, Ness Shroff, Yingbin Liang · arXiv · May 26, 2026

  Discrete diffusion models have achieved strong empirical performance in text and other symbolic domains, but, especially for uniform-rate models, they often require many steps to generate a single sample. Existing acceleration methods eithe…

• OrpQuant: Geometric Orthogonal Residual Projection for Multiplier-Free Power-of-Two Transformer Quantization
  Maoyang Xiang, Bo Wang, Tao Luo · arXiv · May 25, 2026

  The deployment of Large Language Models (LLMs) and Vision Transformers (ViTs) on edge devices is significantly constrained by memory limitations and the critical timing bottlenecks introduced by dense Multiply-Accumulate (MAC) arrays. In th…

• Active Query Synthesis for Preference Learning
  Namrata Nadagouda, Nauman Ahad, Maegan Tucker, Mark A. Davenport · arXiv · May 25, 2026

  Efficient learning of user preferences is crucial for many modern decision making systems but typically requires costly labeled data. Active learning reduces this cost, yet standard methods are computationally expensive due to pool-based ev…

• Accelerating Bayesian inverse design in computational fluid dynamics using neural operators
  Bipin Tiwari, Omer San · arXiv · May 25, 2026

  Bayesian inverse design provides a principled framework for inferring aerodynamic geometries from sparse flow observations while quantifying uncertainty. However, its practical use in computational fluid dynamics (CFD) is severely limited b…

• Efficient Parallelization of Message Passing Neural Network Potentials for Large-Scale Molecular Dynamics
  Junfan Xia, Bin Jiang · JACS Au · May 25, 2026

  Machine learning potentials have achieved great success in accelerating atomistic simulations, among which message passing neural networks (MPNNs) have become increasingly prevalent thanks to their superior accuracy. However, MPNN potential…