Latest Transfer Learning Research Papers

The newest Transfer Learning papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Transfer Learning so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• Training Foundation Models with a Single Algorithm
  Yuda Song, Alvin Zhang, Fahim Tajwar, Daman Arora et al. · arXiv · Oct 29, 2026

  Foundation models are typically trained in stages: pretraining, midtraining, supervised fine-tuning (SFT), and reinforcement learning (RL), with changes in both data and learning algorithms. We show that this algorithmic separation is not f…

• Homebrewed SLMs: Autoregressive and Masked Diffusion Language Models Built at Home for Portuguese
  Pedro Carvalho Brom · arXiv · Oct 29, 2026

  Building a language model of one's own no longer requires a data centre. This tutorial trains one from scratch, tokenizer included, on one consumer GPU, and shows how to tell whether it is good for the tasks of a specialist domain. It propo…

• LCM-Promo: An LLM-Based Uplift Model for Promotion Allocation
  Nesta Midavaine, Floris Fok, Paulo Mendes, Joao Silva et al. · ECML/PKDD 2026 · Oct 27, 2026

  We present LCM-Promo, an LLM-based uplift model for promotion allocation. LCM-Promo is a task-specific Large Commerce Model (LCM): a finetuned LLM trained on transactional commerce data to predict user intent and promotion response. It lear…

• A note on data-efficient finetuning of vision foundation models
  Niclas Hergenröther, Antonio Orvieto · arXiv · Oct 27, 2026

  Adapting foundation models to a new task within or beyond the training domain is the de facto standard for building specialized machine learning models. Recently, Visual Prompting (VP) has emerged as a promising technique for efficient know…

• Follow the Gradient: Tracing Safety Degradation Back to Training Data
  Saket Reddy, Amro Abbas, Anshuman Suri, Ari Morcos et al. · arXiv · Oct 27, 2026

  Fine-tuning Large Language Models (LLMs) on instruction data can quietly erode their safety, even when no training example looks harmful. Because this safety-degrading data looks benign, standard content filters struggle to catch it. We tac…

• Interpretable sparse fine-tuning with TopKLoRA enables faithful circuit discovery
  Marek Masiak, Andrzej Szablewski, Nicola Cancedda, Constantin Venhoff · arXiv · Oct 23, 2026

  Fine-tuning on datasets of unknown origin can create a backdoor in the trained model when the data includes poisoned examples. Difficult to localise and remove, backdoors can persist through safety training. While parameter-efficient fine-t…

• Singular Geometry of Low-Rank Adaptation
  Kamillo Ferry, Anthea Monod, Keisuke Yano · arXiv · Oct 23, 2026

  Low-rank adaptation (LoRA) is now a standard approach to parameter-efficient fine-tuning of large neural networks. Its low-rank factorization is inherently non-identifiable, making LoRA a singular statistical model whose learning complexity…

• SpectralCanary: Embarrassingly Simple Low-Frequency Traces for T2I Dataset-Use Auditing
  Sy-Tuyen Ho, Anirudh Satheesh, Soham Chawathe, Huy Nghiem et al. · arXiv · Oct 22, 2026

  Auditing whether a text-to-image (T2I) model was fine-tuned on a particular image collection is difficult: a model can absorb collection-level visual characteristics without reproducing training images. Proactive dataset-use auditing plants…

• Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination
  Miguel Fernandez-de-Retana, Unai Zulaika, Ruben Sanchez-Corcuera, Aritz Bilbao-Jayo et al. · arXiv · Oct 22, 2026

  Accuracy on medical examinations is common evidence offered for the clinical use of language models, yet a health system that intends to run an open model on its own hardware needs to know considerably more: whether the model recognizes whe…

• When to Follow the Teacher: Advantage-Based On-Policy Distillation
  Mahdi Sabbaghi, George Pappas, Hamed Hassani, Adel Javanmard · arXiv · Oct 22, 2026

  Online Policy Distillation (OPD) often outperforms offline methods such as Supervised Fine-Tuning (SFT) by training on trajectories sampled directly from the student. However, standard OPD trains the student to follow the teacher at every s…

• BatchBench: Benchmarking AI Robustness to Batch Effects and Domain Shift in Biomedical Images - A JUMP Cell Painting Leaderboard
  Sonja Topf, Ana Sanchez-Fernandez, Günter Klambauer · arXiv · Oct 21, 2026

  Despite recent advances, current AI systems remain highly susceptible to domain shifts, particularly in biomedical imaging, where images are usually acquired under substantially different experimental conditions. Although recent domain adap…

• Kronecker Delta Rule for Long Context Linear Attention
  Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer et al. · arXiv · Oct 19, 2026

  We propose KronLSTM, a linear attention architecture based on the Kronecker Delta Rule, a novel update rule for associative matrix memories. KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associati…

• Scaling Parameter-Efficient Fine-Tuning for Brain Encoding
  Zhuofan Josh Ying, Guanglong Sun, Nolan Ho, Kieran Liu et al. · arXiv · Oct 19, 2026

  Brain encoding models aim to predict neural responses to stimuli. The most widely used approach fits readouts from frozen pretrained backbones. Using neural data to train large models from scratch or fully fine-tune them after task-pretrain…

• Behavioral Couplings: From Formation to Behavioral Emergence in Language Models
  Dongyoon Hahm, Woogyeol Jin, Dennis Wei, Yi Zhou et al. · arXiv · Oct 16, 2026

  Fine-tuning large language models (LLMs) can induce behaviors beyond those explicitly targeted. For example, training a model to respond more warmly can make it more sycophantic, and training it to write insecure code can cause broad misali…

• Knowledge That Is Linear Before Fine-Tuning Generalizes After It
  Faridoun Mehri, Hinrich Schütze · arXiv · Oct 15, 2026

  Not all facts a language model can recall support further learning equally well. We distinguish active facts, which support generalization beyond their own recall, from inert facts, which can be recalled but do not support such generalizati…

• Towards Continual Learning: Mitigating Forgetting With Subspace Tuning
  Arnav Chavan, Kusha Sareen, Rishabh Tiwari, Abhranil Chandra et al. · arXiv · Oct 15, 2026

  Large language models (LLMs) acquire new skills through fine-tuning, but it can destroy capabilities they already have: catastrophic forgetting remains a central obstacle to continual learning in LLMs. Recent works connect forgetting to how…

• Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidance
  Zhefei Gong, Hefeifei Jiang, Zehao Wang, Jiachen Li · arXiv · Oct 14, 2026

  Mobile Manipulation (MoMa) is a cornerstone of autonomous robotic systems, yet real-world fine-tuning remains a formidable challenge because of high-dimensional action spaces and intricate coupling between heterogeneous modules. In this pap…

• What Matters in Quality-Aware VLA Pretraining: A Benchmark and Empirical Study
  Mingxuan Yan, Litian Gong, Dihong Huang, Zhixu Li et al. · arXiv · Oct 13, 2026

  How should demonstration quality be used during vision-language-action (VLA) pretraining to improve performance after target-task fine-tuning? We compare weighting, filtering, and conditioning across reward definitions and label granulariti…

• Dream to Remember: Continual Steering of Generalist Robot Policies with World Models
  Bahey Tharwat, Iman Nematollahi, Marvin Alles, Xingyuan Zhang et al. · arXiv · Oct 9, 2026

  Generalist robot policies are trained with imitation learning and are bounded by the quality and coverage of their demonstrations. Reinforcement learning can improve and extend them, either by fine-tuning the policy weights or by training a…

• Overcoming Prior Barriers: Supervised Fine-Tuning under Long-Tail Distribution
  Haohui Wang, Jiahao Xu, Wangzhi Zhan, Tong Zeng et al. · arXiv · Oct 8, 2026

  Supervised fine-tuning (SFT) adapts pretrained large language models (LLMs) to downstream tasks, but the required concepts can receive substantially different levels of pretrained support. Frequent concepts are more likely to be well learne…

• Decoupling Exploration from Optimization in RLVR
  Saif Punjwani, Micah Goldblum · arXiv · Oct 7, 2026

  Modern language models undergo reinforcement learning with verifiable rewards (RLVR) on top of already-trained checkpoints. A key promise of RLVR is the discovery of new reasoning strategies. In principle, a model can sample novel ideas abs…

• Rephrase Before You Act: Characterizing and Mitigating Language Sensitivity in Vision-Language-Action Models
  Mikey Watts, Yuchen Cui · arXiv · Oct 7, 2026

  Vision-language-action models (VLAs) are strikingly sensitive to instruction phrasing and do not inherit the language robustness of the vision-language models they are built on. A one-word edit can move success by tens of points: $π_{0.5}$ …

• Q-Learning with Scalar Adjoint Matching
  Yonghoon Dong, Minsung Yoon, Jaehyuk Kim, Jungwoo Park et al. · arXiv · Oct 7, 2026

  Flow policies capture rich and diverse action distributions, and fine-tuning them with off-policy RL to improve beyond the demonstrations has drawn growing interest. However, fine-tuning a flow policy against a learned value function is not…

• Which Rollout Taught It That? BehaviorTrace and the Limits of Training-Data Attribution in Online RL
  Amit Nautiyal · arXiv · Oct 7, 2026

  When reinforcement learning teaches a language model a new behavior, can we find the training rollouts that taught it? And when an attribution method says it can, how do we know the answer is real? We study both questions on online RL fine-…

• AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model
  Sarim Hashmi, Mukul Ranjan, Kshitij Mishra, Mikhail Kuznetsov et al. · arXiv · Oct 6, 2026

  Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also …

• When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting
  Vedant Palit, Florent Draye, Nicolas Zucchet, Zhijing Jin et al. · arXiv · Oct 6, 2026

  Knowledge that a language model appears to forget during finetuning often remains stored and can be recovered, a phenomenon called spurious forgetting. Finetuning on new facts can even produce forgetting that undoes itself: recall of the ol…

• MemFLoRA: Memory-Floor LoRA for CNN Adaptation at the Edge
  Mehmet Emre Akbulut, Johannes Geier, Ulf Schlichtmann · arXiv · Oct 6, 2026

  On-device learning is necessary when the model encounters user-,sensor-, or environment-specific shifts after deployment. Although parameter-efficient fine-tuning (PEFT) methods, particularly Low-Rank Adaptation (LoRA) variants, enable effi…

• IdeaLens: Detecting AI Ideas in Long-form Writing
  Rishanth Rajendhran, Minjoon Choi, Jenna Russell, Ramya Namuduri et al. · arXiv · Oct 5, 2026

  While modern AI detectors identify who wrote the words, emerging policies on AI use increasingly hinge on a different question: who came up with the ideas? We introduce IdeaLens, a detector that identifies whether a document's ideas came fr…

• IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models
  Vladislav Gromadskii, David Li, Samson Gourevitch, Yazid Janati et al. · arXiv · Oct 2, 2026

  Masked discrete diffusion models offer a promising alternative to autoregressive generation, but iterative sampling can be costly, and intractable sequence likelihoods complicate reward fine-tuning. We introduce IDRF, a framework for reward…

• Learning to Undo: Transfer Reinforcement Learning under State Space Transformations
  [object Object], [object Object], [object Object] · NeurIPS 2026 Workshop PTA · Oct 2, 2026

  Transfer learning in reinforcement learning (RL) has shown strong empirical success. In this work, we take a more principled perspective by studying when and how transferring knowledge between MDPs (a source and a target) can be provably be…