Latest Continual Learning Research Papers

The newest Continual Learning papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Continual Learning so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• OncoTraj: a public benchmark for longitudinal resistance prediction in EGFR-mutant non-small-cell lung cancer on osimertinib
  Abhijoy Sarkar, Aarchi Singh Thakur · arXiv · Jun 9, 2026

  Resistance to first-line osimertinib in EGFR-mutant non-small-cell lung cancer (NSCLC) is the canonical example of predictable clonal evolution under therapeutic pressure, yet no public benchmark exists for training or evaluating computatio…

• Preserving Plasticity in Continual Learning via Dynamical Isometry
  Andries Rosseau, Robert Müller, Ann Nowé · ICML 2026 · Jun 8, 2026

  Continual training of deep neural networks under non-stationarity often leads to a progressive loss of plasticity, eventually limiting further learning. We relate plasticity to the empirical Neural Tangent Kernel, and identify dynamical iso…

• Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning
  Fatema Siddika, Md Anwar Hossen, Tanwi Mallick, Ali Jannesari · arXiv · Jun 5, 2026

  Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge. Existing methods typically treat parameters u…

• Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation
  Mohammadreza Sadeghi, Sareh Soleimani, Zihan Wang, Narges Armanfard · arXiv · Jun 5, 2026

  Unsupervised Continual Learning (UCL) aims to enable neural networks to learn sequential tasks without labels or access to past data. A major challenge in this setting is Catastrophic Forgetting, where models forget previously learned tasks…

• TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
  Marius Dragoi, Ioana Pintilie, Alexandra Dragomir, Antonio Barbalau et al. · arXiv · Jun 4, 2026

  Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed referenc…

• A Local Perturbation Theory for Cross-Domain Interference and Recovery in Multi-Domain RL
  Lei Yang, Siyu Ding, Deyi Xiong · arXiv · Jun 1, 2026

  Reinforcement learning (RL) post-training improves large language models (LLMs) on individual domains such as mathematical reasoning, code generation, question answering, and creative writing (CW), but training on one domain often degrades …

• AREA: Attribute Extraction and Aggregation for CLIP-Based Class-Incremental Learning
  Zhen-Hao Xie, Yu-Cheng Shi, Da-Wei Zhou · arXiv · May 27, 2026

  Class-Incremental Learning (CIL) is important in building real-world learning systems. In CLIP-based CIL, the model performs classification by comparing similarity between visual and textual embeddings obtained from template prompts, e.g., …

• COOPO: Cyclic Offline-Online Policy Optimization Algorithm
  Qisai Liu, Zhanhong Jiang, Joshua Russell Waite, Aditya Balu et al. · arXiv · May 18, 2026

  Offline reinforcement learning struggles with distributional shift and constrained performance due to static dataset limitations, while online RL demands prohibitive environment interactions. The recent advent of hybrid offline-to-online me…

• Learning, Fast and Slow: Towards LLMs That Adapt Continually
  Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal, Joseph E. Gonzalez et al. · arXiv · May 12, 2026

  Large language models (LLMs) are trained for downstream tasks by updating their parameters (e.g., via RL). However, updating parameters forces them to absorb task-specific information, which can result in catastrophic forgetting and loss of…

• ORBIT: Preserving Foundational Language Capabilities in GenRetrieval via Origin-Regulated Merging
  Neha Verma, Nikhil Mehta, Shao-Chuan Wang, Naijing Zhang et al. · arXiv · May 12, 2026

  Despite the rapid advancements in large language model (LLM) development, fine-tuning them for specific tasks often results in the catastrophic forgetting of their general, language-based reasoning abilities. This work investigates and addr…

• Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models
  Gongbo Zhang, Wen Wang, Ye Tian, Li Yuan · arXiv · Apr 29, 2026

  Diffusion large language models (dLLMs) offer parallel decoding and bidirectional context, but state-of-the-art dLLMs require billions of parameters for competitive performance. While existing distillation methods for dLLMs reduce inference…

• Cortex-Inspired Continual Learning: Unsupervised Instantiation and Recovery of Functional Task Networks
  Kevin McKee, Thomas Hazy, Yicong Zheng, Zacharie Bugaud et al. · arXiv · Apr 27, 2026

  Block-sequential continual learning demands that a single model both protect prior solutions from catastrophic forgetting and efficiently infer at inference time which prior solution matches the current input without task labels. We present…

• Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability
  Nicolae Filat, Ahmed Hussain, Konstantinos Kalogiannis, Elena Burceanu · arXiv · Apr 23, 2026

  Streaming Continual Learning (CL) typically converts a continuous stream into a sequence of discrete tasks through temporal partitioning. We argue that this temporal taskification step is not a neutral preprocessing choice, but a structural…

• Fine-Tuning Regimes Define Distinct Continual Learning Problems
  Paul-Tiberiu Iordache, Elena Burceanu · arXiv · Apr 23, 2026

  Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge. Despite substantial progress in benchmarking CL methods, comparative evaluations typically keep the fine-tuning regime fixed…

• Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systems
  Beining Wu, Jun Huang · arXiv · Apr 22, 2026

  Federated continual learning (FCL) allows distributed autonomous fleets to adapt collaboratively to evolving terrain types across extended mission lifecycles. However, current approaches face several key challenges: 1) they use uniform prot…

• VLA Foundry: A Unified Framework for Training Vision-Language-Action Models
  Jean Mercat, Sedrick Keh, Kushal Arora, Isabella Huang et al. · arXiv · Apr 21, 2026

  We present VLA Foundry, an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. Most open-source VLA efforts specialize on the action training stage, often stitching together incompatible pretraining pipelines…

• JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
  Alexandra Dragomir, Ioana Pintilie, Antonio Barbalau, Marius Dragoi et al. · arXiv · Apr 17, 2026

  Adapter-based methods have become a cost-effective approach to continual learning (CL) for Large Language Models (LLMs), by sequentially learning a low-rank update matrix for each task. To mitigate catastrophic forgetting, state-of-the-art …

• MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systems
  Qingyao Ai, Yichen Tang, Changyue Wang, Jianming Long et al. · arXiv.org · Oct 20, 2025

  Scaling up data, parameters, and test-time computation has been the mainstream methods to improve LLM systems (LLMsys), but their upper bounds are almost reached due to the gradual depletion of high-quality data and marginal gains obtained …

• BiLoRA: Almost-orthogonal Parameter Spaces for Continual Learning
  Hao Zhu, Yifei Zhang, Junhao Dong, Piotr Koniusz · Computer Vision and Pattern Recognition · Jun 10, 2025

  Continual learning requires models to learn tasks sequentially while maintaining a delicate balance between stability (retaining knowledge of previous tasks) and plasticity (adapting to new tasks). A key challenge is preventing interference…

• MLLM-CL: Continual Learning for Multimodal Large Language Models
  Hongbo Zhao, Fei Zhu, Meng Wang, Rundong Wang et al. · arXiv.org · Jun 5, 2025

  Recent Multimodal Large Language Models (MLLMs) excel in vision-language understanding but face challenges in adapting to dynamic real-world scenarios that require continuous integration of new knowledge and skills. While continual learning…

• CL-CaGAN: Capsule Differential Adversarial Continual Learning for Cross-Domain Hyperspectral Anomaly Detection
  Jianing Wang, Siying Guo, Zheng Hua, Runhu Huang et al. · IEEE Transactions on Geoscience and Remote Sensing · May 17, 2025

  Anomaly detection (AD) has attracted remarkable attention in hyperspectral image (HSI) processing fields, and most existing deep learning (DL)-based algorithms indicate dramatic potential for detecting anomaly samples through specific train…

• Human-Guided Continual Learning for Personalized Decision-Making of Autonomous Driving
  Haohan Yang, Yanxin Zhou, Jingda Wu, Haochen Liu et al. · IEEE transactions on intelligent transportation systems (Print) · Apr 1, 2025

  Learning-based techniques hold considerable promise in achieving human-like autonomous driving. However, one deployed policy encounters difficulties in satisfying the drivers’ diverse decision-making preferences simultaneously. Meanwhile, t…

• Language Guided Concept Bottleneck Models for Interpretable Continual Learning
  Lu Yu, Haoyu Han, Zhe Tao, Hantao Yao et al. · Computer Vision and Pattern Recognition · Mar 30, 2025

  Continual learning (CL) aims to enable learning systems to acquire new knowledge constantly without forgetting previously learned information. CL faces the challenge of mitigating catastrophic forgetting while maintaining interpretability a…

• LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learning
  Xuan Liu, Xiaobin Chang · Computer Vision and Pattern Recognition · Mar 23, 2025

  In continual learning (CL), catastrophic forgetting often arises due to feature drift. This challenge is particularly prominent in the exemplar-free continual learning (EFCL) setting, where samples from previous tasks cannot be retained, ma…

• Accurate Forgetting for Heterogeneous Federated Continual Learning
  Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan et al. · International Conference on Learning Representations · Feb 20, 2025

  Recent years have witnessed a burgeoning interest in federated learning (FL). However, the contexts in which clients engage in sequential learning remain under-explored. Bridging FL and continual learning (CL) gives rise to a challenging pr…

• From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
  Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou et al. · International Conference on Machine Learning · Feb 20, 2025

  Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language mod…

• On the Computation of the Fisher Information in Continual Learning
  Gido M. van de Ven · arXiv.org · Feb 17, 2025

  One of the most popular methods for continual learning with deep neural networks is Elastic Weight Consolidation (EWC), which involves computing the Fisher Information. The exact way in which the Fisher Information is computed is however ra…

• Federated Continual Learning: Concepts, Challenges, and Solutions
  Parisa Hamedi, R. Razavi-Far, Ehsan Hallaji · Neurocomputing · Feb 10, 2025

  Federated Continual Learning (FCL) has emerged as a robust solution for collaborative model training in dynamic environments, where data samples are continuously generated and distributed across multiple devices. This survey provides a comp…

• Spurious Forgetting in Continual Learning of Language Models
  Junhao Zheng, Xidi Cai, Shengjie Qiu, Qianli Ma · International Conference on Learning Representations · Jan 23, 2025

  Recent advancements in large language models (LLMs) reveal a perplexing phenomenon in continual learning: despite extensive training, models experience significant performance declines, raising questions about task alignment and underlying …

• Continual Learning in Medicine: A Systematic Literature Review
  Pierangela Bruno, A. Quarta, Francesco Calimeri · Neural Processing Letters · Jan 7, 2025