Latest Self-Supervised Learning Research Papers

The newest Self-Supervised Learning papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Self-Supervised Learning so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• Homebrewed SLMs: Autoregressive and Masked Diffusion Language Models Built at Home for Portuguese
  Pedro Carvalho Brom · arXiv · Oct 29, 2026

  Building a language model of one's own no longer requires a data centre. This tutorial trains one from scratch, tokenizer included, on one consumer GPU, and shows how to tell whether it is good for the tasks of a specialist domain. It propo…

• Improving Signals, Declining Encoder: A Known-Answer Test of Label-Free Monitoring in Self-Supervised Learning
  Nilotpal Kakati · arXiv · Oct 26, 2026

  Self-supervised training is monitored without labels, through the loss and statistics of the learned features. We report a case where we know the encoders lost part of what they had learned, so we can check whether these signals noticed. We…

• LoCA: Enabling Multi-Modality in Geospatial Location Encoders
  Panagiotis Koromilas, Eleanna Vali, Diogenis Antonopoulos, Eleni Saka et al. · arXiv · Oct 23, 2026

  Location encoders map geographic coordinates to high-dimensional descriptors through contrastive alignment with paired Earth Observation imagery. We prove that this construction is structurally limited: any Lipschitz location encoder produc…

• Turning TS2Vec into a Foundation Model for Time Series Classification
  Vasilii Feofanov, Zhenghan Tai, Andrei Ermakov, Irina Rish · arXiv · Oct 20, 2026

  TS2Vec is one of the strongest self-supervised methods for time series classification, introduced in 2022. It is designed to be trained anew on every dataset: self-supervised pre-training followed by linear probing. Since then, the field ha…

• WildMAE: Structure-Aware Masked Autoencoding for Wildlife Image Reconstruction
  Yicheng Zhang, Yujie Jin, Ke Zhang, Jiaxin Wu · arXiv · Oct 20, 2026

  Camera-trap imagery presents substantial variation in animal appearance, illumination, background clutter, and visual degradation. Masked autoencoders (MAEs) offer a label-free framework for modeling such data through reconstruction of rand…

• Kronecker Delta Rule for Long Context Linear Attention
  Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer et al. · arXiv · Oct 19, 2026

  We propose KronLSTM, a linear attention architecture based on the Kronecker Delta Rule, a novel update rule for associative matrix memories. KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associati…

• Generative Representation Learning for Scalable Latent Tokenization
  Alan Baade, Kyle Sargent, Vincent Sitzmann, Eric Ryan Chan et al. · arXiv · Oct 14, 2026

  We introduce an approach for self-supervised representation learning designed to be used as a tokenizer for generative modeling. Our objective supervises a representation loss by factorizing the pointwise mutual information of the data, whi…

• DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning
  Minghao Luo, Liang Chen · arXiv · Oct 13, 2026

  Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering …

• HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling
  Qun Dai, Liangjian Wen, Jiang Duan, Yong Dai et al. · arXiv · Oct 8, 2026

  Self-supervised multimodal representation learning has achieved remarkable success across diverse domains, yet capturing synergistic information remains challenging due to the complexity of cross-modal interactions. Unlike the shared inform…

• Decoupling Exploration from Optimization in RLVR
  Saif Punjwani, Micah Goldblum · arXiv · Oct 7, 2026

  Modern language models undergo reinforcement learning with verifiable rewards (RLVR) on top of already-trained checkpoints. A key promise of RLVR is the discovery of new reasoning strategies. In principle, a model can sample novel ideas abs…

• GeneICL: A Tabular Foundation Model for Bulk Transcriptomics
  Michael Bohl, Alexander Theus, David Wissel, Valentina Boeva · arXiv · Oct 6, 2026

  Gene expression is widely measured in biomedicine, yet clinical outcome prediction remains challenging due to high dimensionality, strong feature correlations, and limited labeled data. Large self-supervised transcriptomic foundation models…

• IdeaLens: Detecting AI Ideas in Long-form Writing
  Rishanth Rajendhran, Minjoon Choi, Jenna Russell, Ramya Namuduri et al. · arXiv · Oct 5, 2026

  While modern AI detectors identify who wrote the words, emerging policies on AI use increasingly hinge on a different question: who came up with the ideas? We introduce IdeaLens, a detector that identifies whether a document's ideas came fr…

• Image Classifiers are Efficient Self-Supervised Video Representation Learners
  Owais Iqbal, Sudipta Sarkar, Shyam Marjit, Omprakash Chakraborty et al. · arXiv · Sep 30, 2026

  We introduce VideoMSN, a Masked Siamese Network framework for efficient self-supervised spatio-temporal representation learning in videos. Instead of relying on heavy 3D architectures or reconstruction-based autoencoders for learning with u…

• Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
  Parsa Hosseini, Akasha Tigalappanavara, Sumit Nawathe, Chenrui Fan et al. · arXiv · Sep 25, 2026

  Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency either through inference-time early-stopping mechanisms or by explicitly encouraging sh…

• BeatGraph: Self-Supervised Heartbeat Graphs for Infant ECG Representations from the Home Environment
  Mohammad Nur Hossain Khan, M. S. Krafczyk, Beverly G. Bolster, Nancy McElwain et al. · arXiv · Sep 25, 2026

  Electrocardiogram (ECG) foundation models typically tokenize the signal into fixed-length patches that ignore cardiac structure, so a patch may split a heartbeat and the number of beats in each patch shifts with heart rate. This matters mos…

• Contrastive Learning for Authorship Verification
  Peter Kirby · arXiv · Sep 23, 2026

  Our results show that contrastive learning outperforms a classification-based approach to authorship verification under the tested settings. We identify loss function, batch size, training duration, pre-trained model, input context length, …

• MiTHras: Task-specific Hierarchical Semi-supervised Contrastive Masked Autoencoder for Mitotic Figure Analysis
  Trinh T. L. Vuong, Simon Graham, Quoc Dang Vu, Phat T. H. Ho et al. · arXiv · Sep 21, 2026

  Mitotic figure (MF) analysis supports tumor grading and prognostic assessment, but automated models remain sensitive to differences in tissue type and image acquisition. We present MiTHras, a task-specific pretraining framework that combine…

• Type-IV Code Clone Detection via Layer-Wise Non-Contrastive Representation Learning
  Luciano Marchezan, Kevin Delcourt, Eugene Syriani, Houari Sahraoui · arXiv · Sep 15, 2026

  Software clones are fragments of code that are similar or functionally equivalent to each other. They pose significant challenges for maintenance, refactoring, and bug detection. Detecting Type-IV clones, which are semantically equivalent b…

• Goal-oriented probabilistic forecasting for dynamic PRB allocation in 5G networks
  Oier Larumbe-Lizarraga, Roberto Pereira, Cristian J. Vaca-Rubio · arXiv · Sep 15, 2026

  Efficient physical resource block (PRB) allocation in 5G networks requires accurate demand forecasting. Conventional methods minimize symmetric error metrics (MAE, RMSE), ignoring the operational cost asymmetry where under-provisioning (ser…

• Self-Supervised Continuous Dynamics Learning
  ICLR 2027 Conference Submission · Sep 3, 2026

  Temporal modeling is essential for general intelligence to understand change and anticipate how the world evolves. Consecutive frames in a video often share similar semantics while carrying different state information. The dynamic state-tra…

• LoRA-TSD: Tangent-Space Spectral Descent for LoRA via Muon-Style Updates
  Dmitrii Andriianov, Andrey Veprikov, Aleksandr Beznosikov · arXiv · Sep 2, 2026

  Low-rank adaptation (LoRA) is the standard way to fine-tune large models, yet when its two factors are trained independently, the update ignores the geometry of the low-rank weight change it induces. We introduce LoRA-TSD, an optimizer that…

• Sycophantic Agreement Transfers with Neutral Data via Contrastive Preference Optimization
  Camila Blank, Zhuofan Ying, Christopher Potts, Peter Hase et al. · arXiv · Aug 31, 2026

  Sycophantic agreement refers to a behavior in which language models excessively affirm the user, often at the cost of factual accuracy. Although sycophantic agreement is a well-known failure of model alignment, there is limited understandin…

• Self-Supervised Learning as Discrete Communication
  Kawtar Zaher, Ilyass Moummad, Olivier Buisson, Alexis Joly · ICML 2026 regular · Apr 30, 2026

  Most self-supervised learning (SSL) methods learn continuous visual representations by aligning different views of the same input, offering limited control over how information is structured across representation dimensions. In this work, w…

• On the Alignment Between Supervised and Self-Supervised Contrastive Learning
  Achleshwar Luthra, Priyadarsi Mishra, Tomer Galanti · ICLR 2026 Poster · Jan 26, 2026

  Self-supervised contrastive learning (CL) has achieved remarkable empirical success, often producing representations that rival supervised pre-training on downstream tasks. Recent theory explains this by showing that the CL loss closely app…

• Anomaly detection in encrypted network traffic using self-supervised learning
  Sadaf Sattar, Shumaila Khan, M. I. Khan, A. Akhmediyarova et al. · Scientific Reports · Jul 22, 2025

  Privacy and security in network communication have been enhanced via encryption and traditional anomaly detection methods are no longer effective because of their payload inspection. In this paper, we describe ET-SSL, a new approach for enc…

• Physics-driven self-supervised learning for fast high-resolution robust 3D reconstruction of light-field microscopy
  Zhi Lu, Man-Chang Jin, Shuai Chen, Xiao-Ge Wang et al. · Nature Methods · May 12, 2025

  Light-field microscopy (LFM) and its variants have significantly advanced intravital high-speed 3D imaging. However, their practical applications remain limited due to trade-offs among processing speed, fidelity, and generalization in exist…

• Joint Supervised and Self-supervised Learning for MRI Reconstruction
  George Yiasemis, Nikita Moriakov, Clara I. Sánchez, Jan-Jakob Sonke et al. · MIDL 2025 Poster · Mar 27, 2025

  Magnetic Resonance Imaging (MRI) is a crucial modality but, its inherently slow acquisition process poses challenges in obtaining fully-sampled $k$-space data under motion. The lack of fully-sampled acquisitions, serving as ground truths, c…

• Deep Audio Features and Self-Supervised Learning for Early Diagnosis of Neonatal Diseases: Sepsis and Respiratory Distress Syndrome Classification from Infant Cry Signals
  Somaye Valizade Shayegh, C. Tadj · Electronics · Jan 9, 2025

  Neonatal mortality remains a critical global challenge, particularly in resource-limited settings with restricted access to advanced diagnostic tools. Early detection of life-threatening conditions like Sepsis and Respiratory Distress Syndr…

• Understanding Self-supervised Learning as an Approximation of Supervised Learning
  Byeongchan Lee · Submitted to ICLR 2025 · Sep 26, 2024

  Self-supervised representation learning has mainly advanced in an empirical rather than theoretical manner. Many successful algorithms combine multiple techniques that are supported by experiments. This approach makes it difficult for the c…

• Self-Supervised Adversarial Training via Diverse Augmented Queries and Self-Supervised Double Perturbation
  Ruize Zhang, Sheng Tang, Juan Cao · NeurIPS 2024 poster · Sep 25, 2024

  Recently, there have been some works studying self-supervised adversarial training, a learning paradigm that learns robust features without labels. While those works have narrowed the performance gap between self-supervised adversarial trai…