Latest Knowledge Distillation Research Papers

The newest Knowledge Distillation papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Knowledge Distillation so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• Domain-Specific Hallucination Detection in Large Language Models
  Varun Teja Chundru, Debasmita Biswas · arXiv · Sep 10, 2026

  Large language models generate fluent text that can contain unfaithful claims -- a phenomenon known as hallucination. We present a multi-signal detection pipeline combining fine-tuned DeBERTa-v3 classification, Monte Carlo (MC) Dropout unce…

• A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients
  Suwan Wu, Yumeng Lin, Pengcheng Yuan, Xiaolong Jiang · arXiv · Sep 10, 2026

  Per-token gating of forward/reverse KL losses has become a standard technique for on-policy knowledge distillation (OPD), but existing methods such as EOPD (Jin et al., 2026) and ToDi (Jung et al., 2025) each fix a single gating signal and …

• Negative Self-Distillation: Learning to Reason by Avoiding Flaws
  Rongcan Pei, Zhepei Wei, Shuyao Xu, Xinyu Zhu et al. · arXiv · Sep 10, 2026

  On-Policy Self-Distillation (OPSD) has emerged as a popular paradigm for large language model (LLM) self-improvement, allowing models to act as their own teachers by leveraging privileged information such as ground-truth solutions. However,…

• Edge-Computing-Oriented Lightweight State of Charge Estimation Method for Energy Storage Batteries
  Wenqiang Huang, Ting He, Wenlong Zhu · Journal of The Electrochemi... · Sep 9, 2026

  Abstract Long short-term memory (LSTM) networks have been widely applied to battery state-of-charge (SOC) estimation because of their capability to capture nonlinear battery dynamics and long-term temporal dependencies. However, deploying h…

• Effect of used cooking oil sludge loading on diesel-range fraction yield and fuel properties from ternary waste pyrolysis oil: Toward practical blendstock assessment
  Sommas Kaewluan, Natcha Yongying, Sittinun Tawkaew, Paranee Sriromreun · Next Energy · Sep 9, 2026

  Tri-pyrolysis oil (TPO) produced from oil palm fresh fruit bunches (PFFB), medical waste polypropylene bottles (MWPB), and used cooking oil sludge (UCOS) was separated by atmospheric fractional distillation into pyro-gasoline (PG), pyro-die…

• AnomalyMHKD: A multi-mode hybrid knowledge distillation approach for self-supervised detection of anomalous objects
  Bhuvana Jayaraman, T. T. Mirnalinee, Harini Mohan, Olirva M · Multimedia Tools and Applic... · Sep 9, 2026
• Filtered Distillation from a Large Vision Teacher for Infrared Small Target Detection
  Zhanxu Jiang, Wenbin Chen, Zhi Li, Zhen Yuan et al. · Remote Sensing · Sep 8, 2026

  Infrared target detection supports the continuous observation of traffic participants and low-altitude targets across aerial and fixed-view imaging settings, particularly under weak or changing illumination. However, infrared targets are of…

• A robust XAI guided structured pruning approach for low-complexity and high-performance food classification
  Keerthi Garisa, Ravi Kant Kumar · Scientific Reports · Sep 8, 2026

  Deep convolutional neural networks have become widely adopted for food image classification, but their deployment on resource constrained devices remains challenging due to computational overhead, model redundancy, and increased architectur…

• AKDF: Adaptive knowledge distillation for secure, efficient, copyright-protected model publishing
  Feng Jiang, Honghui Xu, Daehee Seo, Yongjoon Joe et al. · Tsinghua Science & Technology · Sep 8, 2026

  Abstract The rise of large language models (LLMs) has transformed natural language processing, powering applications from creative writing to code generation. However, their vast size and proprietary nature present two major challenges, inc…

• FedPerKD privacy aware personalized federated learning with knowledge distillation for edge oriented tomato leaf disease detection
  Sonam Gupta, Chin-Shiuh Shieh, Vishal Jain, Mong-Fong Horng et al. · Discover Applied Sciences · Sep 7, 2026

  Abstract Tomato leaf diseases threaten global food security by reducing crop yield and quality. Centralized deep-learning detectors require farms to share raw images, which raises privacy, bandwidth, and data-sovereignty concerns. We presen…

• Lightweight defect detection on mica sheets via focused distillation for industrial measurement
  Jianhong Huang, Bo Zhang, Jingnan Zhou, Tong Zhou et al. · Measurement Science and Tec... · Sep 7, 2026

  Abstract To address the dual demands of accuracy and efficiency in detecting irregular surface defects on mica sheets, this paper proposes a comprehensive solution that integrates the design of a vision system and a lightweight detection me…

• CSP-HD: confidence-sensitive progressive hierarchical distillation in cross-view geo-localization
  Hai Yang, Min Xu, Zhihong Xu, Chaoyu Zhu et al. · Multimedia Systems · Sep 6, 2026
• Uncertainty-aware knowledge distillation for semi-supervised 3D MRI segmentation
  Chenghao Qiu, Xian-Shi Zhang, Kai-Fu Yang, Yong-Jie Li · Information Processing & Ma... · Sep 6, 2026
• Single image dehazing via dual-branch spatial network with cross-attention knowledge distillation
  Bin Hu, Sai Yang, Wanzhi Wen, Yonghong Chen et al. · The Imaging Science Journal · Sep 5, 2026

  Single image dehazing is a typical ill-posed image restoration problem, and existing methods struggle to balance global feature extraction and computational efficiency. This paper proposes a dual-branch spatial network fusing frequency doma…

• Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation
  Siliang Liu, Mohammad Ghasemi, Sapan Patel, Amin Banitalebi-Dehkordi · arXiv · Sep 4, 2026

  Trade-up recommendation identifies higher-quality alternatives that preserve a customer's purchase intent while offering upgraded benefits. Large language models (LLMs) can reason about such distinctions, but applying them directly to hundr…

• M-FSAD-KD: Full-Link Multi-Granularity Distillation for SAR Object Detection
  Tong Yu, Kaina Xiong, Jun Liu, Guixing Cao et al. · Remote Sensing · Sep 4, 2026

  Multi-modal synthetic aperture radar (SAR)–optical object detectors raise detection accuracy by fusing complementary physical responses, but require both modalities to be simultaneously available at inference. When the optical stream become…

• DustVeil: Label-Free Real-Time Detection of Airborne Coal-Mine Dust in Camera Streams via Physically-Grounded Multi-Cue Fusion and Knowledge Distillation
  Ziming Huang, Yujia Wang, Kun Huang, Jianwei Yang et al. · Sensors · Sep 4, 2026

  Airborne dust plumes are difficult to localize in underground mine-face video because the scene is dark, illumination moves with machinery, and dust is confused with lamp bloom, reflective steel, and water-spray aerosol. We present DustVeil…

• Antimicrobial Activity and Antioxidant Potential of Peppermint Essential Oil (Mentha piperita L.) Against Selected Pathogenic Microorganisms Extracted via Hydrodistillation with Electromagnetic Field (EMF) Pre-treatment
  Lena Hussein Ali · Baghdad Journal of Biochemi... · Sep 4, 2026

  Background: Antimicrobial resistance is increasing concerns, and there is interest in natural antimicrobials including plant essential oils, which are potentially induced to be of higher quality by non-thermal methods such as low-frequency …

• SFD-KD: Structured Feature Decoupling Knowledge Distillation for Fracture Detection
  Xiangchun Yu, Ding Longjun, Wang Xin, Miaomiao Liang et al. · ACM Transactions on Computi... · Sep 4, 2026

  Accurate fracture detection in medical imaging is pivotal for intelligent orthopedic diagnostic systems, yet deploying high-capacity detection models on resource-constrained platforms remains a critical cyber-physical challenge. Conventiona…

• Quantum-enhanced hybrid-model compression using knowledge distillation
  Luigi Barbato, Massimo Esposito, Francesco Gargiulo · Quantum Machine Intelligence · Sep 4, 2026

  Abstract Quantum computing has emerged as a promising paradigm for addressing computational tasks intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore high…

• Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR
  Boyan Li, Bingsen Chen, Chenghao Yang, Ping Nie et al. · arXiv · Sep 3, 2026

  Reinforcement learning with verifiable rewards (RLVR) and on-policy distillation (OPD) have emerged as two dominant methods for post-training reasoning LLMs. Prior work uses OPD's dense token-level supervision to complement the sparse RL re…

• Anti-Forgetting Adaptive Teacher-Driven Knowledge Distillation for Medical Image Classification
  Tao Chen, Chuan Zhou, Yifan Wang, Lubomir M. Hadjiiski et al. · Applied Sciences · Sep 3, 2026

  Deep neural networks (DNNs) have achieved remarkable success in medical image classification, yet their performance remains sensitive to dataset size. Knowledge distillation (KD) alleviates this issue by transferring knowledge from a high-c…

• Collaborative Knowledge Distillation and Reinforcement Learning for Automated Ticket Triage in Large-Scale Production Systems
  Ruowei Fu, Yang Zhang, Shenglin Zhang, Xin Wu et al. · ACM Transactions on Softwar... · Sep 3, 2026

  In large-scale enterprise environments, growing system complexity makes failures inevitable, threatening business continuity and customer satisfaction. To maintain system stability, efficient ticket triage is crucial for timely incident res…

• Inverse Design of Copolyamides via Sequence‐Sensitive Multifidelity Hybrid Models and Teacher–Student Genetic Algorithm
  Sheng Chen, Chaoran Huang, Jianan Hong, Yuquan He et al. · Materials Genome Engineerin... · Sep 3, 2026

  ABSTRACT As a high‐performance polymeric material, the thermal stability, mechanical strength, and processability of copolyamides (co‐PAs) are determined by the structural design of diamines, diacids, and their stoichiometric ratios. Labor‐…

• DADE: difficulty-aware distillation-enhanced network for single image super-resolution
  Yuxuan Lin · Advances in Engineering Inn... · Sep 3, 2026

  Single image super-resolution (SISR) has achieved remarkable progress with deep neural networks, but the ever-growing model depth brings a heavy computational burden that limits practical deployment. Reducing the inference cost of SR networ…

• Lightweight Transformer-based knowledge distillation framework for high-dimensional spatiotemporal radiomics in breast cancer risk prediction
  Haofan Huang, Han Zhou, Kaibin Huang, Jie Yang et al. · Visual Computing for Indust... · Sep 3, 2026

  Abstract Breast cancer exhibits significant spatiotemporal heterogeneity. Traditional radiomics approaches usually rely on low-temporal-resolution imaging and discrete image phases, failing to capture the rapid and continuous kinetic evolut…

• Cliff: Learning Process Rewards from the First Mistake
  Peixuan Han, Runhui Wang, Ketan Ramaneti, Jie Hao et al. · arXiv · Sep 2, 2026

  Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for large language model (LLM) post-training, but its reliance on coarse outcome rewards leads to limited guidance on intermediate reasoning processes.…

• LoRA-TSD: Tangent-Space Spectral Descent for LoRA via Muon-Style Updates
  Dmitrii Andriianov, Andrey Veprikov, Aleksandr Beznosikov · arXiv · Sep 2, 2026

  Low-rank adaptation (LoRA) is the standard way to fine-tune large models, yet when its two factors are trained independently, the update ignores the geometry of the low-rank weight change it induces. We introduce LoRA-TSD, an optimizer that…

• Beyond Pseudo-Labels: Dual-Level Knowledge Distillation for Enhanced Deep Clustering
  Rehab Alnefaie, Mohamed Maher Ben Ismail, Ouiem Bchir · Algorithms · Sep 2, 2026

  This study introduces a novel clustering approach, namely Teacher–Student-based Deep Clustering (TSDC), that relies on intra- and inter-distillation-based feature representations. In fact, TSDC distils knowledge from (i) high- to low-respon…

• Near Real-Time Multi-Class Segmentation for Intravascular Optical Coherence Tomography using Knowledge Distillation
  Ruben van der Waerden, Rick Volleberg, Pierandrea Cancian, Joske van der Zande et al. · European Heart Journal - Di... · Sep 2, 2026

  Abstract Aims Intravascular optical coherence tomography (OCT) enables high-resolution imaging of the coronary vessel wall, but manual image interpretation is time-consuming and existing automated approaches often require high computational…