Latest Mixture of Experts Research Papers

The newest Mixture of Experts papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Mixture of Experts so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• Homebrewed SLMs: Autoregressive and Masked Diffusion Language Models Built at Home for Portuguese
  Pedro Carvalho Brom · arXiv · Oct 29, 2026

  Building a language model of one's own no longer requires a data centre. This tutorial trains one from scratch, tokenizer included, on one consumer GPU, and shows how to tell whether it is good for the tasks of a specialist domain. It propo…

• Routing-Aware Explanations for Mixture of Experts Graph Models in Malware Detection
  Hossein Shokouhinejad, Roozbeh Razavi‐Far, Griffin Higgins, Ali Akbar Ghorbani · Machine Learning and Knowle... · Oct 28, 2026

  Mixture of Experts offers flexible graph reasoning by combining multiple views of a graph through a learned router. However, existing Graph-MoE approaches do not explicitly account for input-dependent expert routing when generating post-hoc…

• Acoustic-Array Diagnosis of Coupled Planetary Gearbox Faults Under Controlled 3D Spatial Interference with Physically Motivated Structural Priors
  Xingtong Liu, Yuqing Wang, Mei Wang · Applied Sciences · Oct 28, 2026

  Non-contact acoustic arrays offer a complementary route for planetary-gearbox monitoring, but multiple sound sources introduce spatial ambiguity and masking. We propose a multi-modal mixture-of-experts framework (MM-MoE) with physically mot…

• CRISP: Co-designing Pruning and Scheduling for Efficient MoE Inference on Edge Servers
  Congwei Zhang, Youming Tao, Yifei Zou, Yu Yang et al. · arXiv · Oct 27, 2026

  Mixture-of-Experts (MoE) inference on memory-constrained edge platforms is often dominated by expert-weight movement rather than computation. Although each token activates only a small subset of experts, routing is input-dependent, so all e…

• Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data
  Geunhyoung Kim, Sung Ju Hwang · arXiv · Oct 27, 2026

  Mixture-of-Experts (MoE) models are bottlenecked by loading routed-expert weights from memory during interactive decoding at small batch sizes. Because most intermediate channels in a routed expert contribute little to any single token, cha…

• Agent-Conditioned Routing: Diverse Reasoning Paths Within One Frozen Model
  Ananth Kalyanasundaram, Ankush Kadu, Athira Gopal, Aswanth Krishnan · arXiv · Oct 27, 2026

  Generating several solutions can expose alternative methods, but additional trajectories often paraphrase one dominant approach. We investigate whether the conditional capacity of a frozen sparse mixture-of-experts (MoE) model can provide s…

• EnergyCal: Taming Loss Spikes in Muon-Family Optimizers for Sparse MoE Training
  Masato Fujitake · arXiv · Oct 27, 2026

  Sparse mixture-of-experts (MoE) language-model training can exhibit large loss spikes with Muon-family optimizers. We introduce EnergyCal, a lightweight rule that attenuates routed-expert updates according to the normalized squared Frobeniu…

• MOS: Towards Effective Smart Contract Vulnerability Detection through Mixture-of-Experts Tuning of Large Language Models
  Hang Yuan, Lei Yu, Zhirong Huang, Jingyuan Zhang et al. · ACM Transactions on Softwar... · Oct 26, 2026

  Smart contract vulnerabilities pose significant security risks to blockchain systems, potentially leading to severe financial losses. Existing methods face several limitations: (1) Program analysis-based approaches heavily rely on predefine…

• FLAME: improving legal case retrieval through factor-aware graph modeling and mixture-of-experts
  Peng Du, Yongwen Ren, Hui Liao, Hao Li et al. · Frontiers of Computer Science · Oct 26, 2026
• EchoGraph-FusionNet: Adaptive Disease-Graph and Mixture-of-Experts Learning for Predicting Echocardiographic Abnormalities from ECGs
  Çetin Mirzaoğlu, Tolga Çakmak, Melike Demir, Fatih Demir et al. · Life · Oct 25, 2026

  Background/Objectives: Cardiovascular diseases remain a major global health burden, while echocardiography, despite its diagnostic value, is not always readily accessible. This study proposes EchoGraph-FusionNet, a multimodal deep learning …

• Process-Context-Routed Recommendation: A Process-Mining-Guided Mixture-of-Experts Framework
  Imane El Alama, Hanae Sbaï, Soumaya El Mamoune · Information · Oct 25, 2026

  Recommender systems increasingly exploit sequential interactions, but higher-level process context is rarely used to organize model specialization. This study proposes a process-context-routed recommendation framework in which e-commerce in…

• Accordion: Elastic-Depth Inference via Fixed-Prefix Layer Looping
  Yudong Zhang, Wei Ding, Kaili Zheng, Yuchen Ding et al. · arXiv · Oct 23, 2026

  Large language models serve applications with diverse inference budgets, but training and maintaining separate models for each budget incurs substantial costs. Recent Looped Transformers enable flexible inference computation through paramet…

• Compositional Brain Age Profiles Reveal Disease-Related Neuroimaging Patterns Beyond the Brain Age Gap
  Wei Wang, Yihang Xiao, Jiajie Peng · Biology · Oct 22, 2026

  The brain age gap (BAG)—the difference between the predicted and chronological age—does not describe the imaging patterns underlying such deviation. To characterize these patterns, we developed a semantic-guided mixture-of-experts model usi…

• SPARE: Sparsity Allocation via Routing Entropy for Efficient MoEs
  Yifan Yang, Hanling Yi, Dongquan Yang, Feng Lin et al. · arXiv · Oct 21, 2026

  Mixture-of-Experts (MoE) architectures scale model capacity efficiently, yet their substantial memory footprint poses a major barrier to deployment. Expert pruning reduces this footprint, but most methods focus on which experts to remove an…

• Don't Skip the Baseline: A Controlled Comparison of Dynamic Expert Skipping in MoEs
  Jakob Steimle, Thomas Elsken, Daniel Ehrhardt, Lukas Rinder et al. · arXiv · Oct 20, 2026

  A growing body of work proposes dynamic expert-skipping methods for Mixture-of-Experts (MoE) models that adjust the number of experts activated per token during inference, aiming to preserve model quality while further reducing the number o…

• Fed-HARM: A Proposed Federated Hierarchical Adversarial Framework with Mixture-of-Experts Personalization for Multimodal Brain Tumor Segmentation
  Yathesh Kumar · Zenodo (CERN European Organ... · Oct 20, 2026

  Fed-HARM: A Proposed Federated Hierarchical Adversarial Framework with Mixture-of-Experts Personalization for Multimodal Brain Tumor Segmentation presents a conceptual federated learning framework for multimodal brain tumor segmentation usi…

• Kronecker Delta Rule for Long Context Linear Attention
  Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer et al. · arXiv · Oct 19, 2026

  We propose KronLSTM, a linear attention architecture based on the Kronecker Delta Rule, a novel update rule for associative matrix memories. KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associati…

• LaMER: Language-Conditioned Multi-Expert Routing for Weakly Supervised Referring Expression Comprehension
  Yu Yu Chang, Tzu-Hsuan Weng, Youssef Mohamed Amin, Mohaned Ahmed Elsayed et al. · arXiv · Oct 19, 2026

  Referring Expression Comprehension (REC) localizes the image region described by a natural language expression, underpinning applications such as autonomous navigation, human-robot interaction, and medical image analysis. Under weak supervi…

• An Empirical Study of Parameter-Efficient Fine-Tuning for Mixture-of-Experts LLMs
  Minglei Li, Peng Ye, Haonan He, Jingqi Ye et al. · arXiv · Oct 19, 2026

  Parameter-efficient fine-tuning (PEFT) methods are critical for adapting large models under resource-constrained settings. However, as mixture-of-experts (MoE) architectures, which offer efficient inference at massive parameter scales, gain…

• TempoMoE: Identifying and Steering Fast-Slow Thinking Experts for Efficient Multimodal Reasoning
  Xiyue Zhang, Keru Zhou, Ziyi Wang, Bing Hu et al. · arXiv · Oct 19, 2026

  Multimodal Mixture-of-Experts (MoE) reasoning models rely on long chain-of-thought (CoT) for complex tasks, yet often overthink easy inputs, increasing token cost and latency, and sometimes degrading accuracy. Existing methods optimize reas…

• THINKERS-X: multisequence MRI radiomics in a mixture-of-experts model for outcome prediction and exploratory dose-scenario evaluation in lung cancer brain metastases treated with radiosurgery
  Jheremy Sebastian Reyes, Constantinos G. Hadjipanayis, Ajay Niranjan · Clinical & Experimental Met... · Oct 17, 2026
• Boundary-aware Mixture-of-Experts with Calibrated Bayesian Uncertainty for Skin Cancer Detection and Multi-class Classification
  Preeti Kathuria, Deepa Nehra · International journal of in... · Oct 16, 2026

  Dermoscopic analysis is held back by three coupled problems: detection and classification are trained separately, one monolithic backbone must serve lesions whose evidence ranges from keratin micro-texture to longrange pigment topology, and…

• Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts
  TrungTin Nguyen, Hien D. Nguyen, Faïcel Chamroukhi, Geoffrey John McLachlan · Annals of the Institute of ... · Oct 16, 2026

  Abstract We study Lasso estimation for high-dimensional softmax-gated Gaussian mixture-of-experts (SGMoE) regression models, where the predictor dimension P may exceed the sample size N . Under bounded parameters, we establish a non-asympto…

• Rethinking Routing Exploration in Modern Mixture-of-Experts
  Bingcong Li, Hao Ma, Michael Muehlebach, Niao He et al. · arXiv · Oct 15, 2026

  Mixture-of-experts (MoE) models have become an important approach for scaling model parameters. Modern MoE architectures increasingly combine two design choices: fine-grained expert segmentation and shared experts. The former increases expr…

• Learnable Hodge-Laplacian Driven Mixture of Experts for Steady-PDE Surrogates on Coarse Unstructured Meshes
  Stavros Nousias, Akis Nousias, Konstantinos Gkrispanis · arXiv · Oct 14, 2026

  Current approaches for steady partial differential equations surrogate models on unstructured meshes apply a single mixing mechanism to the whole solution. This paper proposes an architecture that partitions the solution among vanilla trans…

• OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning
  Mohamed Bouadi, Nassim Bouarour, Aditya Tanna, Vinay Kumar Sankarapu · arXiv · Oct 14, 2026

  Tabular foundation models enable in-context learning by predicting unseen observations from a labeled support table without task-specific parameter updates. Despite recent advances, most existing approaches rely on dense architectures that …

• Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization
  Renjie Xie, Juncheng Yang, Aoting Hu, Yan Liang et al. · arXiv · Oct 14, 2026

  Mixture-of-experts (MoE) language models activate only a few experts per token, yet all expert weights must be stored. Mixed-precision quantization reduces this storage cost by assigning higher bit-widths to sensitive experts and lower bit-…

• Scaling Multilingual Mixture-of-Experts: How Language Coverage Changes Capacity and Compute Allocation
  Joel Dag, Nikit Srivastava, Hamada M. Zahera, Axel-Cyrille Ngonga Ngomo · arXiv · Oct 14, 2026

  Multilingual models share capacity across languages, so adding languages can hurt performance on existing ones. Mixture-of-experts models expand this capacity with little additional per-token computation. Given this extra capacity, does add…

• MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning
  Junlin Chen, Wenhao Wang, Junchao Xu, Qingsong Yao et al. · arXiv · Oct 14, 2026

  Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental graph learning, predictions are often dominated by recently learned classes, causing sever…

• Confidential Narrative Intelligence on a Local AI Box: Epistemic Symmetry Theory with Algebraic Mixture-of-Experts Routing
  Giovanni P Barabino · Zenodo (CERN European Organ... · Oct 14, 2026

  Financial institutions want to read the narrative of markets with language models, but much of what matters most — internal research, positions, committee notes, client communications — cannot leave their perimeter. This report describes ho…