Latest Prompting & ICL Research Papers

The newest Prompting & ICL papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Prompting & ICL so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• Social Capital and Regeneration in Historical Areas: Case Study of Jianguo Men Market, Xi’an, China
  Min Wang · University of Liverpool · Jan 1, 2029

  In recent years, China has increasingly adopted more inclusive and participatory approaches to urban regeneration, prompted by constrained government funding and the limitations of traditional top-down models. Although existing literature l…

• REALMS-CG: Resilient Ensemble of Autonomous LLM Multi-agent with Spectral clustering for Code Generation
  Anonymous, Anonymous · Zenodo (CERN European Organ... · Sep 1, 2027

  This contains the actual source code, prompts, scripts and all experiment results as described in the paper:REALMS-CG: Resilient Ensemble of Autonomous LLM Multi-agent with Spectral clustering for Code Generation.This data is provided as pa…

• VeriCUA: Verification as Agentic Exploration for Computer-Using Agents
  Qiushi Sun, Da Yu, Alexander Knop, Amer Sinha et al. · arXiv · Oct 29, 2026

  Building and scaling computer-using agents (CUAs) across desktop, mobile, and web environments requires reliable task-completion verification for data curation, evaluation, and reinforcement learning. Typical automated approaches prompt vis…

• Homebrewed SLMs: Autoregressive and Masked Diffusion Language Models Built at Home for Portuguese
  Pedro Carvalho Brom · arXiv · Oct 29, 2026

  Building a language model of one's own no longer requires a data centre. This tutorial trains one from scratch, tokenizer included, on one consumer GPU, and shows how to tell whether it is good for the tasks of a specialist domain. It propo…

• Discovering Interpretable Acoustic Features in Autoregressive Voice-Cloning TTS Models
  Anuj Diwan, David Harwath, Eunsol Choi · arXiv · Oct 29, 2026

  Voice-cloning TTS models synthesize an unseen speaker's voice from brief speech prompts, but it remains unclear which acoustic properties they represent and use. We introduce a feature-based interpretability framework for autoregressive voi…

• Judged in Company: In-Context Batching Makes LLM Verdicts Depend on Other Items
  Peng Xie, Amr Alanwar · arXiv · Oct 29, 2026

  To save cost, practitioners often ask a language model to grade many independent items in one prompt. This assumes that verdicts are separable: that the verdict on an item does not depend on the other items in the prompt or on its position …

• Can LLMs Ignore What They Know? Measuring Knowledge Leakage in Persona-Based User Simulation
  Sangmin Lee, Sangwon Lee, Sungbeom Choi, Jeeseung Park et al. · arXiv · Oct 29, 2026

  When persona-conditioned responses are read as respondent-level simulations, each response should reflect the described persona's knowledge, and the respondent written to know nothing must act as if the model's knowledge were not there. We …

• Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness
  Igor Shilov, Xiaoxue Yang, Alexander Panfilov, Evgenii Kortukov et al. · arXiv · Oct 27, 2026

  We introduce Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates. HoS learns how each head should redistribute attention between trusted and untru…

• GEPA-Curator: Curated Learning for Self Improving Agents
  Zack Yu, Nicholas Lee, Minjae Lee, Lakshya Agarwal et al. · arXiv · Oct 27, 2026

  Large Language Models (LLMs) are increasingly used in multi-turn agentic workloads that require sustained interaction with complex environments. Existing reflective prompt optimizers such as GEPA learn from failed rollouts, but agentic fail…

• Can LLMs Uncover Hidden Physical Laws through Free Exploration?
  Zhen Yang, Wen Wang, Xander Xu, Ying-Cong Chen · arXiv · Oct 27, 2026

  Scientific discovery often begins with an unexpected observation that prompts a new research question. In a world with altered physical laws, can large language models (LLMs) notice unexpected behavior, such as heavier objects falling more …

• REGLAT at ArGuard Shared Tasks: An Ensemble-Based Model Integrating NER for Harmful Content Detection in Arabic
  Ahmed M. Fetouh, Mariam Labib, Nsrin Ashraf, Omer Dawood et al. · arXiv · Oct 27, 2026

  Detecting harmful content in Arabic is challenging because of dialectal variations and the semantic overlap between different harm categories. This paper presents a two-stage approach for classifying Arabic user prompts into seven categorie…

• Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces
  Jian Mu, Ruibo Shen, Yao Shu · arXiv · Oct 27, 2026

  Language model agents are post-trained and deployed through a harness, the prompts, tools, and control logic that present a task to the model and execute its outputs. Harnesses are often revised and judged by how an already trained model pe…

• Understanding Jev-Style System One Models: Does Structure Reduce Selection Bias?
  Mingyang Song, Mao Zheng, Zhiyuan Ma, Xinpeng Zhou · arXiv · Oct 26, 2026

  Selection bias in large language models (LLMs) can persist despite instructions to ignore presentation order or prompts that elicit reasoning. Recent efforts to reproduce Jev-style System One behavior have produced a growing family of model…

• An Algorithmic Account of Few-Shot Concept Induction in Language Models
  Michael A. Lepori, Laila Johnston, Aalok Sathe, Zhuonan Yang et al. · arXiv · Oct 26, 2026

  Both models and minds not only encode features that describe an entity, but can specifically represent which features of that entity are relevant for a given context. This core capability underlies in-context learning in language models and…

• S3AC: Stabilized, Selective, and Scalable Actor-Critic for LLM Reasoning
  Shuo Shen, Guangran Cheng, Chengqi Lyu, Xiangsen Wang et al. · arXiv · Oct 26, 2026

  Critic-free methods are widely used in reinforcement learning with verifiable rewards (RLVR) for long chain-of-thought (CoT) reasoning, avoiding difficult and costly critic training. Common alternatives lack token-level credit assignment an…

• Useful Diversity or Hallucinated Variety? Persona-Conditioned Synthetic Query Generation for E-Commerce Search
  Filippos Ventirozos, Tom Armitage, Mahmoud Oshagh, Peter A. Appleby et al. · arXiv · Oct 26, 2026

  Synthetic supervision can bootstrap semantic parsers in new search domains, but prompt-induced diversity is useful only when generations are faithful to the structured intent. We explore this for cold-start e-commerce search: 118,098 synthe…

• Ask Before You Answer: A Question-Driven Evidence Reasoning Architecture (QERA) for In-Context Learning
  Jinhu Qi, Minda Hu, Wentao Zhang, Zixing Song et al. · arXiv · Oct 26, 2026

  AI assistants are increasingly asked to reason over a user's own web traces: group chats, community threads, activity logs and revision histories. On CL-bench Life, a benchmark of such real-life context learning, frontier LLMs solve fewer t…

• Decoupling Patient Personas and Clinical Workflows: A Dual-Agent Approach to Realistic Medical Dialogue Generation
  Qiyuan Li, Xiaomei Guo, Chao Gao, Haijiang Liu et al. · arXiv · Oct 23, 2026

  Pre-consultation medical dialogues are essential for collecting comprehensive patient information, but are costly to obtain at scale due to privacy and ethical constraints. Existing LLM-based synthesis methods often rely on prompt engineeri…

• Diffusion Language Models for Perplexity-Aware Black-Box Attacks
  Ali Al-Lawati, Elsayed Eshra, Dongwon Lee, Suhang Wang · arXiv · Oct 22, 2026

  \emph{Continuous-space} attacks find prompts that trigger malicious behaviors, such as overthinking, in black-box LLMs by sampling or optimizing over a low-dimensional Gaussian latent space and decoding it to text through a \emph{random map…

• Towards User Simulators That Stay On Task And Still Sound Natural
  Mustafa Omer Gul, Philippe Laban, Tanya Goyal, Jennifer Neville · arXiv · Oct 22, 2026

  Users increasingly rely on AI systems for extended, multi-turn tasks, yet evidence indicates that LLM-based system performance degrades over prolonged interaction. To evaluate and improve AI assistants in such settings, we need user simulat…

• How Well Do Decision Models Transfer to Vietnamese? An Empirical Study of Text Classification
  Phat Huu Ha, Nguyen Xuan Phuc, Nong Nhut Quy, Trinh Tran Tran et al. · arXiv · Oct 22, 2026

  Recent advances in specialized decision models have expanded the possibilities for zero-shot classification, yet their effectiveness on Vietnamese remains underexplored. This study evaluates four specialized decision models across 13 classi…

• How Depth Growth and Looping Shape In-Context Learning in Language Models
  Ferdinand Kapl, Kaitlin Maile, Stefan Bauer, Johannes von Oswald · arXiv · Oct 22, 2026

  Looped and depth-grown language models reuse computations across depth and have attracted interest for their reasoning capabilities. Building on prior suggestive evidence of improved in-context-learning (ICL), we investigate how such reuse …

• Same Entity, Different Script: Diagnosing Cross-Script Entity Matching Failures in Language Models
  Athira Gopal, Patil Prathamesh Pravin, Ankush Kadu, Arpit Jain et al. · arXiv · Oct 22, 2026

  Multilingual language models can give different identity judgments when the same people's names are written in different scripts. We study this effect using 601 audited identity pairs with attested Devanagari and Latin forms, holding the qu…

• HERD: Harness-Evolving Recursive Distillation for Continual Learning
  Xun Sun, Thanh Thi Nguyen, Thuy-Trang Vu · arXiv · Oct 22, 2026

  Continual learning requires a model to keep acquiring new skills and knowledge without compromising its existing capabilities. Parametric methods suffer from catastrophic forgetting and loss of plasticity. Non-parametric methods, such as pr…

• CrisiLangs: A Multilingual Resource for Crisis Warning Generation
  Giacomo Gonella, Stefano Menini, Marco Guerini · arXiv · Oct 22, 2026

  Communicating with the public during an emergency entails issuing warning messages promptly. In Europe, it often requires producing them in more than one language. Recent work has shown that language models can be trained to generate warnin…

• Predicting Music Valence and Arousal via Textual and Multimodal In-Context Learning in Large Language Models
  Chi-En Dai, Chih-Pin Tan, Yi-Hsuan Yang · arXiv · Oct 22, 2026

  While arousal can be reliably predicted from musical audio, emotional valence remains difficult to model due to its subjective and context-dependent nature. In this work, we investigate the potential of in-context learning (ICL) with large …

• Investigating Multi-Cultural Value Selection Biases in LLMs
  Siddhesh Pawar, Arnav Arora, Dustin Wright, Dirk Hovy et al. · arXiv · Oct 22, 2026

  Large language models (LLMs) serve users whose cultural identities are multi-layered, yet cultural evaluations commonly reduce identity to one label. In this work, we study LLMs under simultaneous identity cues, with a country persona in th…

• Prompting Strategy Effects Depend on the Judge: A Multilingual Study of LLM-as-a-Judge Robustness
  Asal Mehrabi, Mohammad Aliannejadi, Clemencia Siro · arXiv · Oct 22, 2026

  LLM-as-a-Judge evaluation is increasingly applied to multilingual tasks, yet prompting strategies such as Chain-of-Thought and Role Conditioning are recommended based solely on English benchmarks. Whether their effectiveness transfers acros…

• Prompt Sensitivity in Tool-Augmented Model Comparisons
  Mateusz Modrzejewski, Tomasz Radzikowski · arXiv · Oct 22, 2026

  Language models are compared on question-answering benchmarks that pose each question in one wording, and a reworded question can change which of two models scores higher. We ask whether giving the models tools makes such comparisons more s…

• Mixing Matters: Evidence Position Bias Across Sequence Mixers in Long-Context Question Answering
  Aman Behera, Namit Solanki, Mehul Anand · arXiv · Oct 22, 2026

  Long-context models often use answer-bearing evidence less effectively when it appears away from a prompt boundary. We compare this position bias across Transformer, state-space, and hybrid Mamba architectures by moving one gold document th…