Latest Video Understanding Research Papers

The newest Video Understanding papers from across the field — arXiv, NeurIPS, CVPR, Nature, and more — refreshed daily and ranked by relevance. Distill AI tracks Video Understanding so you don’t have to: get the standout work delivered to your inbox every morning, with 2-sentence summaries and the option to chat with any paper.

Recent papers

• A Framework for Understanding Moving Image Music Concerts: Repertoire, Reception and Recontextualisation
  Elizabeth Hunt · University of Liverpool · Jan 1, 2029

  This thesis investigates the phenomenon of moving image music concerts, focussing on the live orchestral presentation of music from film, television, and video games. With orchestras increasingly turning to such concerts to attract new audi…

• Video and digital games: Historical controversies, benefits, and risks for children and adolescents
  Hanna David · Journal of Child Developmen... · Dec 30, 2026

  Video and digital games have been the focus of sustained public and scientific debate since their commercial emergence in the late 1970s, particularly regarding their effects on children and adolescents. Early public and scientific discours…

• Multi-dimensional hierarchical temporal alignment for improved temporal commonsense reasoning in large language models
  Ge Yan, Hai-Tao Yu, Lei Chao, 30751052 et al. · Institutional Repositories ... · Nov 1, 2026

  Benefiting from recent advances in generative AI and large language models (LLMs), current LLM-driven AI systems are rapidly reshaping how we reason, plan, and make decisions. Yet endowing these systems with human-like temporal intelligence…

• Luggage re-identification: A study of strategies towards a real-world application
  Quentin Dhôte, Romaric Audigier · arXiv · Oct 29, 2026

  Luggage re-identification (ReID) is a critical task in video surveillance for forensics, security or border management (suspicious or abandoned luggage recognition), as well as luggage management (lost luggage recovery). It enables the asso…

• SonoWorld4D: From One Video to a 4D Audio-Visual Scene via Diffusion Synchronization
  Derong Jin, Xiyi Chen, Ming C. Lin, Ruohan Gao · arXiv · Oct 29, 2026

  Recent progress in visual scene generation has extended beyond static 3D scenes toward the more challenging problem of generating dynamic 4D scenes from minimal input. However, sound, despite being intrinsically coupled with dynamic objects…

• L3G-World: Real-Time Infinite World Generation with Latent 3D Gaussian Memory
  Jinhyeok Choi, JoungBin Lee, Jongmin Lee, Minkyeong Jeon et al. · arXiv · Oct 29, 2026

  Interactive world models generate visual content conditioned on user actions, yet maintaining spatial consistency over long-horizon exploration in real time remains challenging. Existing approaches rely on either temporal context or explici…

• Fight World Model: Customizable Combat through Visual Interaction Proxies
  Jiahua Li, Zixu John, Tom Zhong, Fuping Wu et al. · arXiv · Oct 29, 2026

  A playable, programmable game world must maintain persistent state, enforce executable rules, and support extensible gameplay mechanics. Existing video world models can generate plausible combat, yet struggle to keep hit reactions and healt…

• LucidLyra: Feed-Forward Generative 3D Scene Reconstruction and Joint Understanding via Multi-Task Distillation
  Oliver Hahn, Rodrigo Marcuzzi, Sangeetha Grama Srinivasan, Alessandro Burzio et al. · arXiv · Oct 29, 2026

  We present LucidLyra, a framework that makes generatively reconstructed 3D scenes directly queryable for semantics and instances in a feed-forward manner from video latents. While generative 3D reconstruction can synthesize virtual environm…

• AVPS-360: A Spatial Audio-Visual Benchmark for Instance Segmentation and Tracking of Sound-Producing Persons
  Yifu Sun, Parthasaarathy Sudarsanam, Tuomas Virtanen, Archontis Politis · arXiv · Oct 29, 2026

  Audio-visual instance segmentation (AVIS) segments and tracks individual sound-producing objects in videos. Existing benchmarks predominantly use monaural audio, which lacks inter-channel cues for spatially distinguishing concurrent sound s…

• Continuous-Action World Models: Learning Cross-Embodiment Spatiotemporal Dynamics Priors
  Liangyu Li, Qingwen Liu · arXiv · Oct 29, 2026

  We introduce Continuous-Action World Models (CoAWM), a framework for learning cross-embodiment spatiotemporal dynamics priors that support robot action generation across embodiments and execution frequencies. At its core is a shared latent …

• Wizard: Manipulate Any World in Real Time via Agentic Control
  Yuzhe Wu, Zijun Lin, Yeying Jin · arXiv · Oct 29, 2026

  Recent advances in video world models have enabled the generation of increasingly realistic and interactive virtual environments, opening new possibilities for player-customizable gameplay. Agentic frameworks further advance this direction …

• ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning
  Chaehyun Kim, Chi-Pin Huang, Jisoo Kim, Min-Hung Chen et al. · arXiv · Oct 27, 2026

  Vision-Language-Action (VLA) models have made substantial progress in robotic manipulation, but their generalization remains constrained by the diversity and coverage of action-labeled robot demonstrations. Human videos provide much broader…

• Repurposing Video Diffusion Transformers for Cross-View Temporal Object Correspondence
  Siyoon Jin, Dahyun Chung, Honggyu An, Sangbeom Lim et al. · arXiv · Oct 27, 2026

  Cross-view object correspondence identifies the same object across views. However, existing methods often rely on frame-wise matching or sequential tracking, which can propagate correspondence errors under appearance changes or occlusions. …

• Perceiving and Predicting Dynamic Sports: Spatiotemporal Intelligence in Vision Models for Egocentric Tennis Video
  Tudor Braicu, Joseph DelPreto, Jared Boyer, Mónika Farsang et al. · arXiv · Oct 27, 2026

  Agile robots require spatiotemporal intelligence to interpret scenes, reason about interactions, and anticipate future events. Yet it remains unclear how well foundation vision models support these capabilities, particularly from a moving e…

• VideoLVR: Latent Visual Reasoning over Grounded Video Evidence
  Shilong Li, Xingyuan Bu, Hao Shi, Yun Xing et al. · arXiv · Oct 27, 2026

  Chain-of-thought reasoning has substantially advanced video understanding in multimodal large language models (MLLMs). However, language-based reasoning traces do not provide a reliable mechanism for models to revisit relevant visual eviden…

• Zero-Shot Multimodal Event Spotting in Soccer Broadcasts
  Sadia Kamal, Mahmudul Hasan, Tony Braskich · arXiv · Oct 27, 2026

  Automatic soccer event spotting supports highlight generation, video indexing, and sports analytics, but existing methods depend on large annotated datasets and soccer-specific training. We present a zero-shot framework that detects, classi…

• As the Story Unfolds: Watching a Film and Identifying Characters as a Human Does
  Zhongrui Gui, Junyu Xie, Tengda Han, Weidi Xie et al. · arXiv · Oct 27, 2026

  Films introduce characters incrementally as they progress. We study the problem of identifying principal characters and assigning their names directly from the movie itself, without using external cast lists, actor photographs, or pre-built…

• ReferStream: Interactive Subject Injection for Streaming Video Generation
  Shuolin Xu, Guanyuan Li, Boyu Li, Kelu Yao et al. · arXiv · Oct 27, 2026

  Injecting new subjects naturally during streaming video generation requires integrating newly supplied visual references into the existing video while preserving established subjects and the scene. Encoding references independently of the c…

• GroundControl: Satellite-Conditioned Multi-View Diffusion for Reconstruction-Free Novel View Synthesis
  Akshay Krishnan, Matias Turkulainen, Jamie Wynn, Zawar Qureshi et al. · arXiv · Oct 27, 2026

  Multi-view or video diffusion models have recently emerged as state-of-the-art approaches for novel-view synthesis from sparse image collections. While they can produce high quality images, they tend to hallucinate unrealistic spatial layou…

• PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
  Hanqing Wang, Zhaotian Cai, Ruichuan An, Bohan Zeng et al. · arXiv · Oct 27, 2026

  Vision--Language--Action (VLA) models generate robot actions from visual observations and language instructions, but their understanding of physical dynamics remains limited. Video world models can capture scene dynamics, yet generating fut…

• EmbeddingGemma 2: Unified and Efficient Multimodal Representations
  Henrique Schechter Vera, Sahil Dua, Jovan Andonov, Sara Smoot et al. · arXiv · Oct 27, 2026

  We present EmbeddingGemma 2, an open multimodal embedding model which natively represents text, images, audio, video, and combinations thereof in a unified space. Our approach introduces improvements such as simplified per-layer embeddings …

• TimeLineXR: An Interactive Exploration of Time in Vision-Language Models
  Nidham Tekaya, Markus Passecker, Matthias Zeppelzauer · arXiv · Oct 27, 2026

  Recently, it has been shown that vision-language models (VLMs) have a certain intrinsic understanding of time. They can estimate the temporal origin of objects and encode time along a low-dimensional, non-linear manifold inside their embedd…

• CodeCap-Bench: Can Code Serve as a Good Video Caption?
  Jiafu Tang, Yifan Xu, Eric Tang, Junjie Zhao et al. · arXiv · Oct 27, 2026

  Video captioning aims to preserve visual information through compact descriptions, predominantly using natural language. However, precisely expressing fine-grained spatial structures and temporal dynamics in natural language often leads to …

• Hierarchical Sparse Graph for Multi-dimensional Assignment in Event-based Tracking
  Muxi Zha, Yang Shang, Wenlong Zhang, Qifeng Yu et al. · arXiv · Oct 27, 2026

  Existing event-based tracking methods typically operate in an online manner, associating tracklets across adjacent batches. However, this limited temporal window often leads to failures in challenging scenarios, such as long-term occlusions…

• VibeVideoBench:Evaluating Multimodal Agents on Code-Driven Video Generation
  Muxi Diao, Wenbin Xu, Lele Yang, Xuyang Ma et al. · arXiv · Oct 26, 2026

  Diffusion video models produce striking clips, but they cannot reliably render an exact formula, a correct chart, a ruleconsistent game or narration synchronized to on-screen events. A different route is emerging: multimodal agents that wri…

• FL4N: Feed-forward Latent Model for 4D Novel View Synthesis
  Eunbeen Hong, Hyeonseo Yu, Honggyu An, Jooho Son et al. · arXiv · Oct 26, 2026

  4D novel view synthesis (NVS) aims to render dynamic scenes across viewpoints and timestamps, which requires jointly inferring 3D structure and temporal dynamics. Existing monocular feed-forward methods often rely on explicit 4D Gaussian re…

• Multimodal Concert Video Editing: System Design and Exploratory Evaluation
  Lennart Schiweck, Adrian Kempf, Michelle Retig, Moritz Feucht et al. · arXiv · Oct 26, 2026

  Editing multi-camera concerts requires coordinated decisions about musical relevance, visual quality and cut timing. We present a modular framework that combines instrument activity from multitrack audio, visible instruments and image sharp…

• VidEdit3D: Flexible 3D Gaussian Editing with Video Diffusion
  Yanming Yang, Chenxi Song, Tong Zhao, Juntao Xu et al. · arXiv · Oct 26, 2026

  Recent advancements in text-guided 3D editing, particularly with 3D Gaussian Splatting (3DGS), have shown promise but still lag behind the flexibility of 2D editing tools. Existing methods face a critical trade-off between preserving multi-…

• MatAnyone-E: Sparse Memory Propagation for Efficient High-Resolution Video Matting
  Shuyun Wang, Shangchen Zhou, Peiqing Yang, Yuekun Dai et al. · arXiv · Oct 26, 2026

  High-resolution video matting requires preserving fine details, such as hair strands and soft transparency, while maintaining consistency across frames. Memory-based methods achieve strong matting quality by matching current-frame features …

• Triage: Skip, Skim, or Refine Attention Tiles in Video Diffusion Transformers
  Yeonjae Kim, Omin Kwon, Seong Hoon Seo, Jae W. Lee · arXiv · Oct 26, 2026

  Attention over tens of thousands of visual tokens dominates the inference of video diffusion transformers (DiTs). Existing methods reduce this cost through sparsity, low precision, or both, but use a single precision for all kept tiles. We …