Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videos
Han, Mingfei Yang, Linjie Chang, Xiaojun Yao, Lina Wang, Heng
Han, Mingfei
Yang, Linjie
Chang, Xiaojun
Yao, Lina
Wang, Heng
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
A short clip of video may contain progression of multiple events and an interesting story line. A human need to capture both the event in every shot and associate them together to understand the story behind it. In this work, we present a new multishot video understanding benchmark Shot2Story with detailed shot-level captions, comprehensive video summaries and question-answering pairs. To facilitate better semantic understanding of videos, we provide captions for both visual signals and human narrations. We design several distinct tasks including single-shot video captioning, multi-shot video summarization, and multi-shot video question answering. Preliminary experiments show some challenges to generate a long and comprehensive video summary for multi-shot videos. Nevertheless, the generated imperfect summaries can already achieve competitive performance on existing video understanding tasks such as video question-answering, promoting an under-explored setting of video understanding with detailed summaries. © 2025 13th International Conference on Learning Representations, ICLR 2025. All rights reserved.
Citation
M. Han, L. Yang, X. Chang, L. Yao, and H. Wang, “Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videos,” International Conference on Representation Learning, vol. 2025, pp. 41665–41677, May 2025, Accessed: Jul. 22, 2025
Source
13th International Conference on Learning Representations, ICLR 2025
Conference
13th International Conference on Learning Representations, ICLR 2025
Keywords
Subjects
Source
13th International Conference on Learning Representations, ICLR 2025
Publisher
International Conference on Learning Representations, ICLR