How Effective are Self-supervised Models for Contact Identification in Videos
Peng, Kuan-Chuan Wang, Yizhou Li, Ziyue Chen, Zhenghua Yang, Jianfei Suh, Sungho Wu, Min
Peng, Kuan-Chuan
Wang, Yizhou
Li, Ziyue
Chen, Zhenghua
Yang, Jianfei
Suh, Sungho
Wu, Min
Supervisor
Department
Computer Vision
Embargo End Date
Type
Editorial
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
The exploration of video content via Self-Supervised Learning (SSL) models has unveiled a dynamic field of study, emphasizing both the complex challenges and unique opportunities inherent in this area. Despite the growing body of research, the ability of SSL models to detect physical contacts in videos remains largely unexplored, particularly the effectiveness of methods such as downstream supervision with linear probing or full fine-tuning. This work aims to bridge this gap by employing eight different convolutional neural networks (CNNs) based video SSL models to identify instances of physical contact within video sequences specifically. The Something-Something v2 (SSv2) and Epic-Kitchen (EK-100) datasets were chosen for evaluating these approaches due to the promising results on UCF101 and HMDB51, coupled with their limited prior assessment on SSv2 and EK-100. Additionally, these datasets feature diverse environments and scenarios, essential for testing the robustness and accuracy of video-based models. This approach not only examines the effectiveness of each model in recognizing physical contacts but also explores the performance in the action recognition downstream task. By doing so, valuable insights into the adaptability of SSL models in interpreting complex, dynamic visual information are contributed.
Citation
M. Gunawardhana, L. Sadith, L. David, D. Harari, and M. H. Khan, “How Effective are Self-supervised Models for Contact Identification in Videos,” Communications in Computer and Information Science, vol. 2201, pp. 117–131, Jan. 2025, doi: 10.1007/978-981-97-9003-6_8.
Source
Communications in Computer and Information Science
Conference
Keywords
Contact Identification, Self Supervised Learning, Videos
Subjects
Source
Publisher
Springer Nature