GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
Danish, Muhammad Sohail Munir, Muhammad Akhtar Shah, Syed Roshaan Ali Kuckreja, Kartik Khan, Fahad Shahbaz Fraccaro, Paolo Lacoste, Alexandre Khan, Salman
Danish, Muhammad Sohail
Munir, Muhammad Akhtar
Shah, Syed Roshaan Ali
Kuckreja, Kartik
Khan, Fahad Shahbaz
Fraccaro, Paolo
Lacoste, Alexandre
Khan, Salman
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
While numerous recent benchmarks focus on evaluating generic Vision-Language Models (VLMs), they do not effectively address the specific challenges of geospatial applications. Generic VLM benchmarks are not designed to handle the complexities of geospatial data, an essential component for applications such as environmental monitoring, urban planning, and disaster management. Key challenges in the geospatial domain include temporal change detection, large-scale object counting, tiny object detection, and understanding relationships between entities in remote sensing imagery. To bridge this gap, we present GEOBench-VLM, a comprehensive benchmark specifically designed to evaluate VLMs on geospatial tasks, including scene understanding, object counting, localization, finegrained categorization, segmentation, and temporal analysis. Our benchmark features over 10,000 manually verified instructions and spanning diverse visual conditions, object types, and scales. We evaluate several state-of-the-art VLMs to assess performance on geospatial-specific challenges. The results indicate that although existing VLMs demonstrate potential, they face challenges when dealing with geospatial-specific tasks, highlighting the room for further improvements. Notably, the best-performing LLaVa-OneVision achieves only 41.7% accuracy on MCQs, slightly more than GPT-4o, which is approximately double the random guess performance. Our benchmark is publicly available at https://github.com/The-AI-Alliance/GEO-Bench-VLM.
Citation
M.S. Danish, M.A. Munir, S.R.A. Shah, K. Kuckreja, F.S. Khan, P. Fraccaro , et al., "GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks," 2026, pp. 7132-7142.
Source
2025 IEEE/CVF International Conference on Computer Vision (ICCV)
Conference
2025 IEEE/CVF International Conference on Computer Vision (ICCV)
Keywords
Subjects
Source
2025 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE