Auto-Labelled Prompting for Remote Sensing Scene Classification
Imam, Mohamed
Imam, Mohamed
Author
Supervisor
Department
Machine Learning
Embargo End Date
2025-05-21
Type
Thesis
Date
2024
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
With the rise of large-scale pre-trained Vision-Language Models (VLMs), there s been a surge in interest in specific domains such as remote sensing, leading to the development of domain pre-trained VLMs like GeoRSCLIP. However, achieving robust model performance in remote sensing image scene classification without extensive labeling efforts remains a significant challenge. Current adaptation methods that lack labeled supervision typically involve full fine-tuning of the model. In this work, we introduce a label-free, lightweight adaptation approach called auto-labelled prompt tuning for remote sensing (ALP-RS). Our method capitalizes on the extensive contextual knowledge embedded within Large Language Models (LLMs) to construct a class description-based text embedding (CDTE) classifier. Leveraging this enriched CDTE classifier, we generate pseudo labels that align a pre-trained Semi-Supervised Learning (SSL) encoder, referred to as a text-aligned autolabel (TAAL) network, with the visual features within the VLM embedding space. The TAAL network, when aligned in the VLM space, serves as an auto-labeler for guiding the adaptation of the VLM vision encoder through prompt tuning. Our approach eliminates the necessity for manual choices during training, addressing concerns related to memory consumption for downstream adaptation and issues of overfitting. Through our experimental evaluation, we illustrate the effectiveness of our proposed method as it outperforms existing techniques, including those employing full fine-tuning, in terms of Top-1 accuracy. Furthermore, our method maintains competitive performance across various target datasets, showcasing its effectiveness in remote sensing image scene classification.
Citation
M. Imam, "Auto-Labelled Prompting for Remote Sensing Scene Classification", MS. Thesis, Machine Learning, MBZUAI, Abu Dhabi, UAE, 2024