On the importance of language-driven representation learning for heterogeneous federated learning
Yan, Yunlu Feng, Chun-Mei Zuo, Wangmeng Khan, Salman Zhu, Lei Liu, Yong
Yan, Yunlu
Feng, Chun-Mei
Zuo, Wangmeng
Khan, Salman
Zhu, Lei
Liu, Yong
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Non-Independent and Identically Distributed (Non-IID) training data significantly challenge federated learning (FL), impairing the performance of the global model in distributed frameworks. Inspired by the superior performance and generalizability of language-driven representation learning in centralized settings, we explore its potential to enhance FL for handling non-IID data. In specific, this paper introduces FedGLCL, a novel language-driven FL framework for image-text learning that uniquely integrates global language and local image features through contrastive learning, offering a new approach to tackle non-IID data in FL. FedGLCL redefines FL by avoiding separate local training models for each client. Instead, it uses contrastive learning to harmonize local image features with global textual data, enabling uniform feature learning across different local models. The utilization of a pre-trained text encoder in FedGLCL serves a dual purpose: it not only reduces the variance in local feature representations within FL by providing a stable and rich language context but also aids in mitigating overfitting, particularly to majority classes, by leveraging broad linguistic knowledge. Extensive experiments show that FedGLCL significantly outperforms state-of-the-art FL algorithms across different non-IID scenarios. Codes are available at https://github.com/IAMJackYan/FedGLCL. © 2025 13th International Conference on Learning Representations, ICLR 2025. All rights reserved.
Citation
Y. Yan, C.-M. Feng, W. Zuo, S. Khan, L. Zhu, and Y. Liu, “On the Importance of Language-driven Representation Learning for Heterogeneous Federated Learning,” International Conference on Representation Learning, vol. 2025, pp. 63789–63812, May 2025
Source
13th International Conference on Learning Representations, ICLR 2025
Conference
13th International Conference on Learning Representations, ICLR 2025
Keywords
Subjects
Source
13th International Conference on Learning Representations, ICLR 2025
Publisher
International Conference on Learning Representations, ICLR