SCP-FCA: Superclass Partitioning Federated Clustering Algorithm with Distribution-specific Model Clustering
Alsuwaidi, Ghanim Ahmed Mohammed Butheina
Alsuwaidi, Ghanim Ahmed Mohammed Butheina
Supervisor
Department
Machine Learning
Embargo End Date
Type
Thesis
Date
2024
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
As the research scene of federated learning (FL) progresses, real world trials increasingly highlight the necessity of algorithms that address the issue of data heterogeneity between clients. Classical FL solutions such as FedAvg produce a global model that cannot be ideal for the participant clients in the real world as most practical applications lie in the domain of non-IID optimization. To this end, methods that utilize clustering (CFL) have been developed to group clients in a cluster in hopes of mitigating the drawbacks of the non-IID setting by increasing homogeneity on a per-cluster basis. However, such constructions do not eliminate the heterogeneity, nor do they serve the best interest of all connected clients. One key issue is that the convergence requirements of these algorithms are strict, and must have fine-tuned hyperparameters to ensure correct clustering, without which the method may only achieve the performance that approximates the classical methods at best, and completely collapse at worst. To this end, this paper proposes a novel approach to CFL that leverages the non-IID setting to facilitate a CFL network that achieves the best model on a per-distribution basis, which provides further reduction to heterogeneity compared to standard CFL solutions that mitigate per-client heterogeneity. The proposed method introduces the concept of a consensus map that is agreed upon by all clients (having accepted participation in the CFL network) that is used to partition the local dataset into subsets of individual distributions. This construction allows the client to define a model specific to each subset (more specifically, one for each subset) which is a better approximation to the IID performance of the model on the specific subset as compared to the models that typical CFL solutions produce for the entirety of a client s dataset. Furthermore, the paper provides a custom
Citation
G. A. Alsuwaidi, "SCP-FCA: Superclass Partitioning Federated Clustering Algorithm with Distribution-specific Model Clustering", MS. Thesis, Machine Learning, MBZUAI, Abu Dhabi, UAE, 2024