The Heterogeneity and Privacy in Cross-Silo Federated Learning
Hou, Xiangjian
Hou, Xiangjian
Author
Supervisor
Department
Computer Vision
Embargo End Date
2024-01-01
Type
Thesis
Date
2024
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
In addressing the twin challenges of data heterogeneity and privacy in Federated Learning (FL), this thesis introduces a dual-faceted approach that not only pioneers advancements in FL methodologies but also contributes significantly to the discourse on privacy preserving techniques within the domain. Our research is divided into two primary foci: improving FL performance amid data heterogeneity through a novel Federated Learning with Partially Personalized (FedPP) method and optimizing privacy-preserving mechanisms to balance efficacy and confidentiality in FL implementations. The first segment of our investigation unveils the FedPP method, an innovative approach designed to mitigate the limitations imposed by the heterogeneity of the data in FL. By integrating the Learning Rate Adjustment (LoRA) technique, FedPP not only surpasses the performance benchmarks of centralized training models, but also sets a new standard for personalized federated learning. This methodological advancement underscores the potential of partial personalization in bridging the performance gap often witnessed in conventional FL settings, thus enhancing model accuracy and efficiency across diverse data distributions. At the same time, our exploration extends into the realm of differential privacy (DP), where we posit the existence of an optimal interplay between the number of local updates and communication rounds, a crucial balance to maximize convergence performance within a predefined privacy budget. Through rigorous theoretical analysis, we delineate the optimal configurations for local steps and communication rounds that significantly improve the convergence bounds of the DP-enhanced ScaffNew algorithm, particularly in the landscape of strongly convex optimization problems. Our theoretical propositions are further corroborated by empirical evidence, establishing a direct correlation between these optimal configurations and various critical parameters, including the DP privacy budget.
Citation
X. Hou, "The Heterogeneity and Privacy in Cross-Silo Federated Learning", M.S. Thesis, Computer Vision, MBZUAI, Abu Dhabi, UAE, 2024