Federated Reinforcement Learning Approach for Smart Microgrids Control
Gutierrez Guillen, Roberto Alejandro
Gutierrez Guillen, Roberto Alejandro
Supervisor
Department
Machine Learning
Embargo End Date
Type
Thesis
Date
2023
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
This thesis explores the use of advanced control techniques like reinforcement learning to make grids smarter and move towards renewables to meet climate change goals. It aims to develop a Hierarchical Federated Reinforcement Learning framework to control microgrids and enable households to participate actively in the market while reducing their energy bill and CO2 impact. The A2C algorithm was selected to learn a near-optimal strategy for various microgrids and households with a heterogeneous composition of distributed energy resources and battery energy storage systems. The framework handled the stochasticity of distributed energy resources in all microgrid profiles. Agents with competing objectives could work together to aggregate their local models and share their experiences to create a global model that resulted in an improved policy for the A2C. The research methodology involved developing an OpenAi gym environment to test different microgrid control strategies, such as RL agents. The environment includes functions to create a synthetic dataset tailored to the user's needs. An optimizer was used as a baseline with satisfactory results. Nevertheless, the framework learned a better policy than the GEKKO optimizer and simple A2C algorithm. The research benefits include enabling households to participate actively in the market while reducing their CO2 impact and providing a framework for controlling microgrids using reinforcement learning techniques. The key findings and contributions include demonstrating the effectiveness of the hierarchical federated reinforcement learning framework in controlling microgrids and enabling households to participate actively in the market while reducing their CO2 impact.
Citation
R.A.G. Guillen, "Federated Reinforcement Learning Approach for Smart Microgrids Control", M.S. Thesis, Machine Learning, MBZUAI, Abu Dhabi, UAE, 2023.