Enhanced Modeling and Control of Hybrid Power Systems: A Deep Reinforcement Learning Approach for Optimal Decision-Making

Abstract

The management of present-day power systems has become increasingly complex due to the integration of distributed and renewable energy resources. This added complexity makes the power system optimization problem challenging to solve with conventional methods because of its increased dimensionality. Reinforcement learning offers a model-free approach to optimal decision-making. However, popular algorithms such as Deep Q-Networks (DQN) struggle to produce feasible solutions due to their inability to strictly enforce operational constraints. To address this issue, Hou Shengren et al. (International Journal of Electrical Power & Energy Systems 152, 2023) proposed a variation of the DQN algorithm called “MIP-DQN”. Building on the work of Matteo Fischetti and Jason Jo (Constraints 23.3: 296-309, 2018), this algorithm formulates the trained critic neural network as a mixed-integer programming problem, enabling the algorithm to identify the action that maximises the action-value function while satisfying operational constraints. With this thesis, we aim to provide an accessible study of the use of reinforcement learning, particularly the MIP-DQN algorithm, for the optimal management of hybrid power systems from both theoretical and applied perspectives. Theoretically, we present the necessary mathematical foundations to understand reinforcement learning in the context of power system optimization. This includes a detailed mathematical formulation of the power system optimization problem and an explanation of the novel MIP-DQN algorithm, touching on various mathematical topics such as optimization, Markov Decision Processes, optimal control, and reinforcement learning. Our main contribution is the implementation of the MIP-DQN algorithm and the analysis of its performance in optimising the management of a specified power system.