Robust Deep Reinforcement Learning for Volt-VAR Optimization in Active Distribution System under Uncertainty

Abstract

The deep reinforcement learning (DRL) based Volt-VAR optimization (VVO) methods have been widely studied for active distribution networks (ADNs). However, most of them lack safety guarantees in terms of power injection uncertainties due to the increase in distributed energy resources (DERs) and load demand, such as electric vehicles. This article proposes a robust deep reinforcement learning (RDRL) framework for VVO via a robust deep deterministic policy gradient (DDPG) algorithm. This algorithm can effectively manage hybrid action spaces, considering control devices like capacitors, voltage regulators, and smart inverters. Additionally, it is designed to handle uncertainties by quantifying uncertainty sets with conformal prediction and modeling uncertainties as adversarial attacks to guarantee safe exploration across action spaces. Numerical results on three IEEE test cases demonstrate the sample efficiency and safety of the proposed robust DDPG against uncertainties compared to the benchmark algorithms.