This paper proposes a multi-agent deep reinforcement learning-based approach for distribution system voltage regulation with a high level of PV penetration. The designed agents can learn the coordinated control strategies using only local information from historical data through the counter-training of local policy networks and centric critic networks. The learned strategies allow us to deal with PV generation uncertainty and perform online coordinated control. Comparative results with other methods show the enhanced control capability of the proposed method under various conditions.

This paper proposes a novel model-free/data-driven centralized training and decentralized execution multi-agent deep reinforcement learning (MADRL) framework for distribution system voltage control with high penetration of PVs. The proposed MADRL can coordinate both the real and reactive power control of PVs with existing static var compensators and battery storage systems. Unlike the existing DRL-based voltage control methods, our proposed method does not rely on a system model during both the training and execution stages. This is achieved by developing a new interaction scheme between the surrogate modeling of the original system and the multi-agent soft actor critic (MASAC) MADRL algorithm. In particular, the sparse pseudo-Gaussian process with a few-shots of measurements is utilized to construct the surrogate model of the original environment, i.e., power flow model. This is a data-driven process and no model parameters are needed. Furthermore, the MASAC enabled MADRL allows to achieve better scalability by dividing the original system into different voltage control regions with the aid of real and reactive power sensitivities to voltage, where each region is treated as an agent. This also serves as the foundation for the centralized training and decentralized execution, thus significantly reducing the communication requirements as only local measurements are required for control. Comparative results with other alternatives on the IEEE 123-nodes and 342-nodes systems demonstrate the superiority of the proposed method.

This paper proposes a heuristic planning energy management controller, based on a Dyna agent of reinforcement learning (RL) approach, for real-time fuel saving optimization of a plug-in hybrid electric vehicle (PHEV). The presented method is referred to as the Dyna-H algorithm, which is a model-free online RL algorithm. First, as a case study, a detailed vehicle powertrain modeling of the Chevrolet Volt is built, where all the control components have been experimentally validated. Four traction operation modes are allowed by managing the states of two clutches and one brake. Furthermore, the Dyna-H algorithm is introduced via incorporating a heuristic planning strategy into a Dyna agent. This is the first time to apply the Dyna-H algorithm in the energy management field of PHEVs. Finally, a comparative analysis of the one-step Q-learning, Dyna, and Dyna-H algorithms is conducted in simulations. Numerous testing results indicate that the proposed algorithm leads to definite improvements in equiv-alent fuel economy and computational speed.