The increasing penetration of distributed generation in active distribution networks (ADNs) introduces significant voltage regulation challenges due to the intermittent nature of renewable energy sources. Electric springs (ESs) have emerged as a cost-effective alternative to conventional FACTS devices for voltage regulation, requiring minimal energy storage while providing fast, flexible reactive power compensation. This paper proposes a deep reinforcement learning (DRL)-based approach for voltage regulation in balanced active distribution networks with distributed generation. Electric springs are deployed at selected buses in series with noncritical loads to provide flexible voltage support. The main contributions of this work are: (1) a novel region-based penalized reward function that effectively guides the DRL agent to minimize voltage deviations; (2) a coordinated control strategy for multiple ESs using the Deep Deterministic Policy Gradient (DDPG) algorithm, representing the first application of DRL to ES-based voltage regulation; (3) a systematic hyperparameter tuning methodology that significantly improves controller performance; and (4) comprehensive validation demonstrating an approximately 40% reduction in mean voltage deviation relative to the no-control baseline. Three well-known continuous-control DRL algorithms, Twin Delayed Deep Deterministic Policy Gradient (TD3), Proximal Policy Optimization (PPO), and DDPG, are first evaluated using the default hyperparameter configurations provided by MATLAB R2022b.Based on this baseline comparison, a dedicated hyperparameter-tuning procedure is then applied to DDPG to improve the robustness and performance of the resulting controller. The proposed approach is evaluated through simulation studies on the IEEE 33-bus and IEEE 69-bus test systems with time-varying load profiles and fluctuating renewable generation scenarios.

This study proposes an optimal control strategy for a transactive energy system integrated with the main supply grid. The power infrastructure of this transactive system includes an electrical microgrid integrated with renewable energy sources (RES), energy storage systems (ESS), and smart loads (SLs). Each SL comprises an electric spring (ES) in series with a noncritical load (NCL). The ES regulates the voltage and manages local demand, making it particularly suitable for microgrids with a high RES penetration. The load variations and stochastic nature of RES are modelled using time-varying profiles to facilitate scenarios that closely resemble real-world conditions. The control strategy uses a metaheuristic algorithm to coordinate the operation of SLs and ESS to minimise voltage fluctuations, reduce power losses, and optimise power exchange within the transactive system. Specifically, the Particle Swarm optimisation (PSO) algorithm is used and compared to the Grey Wolf Optimiser (GWO) and the Genetic algorithm (GA). The results show that PSO achieves significant improvements, reducing operating costs by 77.94%, voltage deviation by 50%, and active power losses by 14.54% compared to scenarios without ES support.