The integration of photovoltaic (PV) power generation systems has significantly increased the complexity of voltage distribution in power grids, making it challenging for conventional Load Ratio Control Transformers (LRTs) to manage voltage fluctuations caused by weather-dependent PV output variations. Power Conditioning Systems (PCSs) interconnected with PV installations are increasingly considered for voltage control to address these challenges. This study proposes a Machine Learning (ML)-based control method for sub-transmission grids, integrating long-term LRT tap-changing with short-term reactive power control of PCSs. The approach estimates the voltage at each grid node using a Deep Neural Network (DNN) that processes measurable substation data. Based on these estimated voltages, the method determines optimal LRT tap positions and PCS reactive power outputs using Deep Reinforcement Learning (DRL). This enables real-time voltage monitoring and control using only substation measurements, even in grids without extensive sensor installations, ensuring all node voltages remain within specified limits. To improve the model’s transparency, Shapley Additive Explanation (SHAP), an Explainable AI (XAI) technique, is applied to the DRL model. SHAP enhances interpretability and confirms the effectiveness of the proposed method. Numerical simulations further validate its performance, demonstrating its potential for effective voltage management in modern power grids.