The vehicle sideslip angle is a core state parameter in vehicle dynamics control. Its accurate estimation is critical for vehicle stability control and the development of active safety systems. In the vehicle sideslip angle estimation method using the traditional Unscented Kalman Filter (UKF), the process noise covariance matrix Q and observation noise covariance matrix R are difficult to adjust adaptively, leading to estimation accuracy degradation under complex driving conditions. This paper proposes a vehicle sideslip angle estimation method that integrates UKF and Deep Reinforcement Learning (DRL), leveraging the adaptive decision-making capability of DRL to dynamically optimize the noise parameters in UKF. A state space incorporating vehicle motion states and filtering performance metrics is constructed, along with an action space that outputs adjustment quantities for the noise covariance matrices. A reward function based on estimation errors and uncertainties is formulated, and the Proximal Policy Optimization (PPO) algorithm is employed to train the policy network. The results indicate that the proposed method effectively improves vehicle sideslip angle estimation accuracy under various driving conditions, including different vehicle speeds, road surface adhesion coefficients, and sensor noise disturbances. Compared with the traditional UKF method, the Root Mean Square Error (RMSE) is reduced by over 30%, and the method demonstrates strong stability and robustness under complex scenarios. This approach provides a new solution for the accurate estimation of key vehicle state parameters and can be extended to fields such as autonomous driving and vehicle active safety.