Convergence of the value-iteration-based heuristic dynamic programming (HDP) algorithm is proven in the case of general nonlinear systems. That is, it is shown that HDP converges to the optimal control and the optimal value function that solves the Hamilton-Jacobi-Bellman equation appearing in infinite-horizon discrete-time (DT) nonlinear optimal control. It is assumed that, at each iteration, the value and action update equations can be exactly solved. The following two standard neural networks (NN) are used: a critic NN is used to approximate the value function, whereas an action network is used to approximate the optimal control policy. It is stressed that this approach allows the implementation of HDP without knowing the internal dynamics of the system. The exact solution assumption holds for some classes of nonlinear systems and, specifically, in the specific case of the DT linear quadratic regulator (LQR), where the action is linear and the value quadratic in the states and NNs have zero approximation error. It is stressed that, for the LQR, HDP may be implemented without knowing the system A matrix by using two NNs. This fact is not generally appreciated in the folklore of HDP for the DT LQR, where only one critic NN is generally used.

DC motors are widely used in mechatronic systems; however, their performance degrades significantly in the presence of nonlinear mechanical loads, parameter variations and sensing uncertainties. This paper proposes three control strategies (i.e., PID, optimal, and hybrid controllers) for discrete-time DC motor systems to overcome the disturbances caused by nonlinear mechanical loads and parameter variations. Optimal control of nonlinear discrete-time systems is formally characterized by the Hamilton–Jacobi–Bellman (HJB) equation, whose analytical solution is generally intractable. To address this challenge, a learning-based optimal control strategy based on the Heuristic Dynamic Programming (HDP) framework is developed to approximate the HJB equation, supported by a formal convergence proof. For that purpose, Neural Networks (NNs) are employed to approximate both the cost function and the optimal control policy, enabling near-optimal performance with manageable computational complexity. Although the resulting optimal control achieves fast convergence, it may introduce overshoot and steady-state offset under nonlinear disturbances. To address this limitation, a hybrid control framework is proposed, where nonlinear optimal corrections are integrated with the robustness and adaptability of Proportional–Integral–Derivative (PID) control through error-dependent gating and gain-scheduling mechanisms. A structured evaluation framework is conducted, including nominal analysis, motor-parameter stress testing across nine nonlinear scenarios, controller-design sensitivity analysis, and stochastic measurement-noise assessment under filtered sensing conditions. Results demonstrate that the hybrid controller preserves transient speeds within 5–10% of the optimal controller while effectively eliminating overshoot and steady-state offset under nominal conditions. The hybrid design reduces the accumulated tracking error by more than 95% compared to the optimal controller, while incurring only negligible additional control effort. Under aggressive supply-sag disturbances, the hybrid controller significantly limits peak deviation and reduces accumulated tracking error by over 90%, while maintaining comparable control cost. Overall, the hybrid framework provides a convergence-proven and practically deployable control solution that combines near-optimal convergence speed with robust, overshoot-free performance for intelligent motion-control and robotics applications.

Vehicle suspension systems play a crucial role in ensuring ride comfort, stability, and safety under varying road conditions. While conventional passive suspensions offer limited adaptability, active suspension systems dynamically regulate actuator forces to counteract road-induced disturbances and improve overall performance. This paper introduces and comparatively evaluates two control strategies, sliding mode control and fuzzy logic control, for a full-vehicle active suspension system, aiming to achieve vibration suppression and ride comfort enhancement. Comprehensive simulations were performed in MATLAB/Simulink environment under three severe deterministic road-disturbance profiles with amplitudes ranging from −15 cm to +20 cm. The results demonstrate that both controllers effectively suppress full-vehicle vibrations under challenging road disturbances. Using SMC as the comparative reference, FLC achieved up to a 69.5% reduction in maximum vertical displacement and a 97.5% reduction in peak vertical acceleration, while maintaining a steady-state tracking error below 0.89 cm and comparable control effort. The study’s key contribution lies in demonstrating the superior vibration-suppression and ride-comfort performance of FLC relative to the implemented SMC in a full-vehicle active suspension system.. The large reduction in peak acceleration is mainly associated with the smoother control action of FLC compared with the switching-induced acceleration fluctuations of the implemented SMC. The main contribution of this study is a systematic comparative assessment of decentralized SMC and FLC under identical coupled full-vehicle suspension dynamics and severe asymmetric road disturbance.