This paper proposes a deep reinforcement learning (DRL) method that combines random network distillation (RND) and long short-term memory (LSTM) to address the tracking control problem, while leveraging the inherent symmetry in robotic arm movements to eliminate the need for learning or knowing the system’s dynamic model. In general, the complexity and strong coupling of robotic manipulators make trajectory tracking extremely challenging. Firstly, the prediction network and fixed network are jointly trained using the RND method. The difference in output values between the two networks acts as an internal reward for the robotic manipulator environment. This internal reward mechanism encourages the robotic arm agent to actively explore unpredictable and unknown environmental states, thereby consequently boosting the performance and efficiency of the tracking control for the robotic manipulator. Then, the Soft Actor-Critic (SAC) algorithm, the LSTM network, and the attention mechanism are integrated to resolve the instability problem during training and acquire a stable policy. The LSTM model effectively captures the symmetry and temporal changes in joint angles, while the attention mechanism dynamically prioritizes important features, thereby reducing the instability of the robotic manipulator during tracking tasks and enhancing feature extraction efficiency. The simulation outcomes demonstrate that the proposed method effectively performs the robot tracking task, confirming the efficacy and efficiency of the DRL algorithm.

In this paper, a deep reinforcement learning (DRL) approach based on generative adversarial imitation learning (GAIL) and long short-term memory (LSTM) is proposed to resolve tracking control problems for robotic manipulators with saturation constraints and random disturbances, without learning the dynamic and kinematic model of the manipulator. Specifically, it limits the torque and joint angle to a certain range. Firstly, in order to cope with the instability problem during training and obtain a stability policy, soft actor–critic (SAC) and LSTM are combined. The changing trends of joint position over time are more comprehensively captured and understood by employing an LSTM architecture designed for robotic manipulator systems, thereby reducing instability during the training of robotic manipulators for tracking control tasks. Secondly, the obtained policy by SAC-LSTM is used as expert data for GAIL to learn a better control policy. This SAC-LSTM-GAIL (SL-GAIL) algorithm does not need to spend time exploring unknown environments and directly learns the control strategy from stable expert data. Finally, it is demonstrated by the simulation results that the end effector of the robot tracking task is effectively accomplished by the proposed SL-GAIL algorithm, and more superior stability is exhibited in a test environment with interference compared with other algorithms.

Unmanned aerial vehicles (UAVs) often face significant challenges in trajectory tracking within complex dynamic environments, where uncertainties, external disturbances, and nonlinear dynamics hinder accurate and stable control. To address this issue, a bio-inspired deep reinforcement learning (DRL) algorithm is proposed, integrating behavior cloning (BC) and long short-term memory (LSTM) networks. This method can achieve autonomous learning of high-precision control policy without establishing an accurate system dynamics model. Motivated by the memory and prediction functions of biological neural systems, an LSTM module is embedded into the policy network of the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. This structure captures temporal state patterns more effectively, enhancing adaptability to trajectory variations and resilience to delays or disturbances. Compared to memoryless networks, the LSTM-based design better replicates biological time-series processing, improving tracking stability and accuracy. In addition, behavior cloning is employed to pre-train the DRL policy using expert demonstrations, mimicking the way animals learn from observation. This biomimetic plausible initialization accelerates convergence by reducing inefficient early-stage exploration. By combining offline imitation with online learning, the TD3-LSTM-BC framework balances expert guidance and adaptive optimization, analogous to innate and experience-based learning in nature. Simulation experimental results confirm the superior robustness and tracking accuracy of the proposed method, demonstrating its potential as a control solution for autonomous UAVs.