16 December 2019 in IEEE Access
In model-based methods of reinforcement learning (RL), the probabilistic inference for learning control (PILCO) algorithm, which relies on Gaussian process (GP) for building probabilistic dynamics models, requires only a small amount of data, and can learn from scratch in a few trials by using a probabilistic dynamics model, it explicitly incorporating model uncertainty into long-term planning. However, the time complexity of the GP, which is cubic with respect to the number of trials, makes it challenging to scale the framework to high-dimensional observation spaces. Moreover, the cost function of a task is limited as a locally quadratic function for calculating the policy gradient analytically. To solve these problems, we proposed a probabilistic neural network (PNN) to replace the GP when building the probabilistic dynamics models and develop a deterministic control policy by using long-term predictions. In particular, we determined model uncertainty through basic prior knowledge and calculated cumulative cost by sampling from state distributions. This approach can help reduce the influence of model error and time consumption. Compared with state-of-the-art model-based RL, the proposed approach can reconcile data efficiency and speed of learning even in high-dimensional observation spaces.
1 March 2020 in International Journal of Advanced Robotic Systems
Reinforcement learning has been a promising approach in control and robotics since data-driven learning leads to non-necessity of engineering knowledge. However, it usually requires many interactions with environments to train a controller. This is a practical limitation in some real environments, for example, robots where interactions with environments are restricted and time inefficient. Thus, learning is generally conducted with a simulation environment, and after the learning, migration is performed to apply the learned policy to the real environment, but the differences between the simulation environment and the real environment, for example, friction coefficients at joints, changing loads, may cause undesired results on the migration. To solve this problem, most learning approaches concentrate on retraining, system or parameter identification, as well as adaptive policy training. In this article, we propose an approach where an adaptive policy is learned by extracting more information from the data. An environmental encoder, which indirectly reflects the parameters of an environment, is trained by explicitly incorporating model uncertainties into long-term planning and policy learning. This approach can identify the environment differences when migrating the learned policy to a real environment, thus increase the adaptability of the policy. Moreover, its applicability to autonomous learning in control tasks is also verified.
31 October 2025 in IEEE Transactions on Fuzzy Systems
Recent advancements in artificial intelligence have enhanced the automation of redundant robots, extending their applications from safety cages to unstructured environments with diverse constraints. In such scenarios, these robots typically operate through sequential stages of motion planning and trajectory tracking. However, the decoupling of control objectives between these stages often results in cumulative error. To address this issue, a redundancy adaptive fuzzy controller is proposed. It leverages kinematic redundancy by utilizing redundant DOF to compensate for unknown dynamics through a Fuzzy Logic System (FLS), and achieving fixed-time convergence for task-space pose tracking. To generate constraint satisfying motion during the tracking task, virtual forces are designed in joint space, and are compensated in task space through FLS. Consequently, virtual forces can be integrated with the fuzzy controller to construct Motion Planning Controller (MoPC) that guarantees constraint satisfaction and accurate tracking motion. Additionally, a saturation system is introduced to define a task hierarchy for handling the situation where tracking objective is incompatible with constraints requirements. Lyapunov-based analysis is further conducted to guarantee system stability. The effectiveness of MoPC is verified by precise task-space trajectory tracking and constraint satisfaction of 7-DOF redundant robots in unstructured environments, using both simulations and physical experiments, demonstrating improved convergence rates, tracking accuracy, chattering reduction, and robustness.
9 June 2026 in IEEE Transactions on Automation Science and Engineering
Autonomous dual-arm robotic systems often experience degraded position and force tracking accuracy during cooperative manipulation due to accumulated kinematic and hand-eye calibration errors. This study introduces a vision-based cooperative control approach for tightly coordinated tasks that ensures precise trajectory tracking and force regulation. First, kinematic and dynamic models of the system are formulated to decouple internal and external forces. Furthermore, an asymmetric control scheme is designed in which one arm employs visual feedback to compensate for object-level trajectory tracking errors, while the other is dedicated to internal force regulation; meanwhile, the object-level compliant motion generated by an external force controller is provided as feedforward to the dual arms with closed-chain consistency. Within this framework, the visual feedback adopts a specific design that takes the relative pose residual of auxiliary markers as the feedback signal to resist the influence of multi-source errors. Subsequently, Lyapunov stability theory is employed to validate the convergence of the proposed control methods. Finally, through simulations and physical experiments involving a board transport task with a dual-arm system, the effectiveness of the proposed approach in cooperative position tracking and force control is validated.
4 November 2025 in IEEE Internet of Things Journal
Multitask imitation learning (MTIL) is a pressing issue for robotics applications. However, the inaccuracy and inefficiency of imitation learning frequently limit current approaches. To cope with the problem, this article proposes module generation with an adaptive discriminator-feedback mechanism with diffusion model (MGAD-FMDM), a novel general modular method for robotic MTIL. A transferable adaptive discriminator constitutes the first component whose function is to automatically generate potential primitive modules, which enables robots to improve the imitation accuracy. A universal feedback mechanism for guiding policy updates represents the second development. By employing a diffusion model, the feedback mechanism achieves a two-step process of first recognizing the expert demonstrated distribution and then evaluating the fit between imitated trajectories and expert demonstrations, thereby enhancing learning efficiency. Extensive experiments on practical robot tasks indicate that the proposed MGAD-FMDM achieves superior performance over state-of-the-art methods, including the improvements of 2.22%–34.85% in Spearman’s rank correlation (SRC), 14.29%–100.00% in success rate (SR), 5.43%–64.35% in earth mover’s distance (EMD), and 21.74%–30.10% in convergence speed (CS). The video can be obtained at https://github.com/yueli0827/Robotic-Multi-Task-Imitation-Learning
1 July 2026 in IEEE Robotics and Automation Letters