Virtual reality (VR) is regarded as a useful tool for teleoperation systems and provides operators with immersive visual feedback on the robot and the environment. However, without any haptic feedback or physical constructions, VR-based teleoperation systems normally suffer from poor maneuverability, and operational faults may be caused in some fine movements. In this article, we employ mixed reality (MR), which combines real and virtual worlds, to develop a novel teleoperation system. A new system design and control algorithms are proposed. For the system design, an MR interface is developed based on a virtual environment augmented with real-time data from the task space with the goal of enhancing the operator’s visual perception. To allow the operator to be freely decoupled from the control loop and offload the operator’s burden, a new interaction proxy is proposed to control the robot. For the control algorithms, two control modes are introduced to improve the long-distance movements and fine movements of the MR-based teleoperation system. In addition, a set of fuzzy-logic-based methods are proposed to regulate the orientation, position, velocity, and force of the robot to enhance the system’s maneuverability and address potential operational faults. A barrier Lyapunov function and a backstepping method are leveraged to design the control laws and simultaneously guarantee the system’s stability under state constraints. Experiments conducted using a six-degree-of-freedom robotic arm prove the feasibility of the system.

In this paper, a novel neural network (NN)-based four-channel wave-based time domain passivity approach (TDPA) is proposed for a teleoperation system with time-varying delays. The designed wave-based TDPA aims to robustly guarantee the channels passivity and provide higher transparency than the previous power-based TDPA. The applied NN is used to estimate and eliminate the system's dynamic uncertainties. The system stability with linearity assumption on human and environment has been analyzed using Lyapunov method. The proposed algorithm is validated through experimental work based on a 3-DOF bilateral teleoperation platform in the presence of different time delays.

This paper develops data-driven Type-2 Takagi-Sugeno (T-S) fuzzy modeling and control for bilateral teleoperation with dynamic uncertainties and time-varying delays. The Type-2 T-S fuzzy model identified based on input-output data samples describes the nonlinear teleoperation system by a weighted sum of a group of linear local models, which offers a platform to design robust control algorithms by means of mature linear theories. The fuzzy-model-based four-channel control laws are proposed to guarantee the motion synchronization and enhance the operator's force perception for the environment when the time-varying delays and large dynamic uncertainties, especially the gravity of a heavy end effector of the slave, exist. Markov processes are applied to model the time delays. The stability of the closed-loop system is proved by using the Lyapunov-Krasovskii functions. All the conditions are expressed as linear-matrix inequalities (LMI). By using the MATLAB LMI toolbox, the optimized control gains for each of the fuzzy rules are derived to achieve the optimal performance. Finally, experiments based on an experimental platform consisting of two haptic devices prove the superiority of the proposed strategy through comparison with previous work.

This article proposes a new single master bimanual teleoperation system with an efficient position, orientation, and force regulation strategy. Unlike many existing studies that solely support motion synchronization, the first contribution of the proposed work is to propose a solution for orientation regulation when several slave robots have differing motions. In other words, we propose a solution for self-regulated orientation for dual-arm robots. A second contribution in the article allows the master with fewer degrees of freedom (DoF) to control the slaves (with higher DoF), while the orientation of the slaves is self-regulated. The system further offers a novel force regulation that enables the slave robots to have a smooth and balanced robot-environment interaction with proper force directions. Finally, the proposed approach provides adequate force feedback about the environment to the operator and assists the operator in identifying different motion situations of the slaves. Our approach demonstrates that the forces from the slaves will not interrupt the operator’s perception of the environment. To validate the proposed system, experiments are conducted using a platform consisting of two 7-DoF slave robots and one 3-DoF master haptic device. The experiments demonstrated good results in terms of position, orientation, and force regulation.

This study presents a new point set registration method to align 3D range scans. Fuzzy clusters are utilized to represent a scan, and the registration of two given scans is realized by minimizing a fuzzy weighted sum of the distances between their fuzzy cluster centers. This metric has a broad basin of convergence and is robust to noise. Moreover, it provides analytic gradients, allowing standard gradient-based algorithms to be applied for optimization. For the first time in rigid point set registration, a registration quality assessment in the absence of ground truth is provided. Given specified rotation and translation spaces, we derive the upper and lower bounds of the fuzzy cluster-based metric and develop a branch-and-bound (BnB)-based optimization scheme, which can globally minimize the metric regardless of the initialization. This optimization scheme is performed in an efficient coarse-to-fine fashion: First, fuzzy clustering is applied to describe each of the two given scans by a small number of fuzzy clusters. Then, a global search, which integrates BnB and gradient-based algorithms, is implemented to achieve a coarse alignment for the two scans. During the global search, the registration quality assessment offers a beneficial stop criterion to detect whether a good result is obtained.