Teleoperation provides an innovative approach for completing tasks in environments where humans cannot be physically present, such as hazardous or difficult-to-reach areas. Augmented Reality (AR) enhances this process by overlaying critical information directly into the user’s field of view, streamlining task execution and spatial awareness. This paper presents a comprehensive survey of recent trends in both teleoperation and AR, highlighting their integration to improve remote human-machine interaction. We analyze advancements in both technologies, identify practical applications, and offer insights into their convergence in fields such as aerospace, medical robotics, and industrial automation. By examining academic publications and statistical trends from major databases, we project the future trajectory of AR-teleoperation and discuss potential challenges in implementing these systems, particularly regarding latency, cognitive overload, and security. This survey aims to provide a detailed analysis of the state of the art and offers insights for future research in this rapidly evolving field.
20 November 2023 in IEEE Aerospace and Electronic Systems Magazine
Space assets like communication satellites, space telescopes, and microgravity research stations are essential for human knowledge development. Nevertheless, space assets are subject to failure, damage, or obsolescence in their life cycle. A space asset's failure, damage, or obsolescence is typically addressed using teleoperated space robot systems to diagnose, repair, or upgrade it. In these activities, the region of interest or components to be serviced may be visually occluded by light or a physical obstacle, complicating the teleoperated maneuver and compromising the mission's success and safety as a collision may occur due to blind spots generated by visual occlusions. This paper presents an Augmented Reality (AR)-based robotic framework that allows the robot to perform an on-orbit servicing task despite visually occluded areas. It allows the user to dynamically obtain the best view in a 3D (three-dimensional) model. Besides, for the robot tool to accurately reproduce the natural motion of a human operator's hand, a direct handpresence device is used to directly map the avatar's hand motion to the robot’ send-effector(EE) motion. The system is validate din an AR environment with virtual and physical entities to repair a spacecraft's solar panel in a visually occluded area for the robot. The experimental results demonstrate that because the virtual environment can be manipulated in real-time to show the best perspective to the human operator, the repair trajectory was generated without compromising the safety and operations, even though the robot's EE and the camera-in-hand were not able to observe the area of interest directly.
Partial shading conditions reduce the efficiency of photovoltaic (PV) systems by introducing multiple local maxima in the power–voltage curve, complicating Maximum Power Point Tracking (MPPT). Traditional MPPT methods, such as Perturb and Observe (P&O) and Incremental Conductance (IC), frequently converge to local maxima, leading to suboptimal power extraction. This study proposes a hybrid reinforcement learning-based MPPT approach that combines fuzzy techniques with Deep Deterministic Policy Gradient (DDPG) to enhance tracking accuracy under partial shading. The method integrates fuzzy membership functions into the actor–critic structure, improving state representation and convergence speed. The proposed algorithm is evaluated in a simulated PV environment under various shading scenarios and benchmarked against conventional Perturb and Observe P&O and IC methods. Experimental results demonstrate that the Fuzzy–DDPG approach outperforms these classical techniques by achieving a higher tracking efficiency of 95%, compared to 85% for P&O and 88% for IC in average, while also minimizing steady-state oscillations. Additionally, the proposed method reduces tracking errors by up to 7.9% compared to conventional MPPT algorithms. These findings indicate that the combination of fuzzy logic and deep reinforcement learning provides a more adaptive and efficient MPPT solution, ensuring improved energy harvesting in dynamically changing conditions.
This paper presents the design of a haptic interface called the Haptic Lever, to show four force-related phenomena for haptic augmentation towards Smart Learning Environments. The Haptic Lever is a mechanism with a sensorless torque control developed by means of a Disturbance Observer (DOB) to render forces that allow users to feel virtual phenomena. The control response converges in the torque reference as a result of a gain calibration and the DOB response. The Haptic Lever was evaluated experimentally with four dynamic models for constant (membrane), linear (Hook’s law and viscous damping), and exponential (Coulomb’s law) haptic augmentation. In the first experiment, the user can feel a constant force when passing between two reference points and feel resistance while moving through a virtual membrane. In the second and third experiment, the user can feel and interact with the linear dynamic models of Hook’s law and viscous damping, in the form of a compression or tension spring by pushing or pulling them, respectively. Finally, in the fourth experiment, the user can feel an attraction or repulsion force between two virtual point charges that follows the exponential dynamic model of Coulomb’s law. The results obtained from the experiments showed that the Haptic Lever successfully rendered the equivalent forces to each virtual phenomenon. The haptic sensation is estimated in terms of the torque response under a profile determined by the dynamic models. From the experimental results, it can be observed that the torque in Nm corresponded to each represented phenomenon.
Maximum power point tracking (MPPT) under partial shading is a nonconvex, rapidly varying control problem that challenges multi-agent policies deployed on photovoltaic modules. We present Fuzzy–MAT3D, a fuzzy-augmented multi-agent TD3 (Twin-Delayed Deep Deterministic Policy Gradient) controller trained under centralized training/decentralized execution (CTDE). On the theory side, we prove that differentiable fuzzy partitions of unity endow the actor–critic maps with global Lipschitz regularity, reduce temporal-difference target variance, enlarge the input-to-state stability (ISS) margin, and yield a global -contraction of fixed-policy evaluation (hence, non-expansive with ). We further state a two-time-scale convergence theorem for CTDE-TD3 with fuzzy features; a PL/last-layer-linear corollary implies point convergence and uniqueness of critics. We bound the projected Bellman residual with the correct contraction factor (for and under measure invariance) and quantified the negative bias induced by ; an N-agent extension is provided. Empirically, a balanced common-random-numbers design across seven scenarios and 20 seeds, analyzed by ANOVA and CRN-paired tests, shows that Fuzzy–MAT3D attains the highest mean MPPT efficiency (92.0% ± 4.0%), outperforming MAT3D and Multi-Agent Deep Deterministic Policy Gradient controller (MADDPG). Overall, fuzzy regularization yields higher efficiency, suppresses steady-state oscillations, and stabilizes learning dynamics, supporting the use of structured, physics-compatible features in multi-agent MPPT controllers. At the level of PV plants, such gains under partial shading translate into higher effective capacity factors and smoother renewable generation without additional hardware.