Teleoperation has widely contributed to many applications. Consequently, the design of intuitive and ergonomic control interfaces for teleoperation has become crucial. The rapid advancement of Mixed Reality (MR) has yielded tangible benefits in human-robot interaction. MR provides an immersive environment for interacting with robots, effectively reducing the mental and physical workload of operators during teleoperation. Additionally, the incorporation of haptic rendering, including kinaesthetic and tactile rendering, could further amplify the intuitiveness and efficiency of MR-based immersive teleoperation. In this study, we developed an immersive, bilateral teleoperation system, integrating Digital Twin-driven Mixed Reality (DTMR) manipulation with haptic rendering. This system comprises a commercial remote controller with a kinaesthetic rendering feature and a wearable cost-effective tactile rendering interface, called the Soft Pneumatic Tactile Array (SPTA). We carried out two user studies to assess the system's effectiveness, including a performance evaluation of key components within DTMR and a quantitative assessment of the newly developed SPTA. The results demonstrate an enhancement in both the human-robot interaction experience and teleoperation performance. For more project details, please view our website: https://sites.google.com/view/hbts-brl/home

Magnetic microrobots, with their small size and agile maneuverability, are well-suited for navigating the intricate and confined spaces within the human body. In vivo cargo delivery within the context of microrobotics involves the use of microrobots to transport and administer drugs and cells directly to the targeted regions within a living organism. The principal aim is to enhance the precision, efficiency, and safety of therapeutic interventions. Despite their potential, there is a shortage of comprehensive reviews on the use of magnetic microrobots for in vivo cargo delivery from both research and engineering perspectives, particularly those published after 2019. This review addresses this gap by disentangling recent advancements in magnetic microrobots for in vivo cargo delivery. It summarizes their actuation platforms, structural designs, cargo loading and release methods, tracking methods, navigation algorithms, and degradation and retrieval methods. Finally, it highlights potential research directions. This review aims to provide a comprehensive summary of the current landscape of magnetic microrobot technologies for in vivo cargo delivery. It highlights their present implementation methods, capabilities, and prospective research directions. The review also examines significant innovations and inherent challenges in biomedical applications.

In this paper, we present the design and benchmark of an innovative sensor, ViTacTip, which fulfills the demand for advanced multi-modal sensing in a compact design. A notable feature of ViTacTip is its transparent skin, which incorporates a ‘see-through-skin’ mechanism. This mechanism aims at capturing detailed object features upon contact, significantly improving both vision-based and proximity perception capabilities. In parallel, the biomimetic tips embedded in the sensor's skin are designed to amplify contact details, thus substantially augmenting tactile and derived force perception abilities. To demonstrate the multi-modal capabilities of ViTacTip, we developed a multi-task learning model that enables simultaneous recognition of hardness, material, and textures. To assess the functionality and validate the versatility of ViTacTip, we conducted extensive benchmarking experiments, including object recognition, contact point detection, pose regression, and grating identification. To facilitate seamless switching between various sensing modalities, we employed a Generative Adversarial Network (GAN)-based approach. This method enhances the applicability of the ViTacTip sensor across diverse environments by enabling cross-modality interpretation.

Although the Internet of Robotic Things (IoRT) has enhanced the productivity of robotic systems in conjunction with the Internet of Things (IoT), it does not inherently support seamless human–robot collaboration. This article presents HuBotVerse, a unified framework designed to foster the evolution of the Internet of Human and Intelligent Robotic Things (IoHIRT). HuBotVerse is advantageous due to its unique features, including security, user-friendliness, manageability, and its open source nature. Moreover, this framework can seamlessly integrate various human–robot interaction (HRI) interfaces to facilitate collaborative control between humans and robots. Here, we emphasize a digital twin (DT)-based mixed reality (MR) interface, which enhances teleoperation efficiency by offering users an intuitive and immersive way to interact with. To evaluate the effectiveness of HuBotVerse, we conducted user studies based on a pick-and-place task. Feedback was gathered through questionnaires, complemented by a quantitative analysis of key performance metrics, user experience, and the National Aeronautics and Space Administration Task Load Index (NASA-TLX). Results indicate that the fusion of MR and HuBotVerse within a comprehensive framework significantly improves the efficiency and user experience of teleoperation. Moreover, the follow-up questionnaires reflect the advantages of the HuBotVerse framework in terms of evident user-friendliness, manageability, and usability in home-care or health-care applications. For codes, project videos, tutorials, technical details, case studies, and Q&A, please check our website ( https://sites.google.com/view/iohirtplusmr/home ).