The deadly coronavirus disease (COVID-19) has highlighted the importance of remote health monitoring (RHM). The digital twins (DTs) paradigm enables RHM by creating a virtual replica that receives data from the physical asset, representing its real-world behavior. However, DTs use passive internet of things (IoT) sensors, which limit their potential to a specific location or entity. This problem can be addressed by using the internet of robotic things (IoRT), which combines robotics and IoT, allowing the robotic things (RTs) to navigate in a particular environment and connect to IoT devices in the vicinity. Implementing DTs in IoRT, creates a virtual replica (virtual twin) that receives real-time data from the physical RT (physical twin) to mirror its status. However, DTs require a user interface for real-time interaction and visualization. Virtual reality (VR) can be used as an interface due to its natural ability to visualize and interact with DTs. This research proposes a real-time system for RHM of COVID-19 patients using the DTs-based IoRT and VR-based user interface. It also presents and evaluates robot navigation performance, which is vital for remote monitoring. The virtual twin (VT) operates the physical twin (PT) in the real environment (RE), which collects data from the patient-mounted sensors and transmits it to the control service to visualize in VR for medical examination. The system prevents direct interaction of medical staff with contaminated patients, protecting them from infection and stress. The experimental results verify the monitoring data quality (accuracy, completeness, timeliness) and high accuracy of PT’s navigation.

The digital twins (DTs) paradigm creates a virtual replica of a physical artifact that represents its real-world status in a virtual space. When implemented in the Internet of Robotic Things (IoRT), it develops a virtual model of the physical robot that receives real-time sensor data, acting like an actual robot. The DTs-based IoRT allows remote patient monitoring (RPM) that improves treatment plans and reduces the burden of gathering vital signs on health carers by automating repeated tasks. However, RPM procedures generate large amounts of sensor data, which are difficult to examine. Furthermore, health carers cannot forecast abnormalities based on health data. Machine Learning (ML) can analyze massive amounts of data and perceive patterns to anticipate anomalous health conditions. This research leverages the advantages of ML and DTs-based IoRT to remotely monitor chronic or infectious patients and predict their future status. The proposed system is an extension of our previously published work. It uses the virtual twin (VT) to navigate the physical twin (PT) for collecting data from the patient-mounted sensors and applies ML techniques to predict health anomalies. It evaluated six ML algorithms to determine the most accurate model. The models were trained and tested by using a dataset combining two public datasets and real-world data collected by healthcare providers. Results confirmed that the K Nearest-Neighbor (KNN) has the best accuracy i.e. 97%. The system was also tested in the clinical setting to collect patient data and the best-performing algorithm (KNN) was used for status prediction, obtaining 98% accuracy.

The emergence in computing and the latest hardware technologies realized the use of natural interaction with computers. Gesture-based interaction is one of the prominent fields of natural interactions. The recognition and application of hand gestures in virtual environments (VEs) need extensive calculations due to the complexities involved, which directly affect the performance and realism of interaction. In this paper, we propose a new interaction technique that uses single fingertip-based gestures for interaction with VEs. The objective of the study is to minimize the computational cost, increase performance, and improve usability. The interaction involves navigation, selection, translation, and release of objects. For this purpose, we propose a low-cost camera-based system that uses a colored fingertip for the fastest and accurate recognition of gestures. We also implemented the proposed interaction technique using the Leap Motion controller. We present a comparative analysis of the proposed system with the Leap Motion controller for gesture recognition and operation. A VE was developed for experimental purposes. Moreover, we conducted a comprehensive analysis of two different recognition setups including video camera and the Leap Motion sensor. The key parameters for analysis were task accuracy, interaction volume, update rate, and spatial distortion of accuracy. We used the Standard Usability Scale (SUS) for system usability analysis. The experiments revealed that camera implementation was found with good performance, less spatial distortion of accuracy, and large interaction volume as compared to the Leap Motion sensor. We also found the proposed interaction technique highly usable in terms of user satisfaction, user-friendliness, learning, and consistency.

Conventional patient monitoring methods require skin-to-skin contact, continuous observation, and long working shifts, causing physical and mental stress for medical professionals. Remote patient monitoring (RPM) assists healthcare workers in monitoring patients distantly using various wearable sensors, reducing stress and infection risk. RPM can be enabled by using the Digital Twins (DTs)-based Internet of Robotic Things (IoRT) that merges robotics with the Internet of Things (IoT) and creates a virtual twin (VT) that acquires sensor data from the physical twin (PT) during operation to reflect its behavior. However, manual navigation of PT causes cognitive fatigue for the operator, affecting trust dynamics, satisfaction, and task performance. Also, operating manual systems requires proper training and long-term experience. This research implements autonomous control in the DTs-based IoRT to remotely monitor patients with chronic or contagious diseases. This work extends our previous paper that required the user to manually operate the PT using its VT to collect patient data for medical inspection. The proposed decision-making algorithm enables the PT to autonomously navigate towards the patient’s room, collect and transmit health data, and return to the base station while avoiding various obstacles. Rather than manually navigating, the medical personnel direct the PT to a specific target position using the Menu buttons. The medical staff can monitor the PT and the received sensor information in the pre-built virtual environment (VE). Based on the operator’s preference, manual control of the PT is also achievable. The experimental outcomes and comparative analysis verify the efficiency of the proposed system.