Robotics have advanced significantly over the years, and human–robot interaction (HRI) is now playing an important role in delivering the best user experience, cutting down on laborious tasks, and raising public acceptance of robots. New HRI approaches are necessary to promote the evolution of robots, with a more natural and flexible interaction manner clearly the most crucial. As a newly emerging approach to HRI, multimodal HRI is a method for individuals to communicate with a robot using various modalities, including voice, image, text, eye movement, and touch, as well as bio-signals like EEG and ECG. It is a broad field closely related to cognitive science, ergonomics, multimedia technology, and virtual reality, with numerous applications springing up each year. However, little research has been done to summarize the current development and future trend of HRI. To this end, this paper systematically reviews the state of the art of multimodal HRI on its applications by summing up the latest research articles relevant to this field. Moreover, the research development in terms of the input signal and the output signal is also covered in this manuscript.

Human activity recognition (HAR) using smartphones provides significant healthcare guidance for telemedicine and long-term treatment. Machine learning and deep learning (DL) techniques are widely utilized for the scientific study of the statistical models of human behaviors. However, the performance of existing HAR platforms is limited by complex physical activity. In this article, we proposed an adaptive recognition and real-time monitoring system for human activities (Ada-HAR), which is expected to identify more human motions in dynamic situations. The Ada-HAR framework introduces an unsupervised online learning algorithm that is independent of the number of class constraints. Furthermore, the adopted hierarchical clustering and classification algorithms label and classify 12 activities (five dynamics, six statics, and a series of transitions) autonomously. Finally, practical experiments have been performed to validate the effectiveness and robustness of the proposed algorithms. Compared with the methods mentioned in the literature, the results show that the DL-based classifier obtains a higher recognition rate ( $\text{95.15\%}$ , waist, and $\text{92.20\%}$ , pocket). The decision-tree-based classifier is the fastest method for modal evolution. Finally, the Ada-HAR system can monitor human activity in real time, regardless of the direction of the smartphone.

For bilateral teleoperation, the haptic feedback demands the availability of accurate force information transmitted from the remote site. Nevertheless, due to the limitation of the size, the force sensor is usually attached outside of the patient's abdominal cavity for the surgical operation. Hence, it measures not only the interaction forces on the surgical tip but also the surgical tool dynamics. In this paper, a model-free based deep convolutional neural network (DCNN) structure is proposed for the tool dynamics identification, which features fast computation and noise robustness. After the tool dynamics identification using DCNN, the calibration is performed, and the bilateral teleoperation is demonstrated to verify the proposed method. The comparison results prove that the proposed DCNN model promises prominent performance than other methods. Low computational time (0.0031 seconds) is ensured by the rectified linear unit (ReLU) function, and the DCNN approach provides superior accuracy for predicting the noised dynamics force and enable its feasibility for bilateral teleoperation.

In the field of robotics, soft robots have been showing great potential in the areas of medical care, education, service, rescue, exploration, detection, and wearable devices due to their inherently high flexibility, good compliance, excellent adaptability, and natural and safe interactivity. Pneumatic soft robots occupy an essential position among soft robots because of their features such as lightweight, high efficiency, non-pollution, and environmental adaptability. Thanks to its mentioned benefits, increasing research interests have been attracted to the development of novel types of pneumatic soft robots in the last decades. This article aims to investigate the solutions to develop and research the pneumatic soft robot. This paper reviews the status and the main progress of the recent research on pneumatic soft robots. Furthermore, a discussion about the challenges and benefits of the recent advancement of the pneumatic soft robot is provided.

In next-generation network architecture, the Cybertwin drove the sixth generation of cellular networks sixth-generation (6G) to play an active role in many applications, such as healthcare and computer vision. Although the previous sixth-generation (5G) network provides the concept of edge cloud and core cloud, the internal communication mechanism has not been explained with a specific application. This article introduces a possible Cybertwin based multimodal network (beyond 5G) for electrocardiogram (ECG) patterns monitoring during daily activity. This network paradigm consists of a cloud-centric network and several Cybertwin communication ends. The Cybertwin nodes combine support locator/identifier identification, data caching, behavior logger, and communications assistant in the edge cloud. The application focuses on monitoring the ECG patterns during daily activity because few studies analyze them under different motions. We present a novel deep convolutional neural network based human activity recognition classifier to enhance identification accuracy. The healthcare monitoring values and potential clinical medicine are provided by the Cybertwin based network for ECG patterns observing.

The presence of unknown physical interaction between the patients body and surgical tool in laparoscopic surgery requires a secure end effector positioning whilst assuring a reliable constraint motion. In this work, a task-space control approach based on fuzzy approximation is proposed for a teleoperated surgery scenario utilizing a serial redundant robot manipulator (7 degrees of freedom), respecting the remote center of motion constraint (RCM). The dynamical uncertainties due to the physical interaction are considered and estimated to maintain the operational accuracy by introducing the decoupled adaptive fuzzy approximation technique. Experiments were demonstrated to verify the effectiveness of the presented approach, and its results confirm that the performances are improved with respect to the accuracy of the surgical operation, providing a safe and effective control method for a teleoperated surgery with RCM constraint.