As artificial intelligence and industrial automation are developing, human–robot collaboration (HRC) with advanced interaction capabilities has become an increasingly significant area of research. In this paper, we design and develop a real-time, multi-model HRC system using speech and gestures. A set of 16 dynamic gestures is designed for communication from a human to an industrial robot. A data set of dynamic gestures is designed and constructed, and it will be shared with the community. A convolutional neural network is developed to recognize the dynamic gestures in real time using the motion history image and deep learning methods. An improved open-source speech recognizer is used for real-time speech recognition of the human worker. An integration strategy is proposed to integrate the gesture and speech recognition results, and a software interface is designed for system visualization. A multi-threading architecture is constructed for simultaneously operating multiple tasks, including gesture and speech data collection and recognition, data integration, robot control, and software interface operation. The various methods and algorithms are integrated to develop the HRC system, with a platform constructed to demonstrate the system performance. The experimental results validate the feasibility and effectiveness of the proposed algorithms and the HRC system.

Mechanical devices such as robots are widely adopted for limb rehabilitation. Due to the variety of human body parameters, the rehabilitation motion for different patients usually has its individual pattern; hence, we adopt clustering-based machine learning technique to find a limited number of motion patterns for upper-limb rehabilitation, so that they could represent the large amount of those from people who have various body parameters. By using the regression motion of the clustering result as the target, in this article, we seek to apply kinematic mapping-based motion synthesis framework to design a 1-degree-of-freedom (DOF) mechanism, such that it could lead the patients’ upper limb through the target motion. Also, considering rehab training generally involves a large amount of repetition on a daily basis, this article has developed a rehab system with unity3d based on virtual reality (VR). The proposed device and system could provide an immersive experience to the users, as well as the rehab motion data to the administrative staff for evaluation of users’ status. The construction of the integrated system and the experimental trial of the prototype are presented at the end of this article.

Surface electromyographic (sEMG) signal-driven joint-angle estimation plays a critical role in intelligent rehabilitation systems, as its accuracy directly affects both control performance and rehabilitation efficacy. This study proposes a continuous elbow joint angle estimation method based on time–frequency domain analysis. Raw sEMG signals were processed using the Short-Time Fourier Transform (STFT) to extract time–frequency features. A Scale Temporal–Channel Cross-Encoder (STCCE) network was developed, integrating temporal and channel attention mechanisms to enhance feature representation and establish the mapping from sEMG signals to elbow joint angles. The model was trained and evaluated on a dataset comprising approximately 103,000 samples collected from seven subjects. In the single-subject test set, the proposed STCCE model achieved an average Mean Absolute Error (MAE) of $2.96\pm {0.24}^{\circ }$, Root Mean Square Error (RMSE) of $4.41\pm {0.45}^{\circ }$, Coefficient of Determination ($R^2$) of $0.9924\pm 0.0020$, and Correlation Coefficient (CC) of $0.9963\pm 0.0010$. It achieved a MAE of ${3.30}^{\circ }$, RMSE of ${4.75}^{\circ }$, $R^2$ of 0.9915, and CC of 0.9962 on the multi-subject test set, and an average MAE of $15.53\pm {1.80}^{\circ }$, RMSE of $21.72\pm {2.85}^{\circ }$, $R^2$ of $0.8141\pm 0.0540$, and CC of $0.9100\pm 0.0306$ on the inter-subject test set. These results demonstrated that the STCCE model enabled accurate joint-angle estimation in the time–frequency domain, contributing to a better motion intent perception for upper-limb rehabilitation.

Probability weighting is one of the key components of the modern risky decision-making theories, an effective probability weight function can more accurately describe the decision-makers' subjective response to the event probability. While the probability weighting functions (PWFs) with several different parametric forms and parameter-free elicitation methods have been proposed. This paper first introduces a Lagrange interpolation method (LIM) for building a parameter-free PWF model, then proposes a novel PWF model with the use of the LIM based on Prelec's PWF model. Furthermore, an experiment was designed and carried out. The results not only demonstrate that the novel PWF model could reflect the empirical regularities for maximizing the satisfaction degree of the curve fitting for the preference points obtained from experiment or questionnaire survey and better predict the preferences of decision-makers, but also are found to be consistent with the properties of PWF. This paper makes a significant methodological contribution to developing a numerical method, such as LIM, for constructing the probability weighting model. The finial error analysis suggests that the novel PWF model is a more effective approach.