Vision loss severely impacts object recognition and spatial cognition for limited vision individuals. It is a challenge to compensate for this using other sensory modalities, such as touch or hearing. This paper introduces StereoPilot, a wearable target location system to facilitate the spatial cognition of BVI. Through wearing a head-mounted RGB-D camera, the 3D spatial information of the environment is measured and processed into navigation cues. Leveraging spatial audio rendering (SAR) technology, it allows the navigation cues to be transmitted in a type of 3D sound from which the sound orientation can be distinguished by the sound localization instincts in humans. Three haptic and auditory display strategies were compared with SAR through experiments with three BVI and four sighted subjects. Compared with mainstream speech instructional feedback, the experimental results of the Fitts’ law test showed that SAR increases the information transfer rate (ITR) by a factor of three for spatial navigation, while the positioning error is reduced by 40%. Furthermore, SAR has a lower learning effect than other sonification approaches such as vOICe. In desktop manipulation experiments, StereoPilot was able to obtain precise localization of desktop objects while reducing the completion time of target grasping tasks in half as compared to the voice instruction method. In summary, StereoPilot provides an innovative wearable target location solution that swiftly and intuitively transmits environmental information to BVI individuals in the real world.

Surface electromyography (sEMG) is commonly used to observe the motor neuronal activity within muscle fibers. However, decoding dexterous body movements from sEMG signals is still quite challenging. In this paper, we present a high-density sEMG (HD-sEMG) signal database that comprises simultaneously recorded sEMG signals of intrinsic and extrinsic hand muscles. Specifically, twenty able-bodied participants performed 12 finger movements under two paces and three arm postures. HD-sEMG signals were recorded with a 64-channel high-density grid placed on the back of hand and an 8-channel armband around the forearm. Also, a data-glove was used to record the finger joint angles. Synchronisation and reproducibility of the data collection from the HD-sEMG and glove sensors were ensured. The collected data samples were further employed for automated recognition of dexterous finger movements. The introduced dataset offers a new perspective to study the synergy between the intrinsic and extrinsic hand muscles during dynamic finger movements. As this dataset was collected from multiple participants, it also provides a resource for exploring generalized models for finger movement decoding.

Bionic prosthetic hands hold the potential to replicate the functionality of human hands. The use of bionic limbs can assist amputees in performing everyday activities. This article systematically reviews the research progress on bionic prostheses, with a focus on control mechanisms, sensory feedback integration, and mechanical design innovations. It emphasizes the use of bioelectrical signals, such as electromyography (EMG), for prosthetic control and discusses the application of machine learning algorithms to enhance the accuracy of gesture recognition. Additionally, the paper explores advancements in sensory feedback technologies, including tactile, visual, and auditory modalities, which enhance user interaction by providing essential environmental feedback. The mechanical design of prosthetic hands is also examined, with particular attention to achieving a balance between dexterity, weight, and durability. Our contribution consists of compiling current research trends and identifying key areas for future development, including the enhancement of control system integration and improving the aesthetic and functional resemblance of prostheses to natural limbs. This work aims to inform and inspire ongoing research that seeks to refine the utility and accessibility of prosthetic hands for amputees, emphasizing user-centric innovations.

The traditional way of reading through Braille books is constraining the reading experience of blind or visually impaired (BVI) in the digital age. In order to improve the reading convenience of BVI, this paper proposes a low-cost and refreshable Braille display device, and solves the problems of high energy consumption and low latching force existing in existing devices. Further, the Braille display device was combined with the 3D Systems Touch device to develop an active Braille touch-reading system for digital reading of BVI with the help of the CHAI3D virtual environment. Firstly, according to the actual needs of BVI to touch and read the Braille dots, this paper utilizes the beam structure to provide a full latching function for the raised Braille dot without energy consumption. Through theoretical derivation and finite element analysis, the performance of the Braille dot actuator is optimized to provide sufficient feedback force and latching force for finger’s touch-reading. Then, this paper designs a virtual Braille interactive environment based on the CHAI3D, and combines the sense of touch with audio to effectively improve the recognition accuracy and reading efficiency of BVI for Braille through the multi-modal presentation of Braille information. The performance test results of the device show that the average lifting force of the Braille dot actuator is 101.67 mN, the latching force is over 5 N, and the average refresh frequency is 17.1 Hz, which meets the touch-reading needs of BVI. User experiments show that the average accuracy rate of BVI subjects in identifying digitized Braille is 95.5%, and subjects have a high subjective evaluation of the system.

With the advantages of comfortable wearing and outdoor usage, the myoelectric gesture recognition techniques have gained much attention in the field of human-machine interaction (HMI). The purpose of this study is to optimize model structure and transfer generalized features to improve the robustness of myoelectric hand motion decoding. We derived the hand motion recognition framework from the muscle synergy theory, which is formulated as a temporal convolutional (TC) model of array sEMG signals, then a hierarchical myoelectric decoding model was proposed to predict simultaneous and continuous hand motion. The model was trained by the methods of unsupervised low-level feature learning and automated data labeling to minimize training supervision. Extensive experiments on the public sEMG database (17 subjects in Biopatrec) show that the TC model can extract muscle synergy features with higher fidelity ( R ² = 0.85±0.23) than the traditional instantaneous mixture model, the results of online test demonstrate robust myoelectric decoding on multiple simultaneous and continuous hand motions. More importantly, the analysis of weights visualization shows that the low-level feature representation layer of TC model can be migrated across the individuals, which provides a transferrable feature extraction layer for generalized hand motion decoding.

This letter proposes a novel human-robot co-adaptation framework for robust and accurate user intent recognition, specifically in the context of automatic control in assistance robots such as neural prosthetics and rehabilitation devices empowered by electrophysiological signals. Our goal is to incorporate user adaptability early in the training phase to facilitate both machine recognition and user adaptability, rather than relying solely on brute-force machine learning methods. The proposed framework is featured by applying biofeedback-based user adaptive behavior into model training, while the machine can adapt to those changes through online learning. Specifically, this study focuses on the recognition of two-degree-of-freedom simultaneous and continuous wrist movement intentions based on surface electromyogram (sEMG) array signals, and the performance is tested on twelve able-bodied subjects. The co-adaptive evaluation experiment demonstrates the robust control of this method by introducing sEMG electrode displacement as perturbations. Experimental results show that this method improves the completion time of centre-out tasks by 13% compared to conventional methods (Cohen's d=0.637), and debias 86% of the effect of electrode shift perturbations. This study provides insights into the potential for incorporating human adaptability into machine intelligence to improve user intent recognition and automatic robot control.