Musculoskeletal ultrasound (US) imaging faces challenges such as operator experience, limited spatial flexibility, and high personnel costs. This study introduces an Automated Robotic Ultrasound Scanning (ARUS) system that integrates key technological advancements to automate the ultrasound scanning procedure with the robot, including anatomical target localization, automatic trajectory generation, deep-learning-based segmentation, and 3D reconstruction of musculoskeletal structures. The ARUS system consists of a robotic arm, ultrasound imaging, and stereo vision for precise anatomical area detection. A Graphical User Interface (GUI) facilitates a flexible selection of scanning trajectories, improving user interaction and enabling customized US scans. To handle complex and dynamic curvatures on the skin, together with anatomical area detection, the system employs a hybrid position–force control strategy based on the generated trajectory, ensuring stability and accuracy. Additionally, the utilized RA-UNet model offers multi-label segmentation on the bone and muscle tissues simultaneously, which incorporates residual blocks and attention mechanisms to enhance segmentation accuracy and robustness. A custom musculoskeletal phantom was used for validation. Compared to the reference 3D reconstruction result derived from the MRI scan, ARUS achieved a 3D reconstruction root mean square error (RMSE) of 1.22 mm, with a mean error of 0.94 mm and a standard deviation of 0.77 mm. The ARUS system extends 3D musculoskeletal imaging capacity by enabling both bones and muscles to be segmented and reconstructed into 3D shapes in real time and simultaneously. These features suggest significant potential as a cost-effective and reliable option for musculoskeletal examination and diagnosis in real-time applications.

This paper introduces a method for measuring 3D tibiofemoral kinematics using a multi-channel A-mode ultrasound system under dynamic conditions. The proposed system consists of a multi-channel A-mode ultrasound system integrated with a conventional motion capture system (i.e., optical tracking system). This approach allows for the non-invasive and non-radiative quantification of the tibiofemoral joint’s six degrees of freedom (DOF). We demonstrated the feasibility and accuracy of this method in the cadaveric experiment. The knee joint’s motions were mimicked by manually manipulating the leg through multiple motion cycles from flexion to extension. To measure it, six custom ultrasound holders, equipped with a total of 30 A-mode ultrasound transducers and 18 optical markers, were mounted on various anatomical regions of the lower extremity of the specimen. During experiments, 3D-tracked intra-cortical bone pins were inserted into the femur and tibia to measure the ground truth of tibiofemoral kinematics. The results were compared with the tibiofemoral kinematics derived from the proposed ultrasound system. The results showed an average rotational error of 1.51 ± 1.13° and a translational error of 3.14 ± 1.72 mm for the ultrasound-derived kinematics, compared to the ground truth. In conclusion, this multi-channel A-mode ultrasound system demonstrated a great potential of effectively measuring tibiofemoral kinematics during dynamic motions. Its improved accuracy, nature of non-invasiveness, and lack of radiation exposure make this method a promising alternative to incorporate into gait analysis and prosthetic kinematic measurements later.

In Total Knee Replacement Arthroplasty (TKA), surgical robotics can provide image-guided navigation to fit implants with high precision. Its tracking approach highly relies on inserting bone pins into the bones tracked by the optical tracking system. This is normally done by invasive, radiative manners (implantable markers and CT scans), which introduce unnecessary trauma and prolong the preparation time for patients. To tackle this issue, ultrasound-based bone tracking could offer an alternative. In this study, we proposed a novel deep-learning structure to improve the accuracy of bone tracking by an A-mode ultrasound (US). We first obtained a set of ultrasound dataset from the cadaver experiment, where the ground truth locations of bones were calculated using bone pins. These data were used to train the proposed CasAtt-UNet to predict bone location automatically and robustly. The ground truth bone locations and those locations of US were recorded simultaneously. Therefore, we could label bone peaks in the raw US signals. As a result, our method achieved sub-millimeter precision across all eight bone areas with the only exception of one channel in the ankle. This method enables the robust measurement of lower extremity bone positions from 1D raw ultrasound signals. It shows great potential to apply A-mode ultrasound in orthopedic surgery from safe, convenient, and efficient perspectives.

A-mode ultrasound has not been widely used in medical applications compared to B-mode ultrasound. The primary reason is that the data representation, being 1-dimensional (1D), is less intuitive for users and harder to interpret. However, A-mode ultrasound has several advantageous features, such as faster data acquisition for real-time sensing, direct distance measurement from raw RF data, and a smaller size. Traditionally, A-mode ultrasound has been used to measure biometric distance. However, current distance measurement algorithms are crude, mostly relying on conventional signal processing for peak detection. When the tracking task is under dynamic condition, it becomes challenging to maintain high accuracy and robustness. In this study, we introduced a deep-learning framework to enhance A-mode ultrasound’s bone tracking reliability and accuracy under dynamic conditions. The proposed SIRC-UNet was designed to enhance the perceptual resolution of 1D signal, allowing for accurate analysis of A-mode’s RF data and leading to more precise peak detection. The method was evaluated by analyzing bias between peak locations from the prediction and the ground truth, and analyzing the capability of distinguishing bone peaks from other irrelevant peaks. The results demonstrated that our method could perform real-time high-precision (sub-millimeter accuracy) bone measurements in one cadaver experiment. It showcased the potential to provide accurate dynamic bone tracking and bone position detection, with the possibilities to extend applications to surgical robots and rehabilitation exoskeletons, where real-time bone tracking is crucial.

Tracking forearm movement via measured physiological signals is crucial for understanding human motor control mechanism. Current methods mainly use muscle-derived signals to predict arm movements while often overlooking the potential role of gaze attention, which is important for hand-eye coordination and instant and continuous motion planning and execution. In this study, we explored the impact of gaze on motion tracking. A hierarchical transformer-based structure was developed to integrate gaze into muscle activity signals for recovering the joint trajectory. To collect the dataset, six subjects were recruited to perform arm motions broadly involved in daily activities; the measured signals from the muscle activity and gaze attention were used to train and evaluate the proposed method. A performance comparison was conducted between the models using solely muscle activity signals and both muscle and gaze information. The experimental results showed the important role of gaze information involved in motion prediction and the motor control mechanism. This research also gained insights on how to integrate gaze information into the muscle signals, which offers an alternative to bringing artificial intelligence to be engaged in the framework of motion tracking. Consequently, it is important for future designs of biomechanical sensors and wearable robotics systems.

Quantitatively deciphering the relationship between muscle activation and thickness deformation is essential for diagnosing muscle-related diseases and monitoring muscle health (e.g., Facioscapulohumeral Dystrophy). Despite the potential of ultrasound (US) imaging and sensing to measure changes in muscle thickness during movements, it remains challenging to make a fully portable device, considering the wiring and data collection. On the other hand, surface electromyography (sEMG) can record muscle bioelectrical signals and measure muscle activations, offering a unique perspective that correlates with underlying changes in muscle thickness. This paper introduces a deep-learning-based approach that used sEMG signals to infer muscle deformation. Using a hierarchical combination of self-attention and cross-attention mechanisms, this method predicted muscle deformation directly from sEMG data, eliminating the dependency on applying ultrasound imaging techniques. The experimental results on six healthy subjects indicated that our approach could accurately predict muscle excursion with an average precision of 0.923$\pm$0.900mm, showing benefits in measuring muscle deformation only with a sEMG device. This technique facilitates real-time portable muscle health monitoring by sEMG to provide bioelectrical signals and biomechanical information. It indicates the great potential of using this technique in clinical diagnostics, sports science, and rehabilitation.