4 August 2017 in IEEE Transactions on Robotics
We present a control framework for optimizing image quality during robotic ultrasound acquisitions. The quality of the ultrasound signal across the field of view is represented by a confidence map that is computed online from the B-mode frames, following a model of sound propagation. Moments extracted from this confidence map are used to design a control law for optimizing imaging quality, based on the task function approach. The proposed confidence control is combined with force and position control to build two illustrative applications. First, we use force control and confidence control in order to maintain a correct pressure and a good orientation of the probe during teleoperation. Thus, control is shared between a human operator and the robot. Then, we add an automatic positioning task, so that the quality is optimized while maintaining a target in the image center. We show experimentally that confidence-driven control can effectively optimize the acoustic window in real time. In addition, we show that it can improve the tracking robustness, by preventing the target from being shadowed. Finally, we present the results of experiments performed on a human volunteer.
1 October 2020 in Medical Image Analysis
We present a novel multi-task neural network called Temporal SonoEyeNet (TSEN) with a primary task to describe the visual navigation process of sonographers by learning to generate visual attention maps of ultrasound images around standard biometry planes of the fetal abdomen, head (trans-ventricular plane) and femur. TSEN has three components: a feature extractor, a temporal attention module (TAM), and an auxiliary video classification module (VCM). A soft dynamic time warping (sDTW) loss function is used to improve visual attention modelling. Variants of the model are trained on a dataset of 280 video clips, each containing one of the three biometry planes and lasting 3-7 seconds, with corresponding real-time recorded gaze tracking data of an experienced sonographer. We report the performances of the different variants of TSEN for visual attention prediction at standard biometry plane detection. The best model performance is achieved using bi-directional convolutional long-short term memory (biCLSTM) in both TAM and VCM, and it outperforms a previous spatial model on all static and dynamic saliency metrics. As an auxiliary task to validate the clinical relevance of the visual attention modelling, the predicted visual attention maps were used to guide standard biometry plane detection in consecutive US video frames. All spatio-temporal TSEN models achieve higher scores compared to a spatial-only baseline; the best performing TSEN model achieves F1 scores on these standard biometry planes of 83.7%, 89.9% and 81.1%, respectively.