Robotic breast ultrasound (RBUS) aims to standardize breast ultrasonography, reduce the workload of sonographers, and provide high-quality ultrasound (US) images for subsequent diagnosis. In the process of RBUS screening, adjusting the US probe correctly and efficiently to acquire high-quality US images is fundamental and significant. In this letter, a learning-based US probe adjustment framework is proposed. Firstly, a lightweight multi-task combination approach is utilized for jointly assessing US imaging quality with multiple indicators. Then, an experience-guided learning algorithm, called broad reinforcement learning from demonstration (BRLfD), is proposed to efficiently learn the optimal US probe adjustment policy. The effectiveness of the learned policy is verified in five testing lesion locations. The results show that the US probe can reach the goal state in less than 5 steps on average, and the proposed method can automatically adjust the US probe efficiently to obtain high-quality breast US images for clinical diagnosis.

Ultrasound (US) examination is widely used to diagnose carotid artery plaque, which requires the sonographer to guide the probe to scan along a specific path for complete coverage of the carotid artery region. Meanwhile, stable probe-neck interaction is important for high-quality image acquisition. In this study, a robotic system for autonomous carotid US scanning is proposed. To realize the autonomous visual servo movement of the probe, an object tracking method based on improved Siamese network is proposed. Meanwhile, a local quality assessment algorithm is proposed to ensure that carotid ultrasound images are clear and desirable for diagnosis. To address the issue of poor probe-neck contact and loss of carotid object during ultrasound scanning, an automatic recovery control method is proposed to ensure the continuity of the scanning process without the need to stop and restart the scanning. Experimental results show that the robotic system can successfully navigate the probe to move along a path that meets the clinical standard. In addition, the robot can autonomously rediscover the object and return to the normal scanning state if a stuck condition occurs.

Offline reinforcement learning (ORL) has been getting increasing attention in robot learning, benefiting from its ability to avoid hazardous exploration and learn policies directly from precollected samples. Approximate policy iteration (API) is one of the most commonly investigated ORL approaches in robotics, due to its linear representation of policies, which makes it fairly transparent in both theoretical and engineering analysis. One open problem of API is how to design efficient and effective basis functions. The broad learning system (BLS) has been extensively studied in supervised and unsupervised learning in various applications. However, few investigations have been conducted on ORL. In this article, a novel incremental ORL approach with sparse broad critic approximation (BORL) is proposed with the advantages of BLS, which approximates the critic function in a linear manner with randomly projected sparse and compact features and dynamically expands its broad structure. The BORL is the first extension of API with BLS in the field of robotics and ORL. The approximation ability and convergence performance of BORL are also analyzed. Comprehensive simulation studies are then conducted on two benchmarks, and the results demonstrate that the proposed BORL can obtain comparable or better performance than conventional API methods without laborious hyperparameter fine-tuning work. To further demonstrate the effectiveness of BORL in practical robotic applications, a variable force tracking problem in robotic ultrasound scanning (RUSS) is investigated, and a learning-based adaptive impedance control (LAIC) algorithm is proposed based on BORL. The experimental results demonstrate the advantages of LAIC compared with conventional force tracking methods.

As an effective and efficient discriminative learning method, the broad learning system (BLS) has received increasing attention due to its outstanding performance without large computational resources. The standard BLS is derived under the minimum mean square error (MMSE) criterion, while MMSE is with poor performance when dealing with imbalanced data. However, imbalanced data are widely encountered in real-world applications. To address this issue, a novel cost-sensitive BLS algorithm (CS-BLS) is proposed. In the CS-BLS, many variations can be adopted, and CS-BLS with weighted cross-entropy is analyzed in this paper. Weighted penalty factors are used in CS-BLS to constrain the contribution of each sample in different classes. The samples in minor classes are allocated higher weights to increase their contributions. Four different weight calculation methods are adopted to the CS-BLS, and thus, four CS-BLS methods are proposed: Log-CS-BLS, Lin-CS-BLS, Sqr-CS-BLS, and EN-CS-BLS. Experiments based on artificially imbalanced datasets of MNIST and small NORB are firstly conducted and compared with the standard BLS. The results show that the proposed CS-BLS methods have better generalization and robustness than the standard BLS. Then, experiments on a real ultrasound breast image dataset are conducted, and the results demonstrate that the proposed CS-BLS methods are effective in actual medical diagnosis.

The quality of breast ultrasound images has a significant impact on the accuracy of disease diagnosis. Existing image quality assessment (IQA) methods usually use pixel-level feature statistical methods or end-to-end deep learning methods, which focus on the global image quality but ignore the image quality of the lesion region. However, in clinical practice, doctors’ evaluation of ultrasound image quality relies more on the local area of the lesion, which determines the diagnostic value of ultrasound images. In this study, a global–local integrated IQA framework for breast ultrasound images was proposed to learn doctors’ clinical evaluation standards. In this study, 1285 breast ultrasound images were collected and scored by experienced doctors. After being classified as either images with lesions or images without lesions, they were evaluated using soft-reference IQA or bilinear CNN IQA, respectively. Experiments showed that for ultrasound images with lesions, our proposed soft-reference IQA achieved PLCC 0.8418 with doctors’ annotation, while the existing end-to-end deep learning method that did not consider the local lesion features only achieved PLCC 0.6606. Due to the accuracy improvement for the images with lesions, our proposed global–local integrated IQA framework had better performance in the IQA task than the existing end-to-end deep learning method, with PLCC improving from 0.8306 to 0.8851.

Automatic and accurate segmentation and classification of esophageal lesions are two essential tasks to assist endoscopists in Upper Gastrointestinal Endoscopy. However, there is no intelligent system that can diagnose more lesion types, handle multiple tasks simultaneously, and be more accurate in clinical work. Therefore, we present an innovative Multi-Task deep learning architecture named Attention Gating Pyramid Vision Transformer (AGPVT), which provides a solution for the accurate classification and precise segmentation of lesion types and regions simultaneously. The proposed AGPVT combines the benefits of cutting-edge deep learning model designs with Multi-Task Learning (MTL) in order to advance the field. Furthermore, a patch-wise multi-head attention gating method alongside a hybrid design MTL decoder, is employed as the core driving architecture of the AGPVT. Comprehensive experiments are conducted on a multicenter dataset which contains esophageal cancer, Barrett's esophagus, esophageal protruded lesions, esophagitis, and normal esophagus. Experimental results show that the proposed AGPVT achieves a classification accuracy of 96.84%, an IoU score of 85.61%, and a Dice score of 90.75%, outperforming existing methods and demonstrating its effectiveness in this domain.