Deep convolutional neural networks (CNNs) have been widely used in various medical imaging tasks. However, due to the intrinsic locality of convolution operations, CNNs generally cannot model long-range dependencies well, which are important for accurately identifying or mapping corresponding breast lesion features computed from unregistered multiple mammograms. This motivated us to leverage the architecture of Multi-view Vision Transformers to capture long-range relationships of multiple mammograms from the same patient in one examination. For this purpose, we employed local transformer blocks to separately learn patch relationships within four mammograms acquired from two-view (CC/MLO) of two-side (right/left) breasts. The outputs from different views and sides were concatenated and fed into global transformer blocks, to jointly learn patch relationships between four images representing two different views of the left and right breasts. To evaluate the proposed model, we retrospectively assembled a dataset involving 949 sets of mammograms, which included 470 malignant cases and 479 normal or benign cases. We trained and evaluated the model using a five-fold cross-validation method. Without any arduous preprocessing steps (e.g., optimal window cropping, chest wall or pectoral muscle removal, two-view image registration, etc.), our four-image (two-view-two-side) transformer-based model achieves case classification performance with an area under ROC curve (AUC = 0.818 ± 0.039), which significantly outperforms AUC = 0.784 ± 0.016 achieved by the state-of-the-art multi-view CNNs (p = 0.009). It also outperforms two one-view-two-side models that achieve AUC of 0.724 ± 0.013 (CC view) and 0.769 ± 0.036 (MLO view), respectively. The study demonstrates the potential of using transformers to develop high-performing computer-aided diagnosis schemes that combine four mammograms.

Deep convolutional neural networks (CNNs) have been widely used in various medical imaging tasks. However, due to the intrinsic locality of convolution operation, CNNs generally cannot model long-range dependencies well, which are important for accurately identifying breast cancer from unregistered multi-view mammograms. This motivates us to leverage the architecture of Vision Transformers to capture long-range relationships of multiple mammograms from the same patient. For this purpose, we employ local Transformer blocks to separately learn patch relationships within a specific view (CC/MLO) of one side (right/left) of mammogram. The outputs from different views and sides are concatenated and fed into global Transformer blocks, to jointly learn patch relationships between two different views of the left and right breasts. To evaluate the proposed model, we retrospectively assembled a dataset involving 949 sets of CC and MLO view mammograms, which include 470 malignant cases and 479 normal or benign cases. We trained and evaluated the model using a five-fold cross-validation method. Without any arduous preprocessing steps (e.g., optimal window cropping, chest wall or pectoral muscle removal, two-view image registration, etc.), our two-view Transformer-based model achieves lesion classification accuracy of 77.0% and area under ROC curve (AUC = 0.814), outperforming state-of-the-art multi-view CNNs by 3.1% and 3%, respectively. Meanwhile, the new two-view model improves mammographic case classification accuracies of two single-view models by 7.4% (CC) and 4.5% (MLO), respectively. The promising results unveil the great potential of using Transformers to develop high-performing computer-aided diagnosis (CADx) schemes of mammography.

Leukemia is one category of blood cancers that commonly occur in both female and male patients.¹ For the diagnosis and treatment of leukemia, chromosome karyotyping² is critically important; it organizes and pairs all chromosomes in order of decreasing length. In this process, cytogeneticists have to search the whole specimen slide and identify the analyzable metaphase chromosomes, which is a time-intensive and tedious operation. To reduce the clinician’s workload and enhance the diagnostic efficiency, many research studies have been focused on the development of automated slide scanners³ and the associated computer-aided detection schemes,^4,5 aiming to accomplish quick cell digitization and identification. Despite some encouraging progress, the performance of the current scanners is majorly impeded by the long scanning time, which can be attributed to the limited space-bandwidth product of the objective lens.^6,7 With few high-cost exceptions,⁸ this limitation indicates that it is very difficult for the objective lens to simultaneously accomplish a large field of view (FOV) and high spatial resolution, which implies that high-resolution objective lenses will have a much smaller FOV than the low-resolution lens. Considering that the chromosome images must be obtained with high spatial resolution to ensure enough details for clinical karyotyping, the corresponding high-magnification scanning will be significantly slower as each acquisition only covers a much smaller area of the specimen.

Background and Objective: 2D and 3D tumor features are widely used in a variety of medical image analysis tasks. However, for chemotherapy response prediction, the effectiveness between different kinds of 2D and 3D features are not comprehensively assessed, especially in ovarian-cancer-related applications. This investigation aims to accomplish such a comprehensive evaluation. Methods: For this purpose, CT images were collected retrospectively from 188 advanced-stage ovarian cancer patients. All the metastatic tumors that occurred in each patient were segmented and then processed by a set of six filters. Next, three categories of features, namely geometric, density, and texture features, were calculated from both the filtered results and the original segmented tumors, generating a total of 1403 and 1595 features for the 2D and 3D tumors, respectively. In addition to the conventional single-slice 2D and full-volume 3D tumor features, we also computed the incomplete-3D tumor features, which were achieved by sequentially adding one individual CT slice and calculating the corresponding features. Support vector machine (SVM)-based prediction models were developed and optimized for each feature set. Five-fold cross-validation was used to assess the performance of each individual model. Results: The results show that the 2D feature-based model achieved an AUC (area under the ROC curve (receiver operating characteristic)) of 0.84 ± 0.02. When adding more slices, the AUC first increased to reach the maximum and then gradually decreased to 0.86 ± 0.02. The maximum AUC was yielded when adding two adjacent slices, with a value of 0.91 ± 0.01. Conclusions: This initial result provides meaningful information for optimizing machine learning-based decision-making support tools in the future.