Deep learning models have been instrumental in extracting critical indicators for breast cancer diagnosis - the prevalent malignancy among women worldwide - from baseline magnetic resonance imaging. However, many existing models do not fully leverage the rich spatial information available in the 3D structure of medical imaging data, potentially overlooking important contextual details. This develops an explainable deep learning framework for classifying breast cancer that leverages the complete 3D and provides classification results alongside visual explanations of the decision-making process. The preprocessing pipeline is fed with 3D sequences containing ‘tumour’ and ‘non-tumour’ regions. It includes a 3D Adaptive Unsharp Mask (AUM) filter to reduce noise and augment image class, followed by normalisation and data augmentation. Classification is then achieved by training an augmented ResNet150 model. Three explainable artificial intelligence (XAI) techniques, including Shapley Additive Explanations, 3D Gradient-Weighted Class Activation Mapping, and Contextual Importance and Utility, are employed to provide improved interpretability. The model demonstrates state-of-the-art performance over the QIN-BREAST dataset, achieving testing accuracies of 98.861% for ‘tumours’ and 99.447% for ‘non-tumours’, as well as over the Duke Breast Cancer Dataset, where it achieves 99.104% for ‘tumours’ and 99.753% for ‘non-tumours’, while offering enhanced interpretability through XAI methods.

Physicians typically combine multi-modal data to make a graded diagnosis of breast tumors. However, most existing breast tumor grading methods rely solely on image information, resulting in limited accuracy in grading. This paper proposes a Multi-information Selection Aggregation Graph Convolutional Networks (MSA-GCN) for breast tumor grading. Firstly, to fully utilize phenotypic data reflecting the clinical and pathological characteristics of tumors, an automatic combination screening and weight encoder is proposed for phenotypic data, which can construct a population graph with improved structural information. Then, a graph structure is designed through similarity learning to reflect the correlation between patient image features. Finally, a multi-information selection aggregation mechanism is employed in the graph convolution model to extract the effective features of multi-modal data and enhance the classification performance of the model. The proposed method is evaluated on different clinical datasets from the Digital Database for Screening Mammography (DDSM) and INbreast. The average classification accuracies are 90.74% and 85.35%, respectively, surpassing the performance of existing methods. In conclusion, our method effectively fuses image and non-image information, leading to a significant improvement in the accuracy of breast tumor grading.

Breast cancer is a serious threat to women's life and health. Early grading of breast tumor malignancy is the key for improving patient survival rate. In clinical diagnosis, physicians often utilize multi-modal data information to accurately grade. As a new multi-modal data processing method, graph convolutional network is expected to improve the grading accuracy of breast tumors. To solve the problem that a single graph cannot effectively utilize multi-modal information, this paper proposes a multi-graph adaptive fusion network to perform accurate breast tumor grading by using multi-modal data. Firstly, a multi-modal information extraction network is designed to extract specific modal information and graph structure from multi-modal data, which is composed of graph structure learning layer and graph auto-encoder. Secondly, by calculating the attention scores of different modalities as weight coefficients, the modal aggregation features, and modal aggregation graphs are obtained by fusing in a weighted manner. Finally, a neighborhood adaptive aggregation module is designed to gather multi graphs and optimize the conventional graph convolution aggregation mode by calculating the neighborhood aggregation coefficient. Experimental results on both public and private breast tumor datasets show that the graph learning-based diagnosis model can effectively process multi-modal medical data and improve the accuracy of multi grading for breast tumors.