26 October 2023 in IEEE Journal of Biomedical and Health Informatics
Physicians typically combine multi-modal data to make a graded diagnosis of breast tumors. However, most existing breast tumor grading methods rely solely on image information, resulting in limited accuracy in grading. This paper proposes a Multi-information Selection Aggregation Graph Convolutional Networks (MSA-GCN) for breast tumor grading. Firstly, to fully utilize phenotypic data reflecting the clinical and pathological characteristics of tumors, an automatic combination screening and weight encoder is proposed for phenotypic data, which can construct a population graph with improved structural information. Then, a graph structure is designed through similarity learning to reflect the correlation between patient image features. Finally, a multi-information selection aggregation mechanism is employed in the graph convolution model to extract the effective features of multi-modal data and enhance the classification performance of the model. The proposed method is evaluated on different clinical datasets from the Digital Database for Screening Mammography (DDSM) and INbreast. The average classification accuracies are 90.74% and 85.35%, respectively, surpassing the performance of existing methods. In conclusion, our method effectively fuses image and non-image information, leading to a significant improvement in the accuracy of breast tumor grading.
1 January 2026 in IEEE Journal of Biomedical and Health Informatics
Breast cancer is a serious threat to women's life and health. Early grading of breast tumor malignancy is the key for improving patient survival rate. In clinical diagnosis, physicians often utilize multi-modal data information to accurately grade. As a new multi-modal data processing method, graph convolutional network is expected to improve the grading accuracy of breast tumors. To solve the problem that a single graph cannot effectively utilize multi-modal information, this paper proposes a multi-graph adaptive fusion network to perform accurate breast tumor grading by using multi-modal data. Firstly, a multi-modal information extraction network is designed to extract specific modal information and graph structure from multi-modal data, which is composed of graph structure learning layer and graph auto-encoder. Secondly, by calculating the attention scores of different modalities as weight coefficients, the modal aggregation features, and modal aggregation graphs are obtained by fusing in a weighted manner. Finally, a neighborhood adaptive aggregation module is designed to gather multi graphs and optimize the conventional graph convolution aggregation mode by calculating the neighborhood aggregation coefficient. Experimental results on both public and private breast tumor datasets show that the graph learning-based diagnosis model can effectively process multi-modal medical data and improve the accuracy of multi grading for breast tumors.