Breast cancer remains a leading cause of mortality among women, underlining the need for more effective detection techniques. Recently, AI technologies such as computer vision, machine learning, and deep learning have become increasingly popular in medical imaging and have significantly advanced early cancer detection, enhancing patient outcomes and survival rates. This study presents a novel Hybrid deep learning model EfficientNetV2 and Vision Transformer (EffNetV2+ViT) for identifying breast cancer. The hybrid model is a combination of EfficientNetV2, which is ideal for initially extracting key features from the data and passing those features to the vision transformer, which is effective in data classification. We employed this model on the “BreakHis” dataset. As in this study, three different combinations of the EfficientNetV2 (small, medium, and large) are used, in order to extract the key features from the dataset and optimize the feature extraction process. These features are then fed to the vision transformer, to determine the presence of breast cancer. The models were trained and evaluated for binary and multi-class classification using different magnification levels of BrakHis such as 40x,100x,200x and 400x. Our experiments demonstrate promising results, particularly when using the “large” version of the EfficientNetV2 with the vision transformer, which is successful in detecting breast cancer, achieving an accuracy of 99.83% in binary classifications (benign, malignant) and 98.10% in multi-class scenario (fibro adenoma, ad enosis, tubular adenoma, phyllodes tumour, mucinous carcinoma, papillary carcinoma, lobular carcinoma, ductal carcinoma). These findings suggest that integrating these two approaches could significantly enhance the accuracy of breast cancer detection.

Background: Accurate tumor detection and quantification are important for optimized therapy planning and evaluation. Total tumor burden is also an appealing biomarker for clinical trials. Manual examination and annotation of oncologic PET/CT is labor-intensive and demands a high level of expertise. One significant challenge is the risk for human error, leading to potential omission of especially small tumors and tumors with low FDG uptake. Purpose: In this study, we introduced an automated framework with segmentation prior, from a tissue-wise multi-channel multi-angled based approach, to enhance tumor segmentation in whole-body FDG-PET/CT. Method: The proposed framework utilized a segmentation prior generated from tumor segmentations in tissue-wise multi-channel projections of the standardized uptake value (SUV) from PET. Projections were created from various angles and the tissues were identified based on their CT Hounsfield values. The resulting segmentation masks were subsequently backprojected into a unified 3D volume for creation of the segmentation prior. Finally, the segmentation prior was provided as an additional input channel along with the CT and SUV images to three variants of 3D segmentation networks (3D UNet, dynUNet, nnUNet) to enhance the overall tumor segmentation performance. All the methods were independently evaluated using 5-fold cross-validation on the autoPET dataset and subsequently tested on the U-CAN dataset. Results: Combining the segmentation prior with the original SUV and CT images improved overall tumor segmentation performance significantly compared to a baseline network. The increase in Dice coefficient for lymphoma, lung cancer, and melanoma across different segmentation networks were: 3D UNet ( 0.04 ⁎ , 0.02 ⁎ , 0.11 ⁎ ), dynUNet ( 0.05 ⁎ , 0.04 ⁎ , 0.08 ⁎ ), and nnUNet ( 0.02 ⁎ , 0.00 n s , 0.03 ⁎ ), respectively; *, p-value < 0.05; ns, non-significance. Conclusion: The increased segmentation accuracy could be attributed to the segmentation prior generated from tissue-wise SUV projections, revealing information from various tissues that was useful for segmentation of tumors. The results from this study highlight the potential of the proposed method as a valuable future tool for time-efficient quantification of tumor burden in oncologic FDG-PET/CT.