With the advancements in deep learning methods, AI systems now perform at the same or higher level than human intelligence in many complex real-world problems. The data and algorithmic opacity of deep learning models, however, make the task of comprehending the input data information, the model, and model’s decisions quite challenging. This lack of transparency constitutes both a practical and an ethical issue. For the present study, it is a major drawback to the deployment of deep learning methods mandated with detecting patterns and prognosticating Alzheimer’s disease. Many approaches presented in the AI and medical literature for overcoming this critical weakness are sometimes at the cost of sacrificing accuracy for interpretability. This study is an attempt at addressing this challenge and fostering transparency and reliability in AI-driven healthcare solutions. The study explores a few commonly used perturbation-based interpretability (LIME) and gradient-based interpretability (Saliency and Grad-CAM) approaches for visualizing and explaining the dataset, models, and decisions of MRI image-based Alzheimer’s disease identification using the diagnostic and predictive strengths of an ensemble framework comprising Convolutional Neural Networks (CNNs) architectures (Custom multi-classifier CNN, VGG-19, ResNet, MobileNet, EfficientNet, DenseNet), and a Vision Transformer (ViT). The experimental results show the stacking ensemble achieving a remarkable accuracy of 98.0% while the hard voting ensemble reached 97.0%. The findings present a valuable contribution to the growing field of explainable artificial intelligence (XAI) in medical imaging, helping end users and researchers to gain deep understanding of the backstory behind medical image dataset and deep learning model’s decisions.

The lack of extensive, varied, and thoroughly annotated datasets impedes the advancement of artificial intelligence (AI) for medical applications, especially colorectal cancer detection. Models trained with limited diversity often display biases, especially when utilized on disadvantaged groups. Generative models (e.g., DALL-E 2, Vector-Quantized Generative Adversarial Network (VQ-GAN)) have been used to generate images but not colonoscopy data for intelligent data augmentation. This study developed an effective method for producing synthetic colonoscopy image data, which can be used to train advanced medical diagnostic models for robust colorectal cancer detection and treatment. Text-to-image synthesis was performed using fine-tuned Visual Large Language Models (LLMs). Stable Diffusion and DreamBooth Low-Rank Adaptation produce images that look authentic, with an average Inception score of 2.36 across three datasets. The validation accuracy of various classification models Big Transfer (BiT), Fixed Resolution Residual Next Generation Network (FixResNeXt), and Efficient Neural Network (EfficientNet) were 92%, 91%, and 86%, respectively. Vision Transformer (ViT) and Data-Efficient Image Transformers (DeiT) had an accuracy rate of 93%. Secondly, for the segmentation of polyps, the ground truth masks are generated using Segment Anything Model (SAM). Then, five segmentation models (U-Net, Pyramid Scene Parsing Network (PSNet), Feature Pyramid Network (FPN), Link Network (LinkNet), and Multi-scale Attention Network (MANet)) were adopted. FPN produced excellent results, with an Intersection Over Union (IoU) of $0.64$, an F1 score of $0.78$, a recall of $0.75$, and a Dice coefficient of 0.77. This demonstrates strong performance in terms of both segmentation accuracy and overlap metrics, with particularly robust results in balanced detection capability as shown by the high F1 score and Dice coefficient. This highlights how AI-generated medical images can improve colonoscopy analysis, which is critical for early colorectal cancer detection.

Breast cancer detection through mammography interpretation remains difficult because of the minimal nature of abnormalities that experts need to identify alongside the variable interpretations between readers. The potential of CNNs for medical image analysis faces two limitations: they fail to process both local information and wide contextual data adequately and do not provide explainable AI (XAI) operations which doctors need to accept them in clinics. The researcher developed the MammoFormer framework which unites transformer-based architecture with multi-feature enhancement components and XAI functionalities within one framework. Seven different architectures consisting of CNNs, Vision Transformer, Swin Transformer, and ConvNext were tested alongside four enhancement techniques, including original images, negative transformation, adaptive histogram equalization, and histogram of oriented gradients. The MammoFormer framework addresses critical clinical adoption barriers of AI mammography systems through: (1) systematic optimization of transformer architectures via architecture-specific feature enhancement, achieving up to 13% performance improvement, (2) comprehensive explainable AI integration providing multi-perspective diagnostic interpretability, and (3) a clinically deployable ensemble system combining CNN reliability with transformer global context modeling. The combination of transformer models with suitable feature enhancements enables them to achieve equal or better results than CNN approaches. ViT achieves 98.3% accuracy alongside AHE while Swin Transformer gains a 13.0% advantage through HOG enhancements. Five XAI techniques, including Integrated Gradients, GradCAM, Occlusion, DeepLIFT and Saliency maps demonstrate that transformer models effectively recognize diagnostically significant features by capturing long-range dependencies in mammograms. Using HOG features provides the most reliable enhancement impact (98.4% average accuracy) among all visualization methods. MammoFormer establishes a clinical breast cancer screening workflow with diagnostic precision and interpretability through its simultaneous implementation of architectural design with feature enhancement and explainability features.

Multimodal artificial intelligence (AI) models offer a promising approach for automatically generating captions from radiology images, bridging visual and textual data to enhance medical image interpretation. This study compares a selectively fine-tuned Large Multimodal Model (LMM) with a fully fine-tuned multimodal encoder-decoder architecture on a subset of the Radiology Objects in Context version 2 (ROCOv2) dataset. Specifically, Vision Generative Pretrained Transformer 2 (VisionGPT-2) is evaluated against Large Language and Vision Assistant (LLaVA) version 1.6, a Mistral-7B-based LMM adapted using Low-Rank Adaptation (LoRA) on selected projection matrices. Experimental results show that LLaVA-1.6 outperforms VisionGPT-2 while significantly reducing the computational cost of full fine-tuning through selective adaptation with LoRA. Performance gains are observed across multiple metrics, including MedBERTScore (0.633), ClinicalBLEURT (0.456), ROUGE (0.251), BLEU-1 (0.209), BLEURT (0.317), METEOR (0.092), CIDEr (0.245), CLIPScore (0.816), RefCLIPScore (0.815), and BERTScore (0.628). These findings underscore the potential of selectively fine-tuned LMMs for efficient, semantically rich caption generation in medical imaging tasks, particularly in contexts constrained by sensitive data and limited computational resources.

Rapidly growing medical imaging volumes have led to an increasing workload for radiologists, creating the need for automated tools that can support interpretation and reduce reporting delays. Vision-language models (VLMs) can generate clinically relevant captions to accelerate report drafting, but their varying parameter scales require evaluation for clinical utility. This study evaluated fine-tuned VLMs on the Radiology Objects in Context version 2 (ROCOv2) dataset containing 116,635 images across multiple modalities. We compared four Large VLMs (LVLMs) (LLaVA-Mistral-7B, LLaVA-Vicuna-7B, LLaVA-1.5-LLaMA-7B, IDEFICS-9B) against four Smaller VLMs (SVLMs) (MoonDream2, Qwen 2-VL, Qwen-2.5, SmolVLM) alongside two fully fine-tuned baseline architectures (VisionGPT2 and CNN-Transformer). Low-Rank Adaptation (LoRA), applied to fewer than 1% of selected model parameters, proved the optimal performance among adaptation strategies, outperforming broader LoRA configurations that we evaluated. LLaVA-Mistral-7B achieved the highest performance (Relevance: 0.516, Factuality: 0.118), substantially exceeding VisionGPT2 baseline (0.325, 0.028). Among SVLMs, MoonDream2 reached a relevance score of 0.466, surpassing LLaVA-1.5 (0.462) despite using approximately 74% fewer parameters. Models showed distinct performance hierarchy with LVLMs (0.273–0.317 overall), SVLMs (0.188–0.279), and baselines (0.154–0.177). To investigate performance enhancement strategies, we prepended ResNet-50-predicted image modality labels at inference time for underperforming SVLMs. This intervention produced variable results, with SmolVLM showing marginal improvement, Qwen-2.5 gaining 6.4%, but Qwen 2-VL experiencing a 21.6% performance reduction. Our results provide quantitative guidance for VLM selection in medical imaging. Although model size strongly influences performance, the findings indicate that architectural design and lightweight adaptation can enable select small models to achieve viable performance for resource-constrained screening scenarios.

Rapidly growing medical imaging volumes have increased radiologist workloads, creating demand for automated tools that support interpretation and reduce reporting delays. Vision-language models (VLMs) can generate clinically relevant captions to accelerate report drafting, yet their varying parameter scales require systematic evaluation for clinical utility. This study evaluated ten multimodal models fine-tuned on the Radiology Objects in Context version 2 (ROCOv2) dataset containing 116,635 images across eight modalities. We compared four Large VLMs (LVLMs) including LLaVA variants and IDEFICS-9B against four Small VLMs (SVLMs) including MoonDream2, Qwen variants, and SmolVLM, alongside two fully fine-tuned baseline architectures (VisionGPT2 and CNN-Transformer). Low-Rank Adaptation (LoRA), applied to fewer than 1% of selected model parameters, proved optimal among adaptation strategies, outperforming broader LoRA configurations. Models were assessed on relevance (semantic similarity) and factuality (concept-level correctness) metrics. Performance showed clear stratification: LVLMs (0.273 to 0.317 overall), SVLMs (0.188 to 0.279), and baselines (0.154 to 0.177). LLaVA-Mistral-7B achieved the highest performance with relevance and factuality scores of 0.516 and 0.118, respectively, substantially exceeding the VisionGPT2 baseline (0.325, 0.028). Among the SVLMs, MoonDream2 demonstrated competitive relevance (0.466), approaching the performance of some LVLMs despite its smaller size. To investigate performance enhancement strategies for underperforming SVLMs, we prepended predicted imaging modality labels at inference time, which yielded variable results. These findings provide quantitative benchmarks for VLM selection in medical imaging, demonstrating that while model scale influences performance, architectural design and targeted adaptation enable select compact models to achieve competitive results.