22 October 2021 in IEEE Transactions on Instrumentation and Measurement
Genitourinary syndrome of menopause (GSM) is a disease caused by a physiological decline in estrogen levels, and it can negatively affect a woman’s overall health and quality of life in terms of sexual function. Real-time optical biopsy images can now be obtained with optical coherence tomography (OCT) systems. In this study, we introduce vision transformer (ViT) to the field of medical OCT images for the first time and propose a deep learning-based approach for GSM lesion screening. Specifically, we first build a GSM dataset to train and evaluate the experimental model performance. The study aims to propose a method that combines null convolution with a deep convolutional adversarial generative network classifier to generate the samples needed for training to alleviate the hindrance of such problems, in response to certain practical problems, such as category imbalance that occur during data collection. Next, the experiments present ViT PLUS (ViT-P) for the vaginal OCT image classification task used, which effectively improves the shortcomings of ViT in extracting Patch Embedding using a multibranch convolutional neural network combined with a channel attention mechanism. The clinical images acquired by the OCT device are then automatically classified on the basis of the OCT device to reduce the medical workload of gynecologists. Experimental results show that the ViT-P model outperforms the CNN model and ViT for case screening in the GSM and UCSD datasets, and the accuracy can reach 99.9% and 99.69%, respectively.
3 June 2024 in Computers and Geotechnics
1 November 2023 in IEEE Transactions on Instrumentation and Measurement
Diabetic retinopathy (DR) is a common complication of diabetes and one of the main causes of blindness in humans, which can be prevented by early-stage detection and treatment. Clinically, ophthalmologists use optical coherence tomography (OCT) image analysis as a basis for diagnosing DR. The existing medical resources can no longer meet the needs of the escalating patient population. Therefore, deep learning technology has become a mainstream solution for medical image analysis. Vision Transformer (ViT), a new neural network structure, has demonstrated great performance in analyzing images. However, due to the lack of inductive bias and prohibition of input image changes in size, ViT cannot avoid over-fitting problems on small datasets and limits the model to biological tissue characteristics. Thus, we propose an OCT multi-head self-attention (OMHSA) block that especially calculates OCT image information based on a hybrid CNN-Transformer strategy. Compared to traditional MHSA, OMHSA integrates local information extraction differences into the calculation of self-attention and adds local information to the transformer model without relying on a multi-branch network establishment. We built a neural network architecture (OCT-Former) by stacking convolutional layers and OMHSA blocks repeatedly in each stage. Similar to CNN, OCTFormer allows input size change at each stage to achieve a hierarchical structure effect. The model diagnosis effectiveness on the collected retinal OCT dataset was evaluated, and the accuracy reached 98.60%, surpassing the state-of-the-art (SOTA) model. The OCTFormer deployment to mobile terminals through knowledge distillation technology was shown, which presented a reference for deploying transformer models to actual clinical environments.
18 October 2024 in Neural Networks