Anomaly detection is an important task for medical image analysis, which can alleviate the reliance of supervised methods on large labelled datasets. Most existing methods use a pixel-wise self-reconstruction framework for anomaly detection. However, there are two challenges of these studies: 1) they tend to overfit learning an identity mapping between the input and output, which leads to failure in detecting abnormal samples; 2) the reconstruction considers the pixel-wise differences which may lead to an undesirable result. To mitigate the above problems, we propose a novel heterogeneous Auto-Encoder (Hetero-AE) for medical anomaly detection. Our model utilizes a convolutional neural network (CNN) as the encoder and a hybrid CNN-Transformer network as the decoder. The heterogeneous structure enables the model to learn the intrinsic information of normal data and enlarge the difference on abnormal samples. To fully exploit the effectiveness of Transformer in the hybrid network, a multi-scale sparse Transformer block is proposed to trade off modelling long-range feature dependencies and high computational costs. Moreover, the multi-stage feature comparison is introduced to reduce the noise of pixel-wise comparison. Extensive experiments on four public datasets (i.e., retinal OCT, chest X-ray, brain MRI, and COVID-19 ) verify the effectiveness of our method on different imaging modalities for anomaly detection. Additionally, our method can accurately detect tumors in brain MRI and lesions in retinal OCT with interpretable heatmaps to locate lesion areas, assisting clinicians in diagnosing abnormalities efficiently.

Unsupervised anomaly detection (UAD) aims to recognize anomalous images based on the training set that contains only normal images. In medical image analysis, UAD benefits from leveraging the easily obtained normal (healthy) images, avoiding the costly collecting and labeling of anomalous (unhealthy) images. Most advanced UAD methods rely on frozen encoder networks pre-trained using ImageNet for extracting feature representations. However, the features extracted from the frozen encoders that are borrowed from natural image domains coincide little with the features required in the target medical image domain. Moreover, optimizing encoders usually causes pattern collapse in UAD. In this paper, we propose a novel UAD method, namely Encoder-Decoder Contrast (EDC), which optimizes the entire network to reduce biases towards pre-trained image domain and orient the network in the target medical domain. We start from feature reconstruction approach that detects anomalies from reconstruction errors. Essentially, a contrastive learning paradigm is introduced to tackle the problem of pattern collapsing while optimizing the encoder and the reconstruction decoder simultaneously. In addition, to prevent instability and further improve performances, we propose to bring globality into the contrastive objective function. Extensive experiments are conducted across four medical image modalities including optical coherence tomography, color fundus image, brain MRI, and skin lesion image, where our method outperforms all current state-of-the-art UAD methods. Code is available at: https://github.com/guojiajeremy/EDC

Glaucoma is a serious eye disease that can cause permanent blindness and is difficult to diagnose early. Optic disc (OD) and optic cup (OC) play a pivotal role in the screening of glaucoma. Therefore, accurate segmentation of OD and OC from fundus images is a key task in the automatic screening of glaucoma. In this paper, we designed a U-shaped convolutional neural network with multi-scale input and multi-kernel modules (MSMKU) for OD and OC segmentation. Such a design gives MSMKU a rich receptive field and is able to effectively represent multi-scale features. In addition, we designed a mixed maximum loss minimization learning strategy (MMLM) for training the proposed MSMKU. This training strategy can adaptively sort the samples by the loss function and re-weight the samples through data enhancement, thereby synchronously improving the prediction performance of all samples. Experiments show that the proposed method has obtained a state-of-the-art breakthrough result for OD and OC segmentation on the RIM-ONE-V3 and DRISHTI-GS datasets. At the same time, the proposed method achieved satisfactory glaucoma screening performance on the RIM-ONE-V3 and DRISHTI-GS datasets. On datasets with an imbalanced distribution between typical and rare sample images, the proposed method obtained a higher accuracy than existing deep learning methods.