We present a convolutional neural network (CNN) equipped with a novel and efficient adaptive dual attention module (ADAM) for automated skin lesion segmentation from dermoscopic images, which is an essential yet challenging step for the development of a computer-assisted skin disease diagnosis system. The proposed ADAM has three compelling characteristics. First, we integrate two global context modeling mechanisms into the ADAM, one aiming at capturing the boundary continuity of skin lesion by global average pooling while the other dealing with the shape irregularity by pixel-wise correlation. In this regard, our network, thanks to the proposed ADAM, is capable of extracting more comprehensive and discriminative features for recognizing the boundary of skin lesions. Second, the proposed ADAM supports multi-scale resolution fusion, and hence can capture multi-scale features to further improve the segmentation accuracy. Third, as we harness a spatial information weighting method in the proposed network, our method can reduce a lot of redundancies compared with traditional CNNs. The proposed network is implemented based on a dual encoder architecture, which is able to enlarge the receptive field without greatly increasing the network parameters. In addition, we assign different dilation rates to different ADAMs so that it can adaptively capture distinguishing features according to the size of a lesion. We extensively evaluate the proposed method on both ISBI2017 and ISIC2018 datasets and the experimental results demonstrate that, without using network ensemble schemes, our method is capable of achieving better segmentation performance than state-of-the-art deep learning models, particularly those equipped with attention mechanisms.

Polyp segmentation plays an important role in preventing Colorectal cancer. Although Vision Transformer has been widely introduced in medical image segmentation to compensate the limitations of traditional CNN in modeling global context, its shortcomings in learning the fine-detailed features and the heavy computation cost also hinder its application in challenging polyp segmentation due to the various shapes and sizes of polyps, the low-intensity contrast between polyps and surrounding tissues, and the inherent real-time requirement. In this paper, we propose a multi-scale efficient transformer attention (META) mechanism for fast and high-accuracy polyp segmentation, where efficient transformer blocks are employed to generate multi-scale element-wise attentions for adaptive feature fusion in the famous U-shape encoder-decoder architecture. Specifically, our META mechanism includes two branches to capture multi-scale long-term dependencies, which are implemented via two efficient transformer blocks with different resolutions. The local branch is used to capture a relatively smaller transform attention under a relatively lower resolution, while the global branch is used to capture high-resolution transform attention. The final poly segmentation results are progressively integrated based on the META mechanism in each layer of the decoder. Extensive experiments are conducted on four polyp segmentation datasets (CVC-ClinicDB, Endoscenestill, Kvasir-SEG and ETIS-Larib) to demonstrate its advantages, consistently outperforming different competitors. While using ResNet34 as backbones, it can achieve 85.78% IoU and 92.03% Dice, 88.99% IoU and 93.85% Dice, 86.42% IoU and 91.86% Dice respectively in CVC-ClinicDB, Endoscenestill, and Kvasir-SEG, and a speed of 98 FPS at the input size of $3 \times 512 \times 512$ on a NVIDIA GeForce RTX 3090 card. The code is available at https://github.com/szuzzb/META-Unet. Note to Practitioners —Automatic polyp segmentation is a crucial step of polyp recognition and diagnostic of colonoscopy, which usually require both high-accuracy and real-time performance. This article proposes a novel polyp segmentation method, namely META-Unet, by modeling multi-scale attention maps effectively and efficiently based on a novel multi-scale efficient transformer attention (META) mechanism, for faster and higher-accuracy polyp segmentation. We evaluate our META-Unet on four public polyp image segmentation datasets (CVC-ClinicDB, Endoscenestill, Kvasir-SEG and ETIS-Larib). Comprehensive experimental results validate its outstanding performance with a better balance in both accuracy and inference speed. The proposed META mechanism is potentially to be embedded in various deep learning frameworks and facilitates more computer-aided applications in clinical practice.

Alzheimer’s disease (AD) is characterized by alterations of the brain’s structural and functional connectivity during its progressive degenerative processes. Existing auxiliary diagnostic methods have accomplished the classification task, but few of them can accurately evaluate the changing characteristics of brain connectivity. In this work, a prior-guided adversarial learning with hypergraph (PALH) model is proposed to predict abnormal brain connections using triple-modality medical images. Concretely, a prior distribution from anatomical knowledge is estimated to guide multimodal representation learning using an adversarial strategy. Also, the pairwise collaborative discriminator structure is further utilized to narrow the difference in representation distribution. Moreover, the hypergraph perceptual network is developed to effectively fuse the learned representations while establishing high-order relations within and between multimodal images. Experimental results demonstrate that the proposed model outperforms other related methods in analyzing and predicting AD progression. More importantly, the identified abnormal connections are partly consistent with previous neuroscience discoveries. The proposed model can evaluate the characteristics of abnormal brain connections at different stages of AD, which is helpful for cognitive disease study and early treatment.

Automatic polyp segmentation from colonoscopy videos is a prerequisite for the development of a computer-assisted colon cancer examination and diagnosis system. However, it remains a very challenging task owing to the large variation of polyps, the low contrast between polyps and background, and the blurring boundaries of polyps. More importantly, real-time performance is a necessity of this task, as it is anticipated that the segmented results can be immediately presented to the doctor during the colonoscopy intervention for his/her prompt decision and action. It is difficult to develop a model with powerful representation capability, yielding satisfactory segmentation results and, simultaneously, maintaining real-time performance. In this article, we present a novel lightweight context-aware network, namely, PolypSeg+, attempting to capture distinguishable features of polyps without increasing network complexity and sacrificing time performance. To achieve this, a set of novel lightweight techniques is developed and integrated into the proposed PolypSeg+, including an adaptive scale context (ASC) module equipped with a lightweight attention mechanism to tackle the large-scale variation of polyps, an efficient global context (EGC) module to promote the fusion of low-level and high-level features by excluding background noise and preserving boundary details, and a lightweight feature pyramid fusion (FPF) module to further refine the features extracted from the ASC and EGC. We extensively evaluate the proposed PolypSeg+ on two famous public available datasets for the polyp segmentation task: 1) Kvasir-SEG and 2) CVC-Endoscenestill. The experimental results demonstrate that our PolypSeg+ consistently outperforms other state-of-the-art networks by achieving better segmentation accuracy in much less running time. The code is available at https://github.com/szu-zzb/polypsegplus .

Existing federated learning works mainly focus on the fully supervised training setting. In realistic scenarios, however, most clinical sites can only provide data without annotations due to the lack of resources or expertise. In this work, we are concerned with the practical yet challenging federated semi-supervised segmentation (FSSS) , where labeled data are only with several clients and other clients can just provide unlabeled data. We take an early attempt to tackle this problem and propose a novel FSSS method with prototype-based pseudo-labeling and contrastive learning. First, we transmit a labeled-aggregated model, which is obtained based on prototype similarity, to each unlabeled client, to work together with the global model for debiased pseudo labels generation via a consistency- and entropy-aware selection strategy. Second, we transfer image-level prototypes from labeled datasets to unlabeled clients and conduct prototypical contrastive learning on unlabeled models to enhance their discriminative power. Finally, we perform the dynamic model aggregation with a designed consistency-aware aggregation strategy to dynamically adjust the aggregation weights of each local model. We evaluate our method on COVID-19 X-ray infected region segmentation, COVID-19 CT infected region segmentation and colorectal polyp segmentation, and experimental results consistently demonstrate the effectiveness of our proposed method. Codes will be released upon publication.