Insufficient annotated data and minor lung lesions pose big challenges for computed tomography (CT)-aided automatic COVID-19 diagnosis at an early outbreak stage. To address this issue, we propose a Semi-Supervised Tri-Branch Network (SS-TBN). First, we develop a joint TBN model for dual-task application scenarios of image segmentation and classification such as CT-based COVID-19 diagnosis, in which pixel-level lesion segmentation and slice-level infection classification branches are simultaneously trained via lesion attention, and individual-level diagnosis branch aggregates slice-level outputs for COVID-19 screening. Second, we propose a novel hybrid semi-supervised learning method to make full use of unlabeled data, combining a new double-threshold pseudo labeling method specifically designed to the joint model and a new inter-slice consistency regularization method specifically tailored to CT images. Besides two publicly available external datasets, we collect internal and our own external datasets including 210,395 images (1,420 cases versus 498 controls) from ten hospitals. Experimental results show that the proposed method achieves state-of-the-art performance in COVID-19 classification with limited annotated data even if lesions are subtle, and that segmentation results promote interpretability for diagnosis, suggesting the potential of the SS-TBN in early screening in insufficient labeled data situations at the early stage of a pandemic outbreak like COVID-19.

Automatic pill recognition (APR) systems are crucial for enhancing hospital efficiency, assisting visually impaired individuals, and preventing cross-infection. However, most existing deep learning-based pill recognition systems can only perform classification on classes with sufficient training data. In practice, the high cost of data annotation and the continuous increase in new pill classes necessitate the development of a few-shot class-incremental pill recognition (FSCIPR) system. This article introduces the first FSCIPR framework, discriminative and bidirectional compatible few-shot class-incremental learning (DBC-FSCIL). It encompasses forward-compatible and backward-compatible learning components. In forward-compatible learning, we propose an innovative virtual class generation strategy and a center–triplet (CT) loss to enhance discriminative feature learning. These virtual classes serve as placeholders in the feature space for future class updates, providing diverse semantic knowledge for model training. For backward-compatible learning, we develop a strategy to synthesize reliable pseudo-features of old classes using uncertainty quantification, facilitating data replay (DR) and knowledge distillation (KD). This approach allows for the flexible synthesis of features and effectively reduces additional storage requirements for samples and models. Additionally, we construct a new pill image dataset for FSCIL and assess various mainstream FSCIL methods, establishing new benchmarks. Our experimental results demonstrate that our framework surpasses existing state-of-the-art (SOTA) methods.

Limited sample size hinders the application of deep learning in brain image analysis, and transfer learning is a possible solution. However, most pretrained models are 2D based and cannot be applied directly to 3D brain images. In this study, we propose a novel framework to apply 2D pretrained models to 3D brain images by projecting surface-based cortical morphometry into planar images using computational geometry mapping. Firstly, 3D cortical meshes are reconstructed from magnetic resonance imaging (MRI) using FreeSurfer and projected into 2D planar meshes with topological preservation based on area-preserving geometry mapping. Then, 2D deep models pretrained on ImageNet are adopted and fine-tuned for cortical image classification on morphometric shape metrics. We apply the framework to sex classification on the Human Connectome Project dataset and autism spectrum disorder (ASD) classification on the Autism Brain Imaging Data Exchange dataset. Moreover, a 2-stage transfer learning strategy is suggested to boost the ASD classification performance by using the sex classification as an intermediate task. Our framework brings significant improvement in sex classification and ASD classification with transfer learning. In summary, the proposed framework builds a bridge between 3D cortical data and 2D models, making 2D pretrained models available for brain image analysis in cognitive and psychiatric neuroscience.