1 October 2024 in IEEE Journal of Biomedical and Health Informatics
Diffusion tensor imaging (DTI) is a prevalent magnetic resonance imaging (MRI) technique, widely used in clinical and neuroscience research. However, the reliability of DTI is affected by the low signal-to-noise ratio inherent in diffusion-weighted (DW) images. Deep learning (DL) has shown promise in improving the quality of DTI, but its limited generalization to variable acquisition schemes hinders practical applications. This study aims to develop a generalized, accurate, and efficient DL-based DTI method. By leveraging the representation of voxel-wise diffusion MRI (dMRI) signals on the sphere using spherical harmonics (SH), we propose a novel approach that utilizes SH coefficient maps as input to a network for predicting the diffusion tensor (DT) field, enabling improved generalization. Extensive experiments were conducted on simulated and in-vivo datasets, covering various DTI application scenarios. The results demonstrate that the proposed SH-DTI method achieves advanced performance in both quantitative and qualitative analyses of DTI. Moreover, it exhibits remarkable generalization capabilities across different acquisition schemes, centers, and scanners, ensuring its broad applicability in diverse settings.
9 November 2023 in Journal of Magnetic Resonance Imaging
21 March 2024 in Medical Image Analysis
10 March 2026 in Nature Communications
Deep learning has demonstrated remarkable abilities in restoring fluorescence microscopy images degraded by noise, blur, or undersampling. However, most existing models are task-specific and trained on limited, homogeneously distributed data, which restricts their generalizability and practicality. Here, we present FluoResFM, a foundation model for multi-task and cross-distribution fluorescence microscopy image restoration in a unified model. FluoResFM leverages textual prior information to adapt to specific tasks and data distributions. Trained on datasets across three tasks (image denoising, deconvolution, and super-resolution) and over 20 biological structures, FluoResFM demonstrates superior restoration performance and enhanced generalization across datasets with varied biological structures and imaging conditions. Through fine-tuning with only a single sample, FluoResFM can further improve its performance on unseen data, achieving results comparable to conventional models trained on hundreds of samples, and be easily adapted to additional tasks, including 3D image restoration, surface projection, isotropic reconstruction, and super-resolution with various scale factors. Moreover, the performance of existing cell/organelle segmentation models can be enhanced using the high-quality images restored by FluoResFM.
5 September 2025 in Research Square
Deep learning-based methods have demonstrated remarkable abilities in restoring high-quality fluorescence microscopy images from those degraded by noise, blur, or undersampling. However, most existing deep networks are task-specific and trained on limited, homogeneously distributed data, which restricts their generalizability and practicality in biological research. Here, we present FluoResFM, a foundation model designed for multi-task and cross-distribution fluorescence microscopy image restoration in a unified model. FluoResFM leverages textual prior information (i.e., task type, imaging object, and imaging condition) to adapt the model to specific task and data distribution. It was trained using more than 4.3 million paired patches across three tasks (image denoising, deconvolution, and super-resolution) and over 20 biological structure types. FluoResFM demonstrates superior restoration performance and enhanced generalization, delivering high-fidelity reconstruction results across the three tasks and diverse internal and unseen external datasets encompassing varied biological structures and imaging conditions. Leveraging its strong generalization capability, FluoResFM can further improve its performance on unseen data through fine-tuning with only a single sample, achieving results comparable to those of conventional deep networks trained on hundreds of samples. Furthermore, the performance of existing cell/organelle segmentation models can be further improved using the high-quality image restored by FluoResFM. To make FluoResFM widely accessible to the biology research community, we developed a user-friendly napari plugin. These establish FluoResFM as a versatile foundation model for fluorescence microscopy image processing and analysis.