16 August 2019 in Frontiers in Neuroscience
Gliomas are the most common primary brain malignancies. Accurate and robust tumor segmentation and prediction of patients' overall survival are important for diagnosis, treatment planning and risk factor identification. Here we present a deep learning-based framework for brain tumor segmentation and survival prediction in glioma, using multimodal MRI scans. For tumor segmentation, we use ensembles of three different 3D CNN architectures for robust performance through a majority rule. This approach can effectively reduce model bias and boost performance. For survival prediction, we extract 4,524 radiomic features from segmented tumor regions, then, a decision tree and cross validation are used to select potent features. Finally, a random forest model is trained to predict the overall survival of patients. The 2018 MICCAI Multimodal Brain Tumor Segmentation Challenge (BraTS), ranks our method at 2nd and 5th place out of 60+ participating teams for survival prediction tasks and segmentation tasks respectively, achieving a promising 61.0% accuracy on the classification of short-survivors, mid-survivors and long-survivors.
26 June 2023 in Proceedings of the AAAI Conference on Artificial Intelligence
Continual Test-Time Adaptation (CTTA) aims to adapt the source model to continually changing unlabeled target domains without access to the source data. Existing methods mainly focus on model-based adaptation in a self-training manner, such as predicting pseudo labels for new domain datasets. Since pseudo labels are noisy and unreliable, these methods suffer from catastrophic forgetting and error accumulation when dealing with dynamic data distributions. Motivated by the prompt learning in NLP, in this paper, we propose to learn an image-layer visual domain prompt for target domains while having the source model parameters frozen. During testing, the changing target datasets can be adapted to the source model by reformulating the input data with the learned visual prompts. Specifically, we devise two types of prompts, i.e., domains-specific prompts and domains-agnostic prompts, to extract current domain knowledge and maintain the domain-shared knowledge in the continual adaptation. Furthermore, we design a homeostasis-based adaptation strategy to suppress domain-sensitive parameters in domain-invariant prompts to learn domain-shared knowledge more effectively. This transition from the model-dependent paradigm to the model-free one enables us to bypass the catastrophic forgetting and error accumulation problems. Experiments show that our proposed method achieves significant performance gains over state-of-the-art methods on four widely-used benchmarks, including CIFAR-10C, CIFAR-100C, ImageNet-C, and VLCS datasets.
27 May 2025 in Nature Communications
Tuneable retarder arrays, such as spatially patterned liquid crystal devices, have given rise to impressive photonic functionality, fuelling diverse applications ranging from microscopy and holography to encryption and communications. Presently these solutions are limited by the controllable degrees of freedom of structured matter, hindering applications that demand photonic systems with high flexibility and reconfigurable topologies. Here we demonstrate a compound modulator that implements a synthetic tuneable arbitrary retarder array as virtual pixels derived by cascading low functionality tuneable devices, realising full dynamic control of its arbitrary elliptical axis geometry, retardance value, and induced phase. Our approach offers unprecedented functionality that is user-defined and possesses high flexibility, allowing our modulator to act as a new beam generator, analyser, and corrector, opening an exciting path to tuneable topologies of light and matter.
26 January 2023 in Frontiers in Medicine
Pathology is the gold standard of clinical diagnosis. Artificial intelligence (AI) in pathology becomes a new trend, but it is still not widely used due to the lack of necessary explanations for pathologists to understand the rationale. Clinic-compliant explanations besides the diagnostic decision of pathological images are essential for AI model training to provide diagnostic suggestions assisting pathologists practice. In this study, we propose a new annotation form, PathNarratives, that includes a hierarchical decision-to-reason data structure, a narrative annotation process, and a multimodal interactive annotation tool. Following PathNarratives, we recruited 8 pathologist annotators to build a colorectal pathological dataset, CR-PathNarratives, containing 174 whole-slide images (WSIs). We further experiment on the dataset with classification and captioning tasks to explore the clinical scenarios of human-AI-collaborative pathological diagnosis. The classification tasks show that fine-grain prediction enhances the overall classification accuracy from 79.56 to 85.26%. In Human-AI collaboration experience, the trust and confidence scores from 8 pathologists raised from 3.88 to 4.63 with providing more details. Results show that the classification and captioning tasks achieve better results with reason labels, provide explainable clues for doctors to understand and make the final decision and thus can support a better experience of human-AI collaboration in pathological diagnosis. In the future, we plan to optimize the tools for the annotation process, and expand the datasets with more WSIs and covering more pathological domains.
3 October 2025 in Science Advances
With the growing use of optical polarization in applications ranging from communications to medical diagnoses, adaptive correction of complex vectorial aberrations in optical systems has become an increasingly important area of research. However, research to date has focused primarily on phase and retardance aberrations, whereas another major source of aberration—diattenuation—remains largely unexplored. Unlike the others, diattenuation affects intensity in addition to phase and polarization, limiting the intrinsic correction capability of adaptive systems. In this work, we propose the use of optical skyrmions to probe diattenuation-aberrated systems and provide metrics that characterize the performance of vectorial adaptive optics (V-AO), with theoretical and experimental validations. Based on the probed results, we demonstrate V-AO correction under real-world aberrations for complex media imaging and analyze correction strategies to optimize measurements in aberrated polarimetric systems. This work paves the way for high-dimensional aberration correction, introduces a previously unidentified use of optical skyrmions, and provides insights that will aid the development of vectorial measurement systems.