Brain tumors can appear anywhere in the brain and have vastly different sizes and morphology. Additionally, these tumors are often diffused and poorly contrasted. Consequently, the segmentation of brain tumor and intratumor subregions using magnetic resonance imaging (MRI) data with minimal human interventions remains a challenging task. In this paper, we present a novel fully automatic segmentation method from MRI data containing in vivo brain gliomas. This approach can not only localize the entire tumor region but can also accurately segment the intratumor structure. The proposed work was based on a cascaded deep learning convolutional neural network consisting of two subnetworks: (1) a tumor localization network (TLN) and (2) an intratumor classification network (ITCN). The TLN, a fully convolutional network (FCN) in conjunction with the transfer learning technology, was used to first process MRI data. The goal of the first subnetwork was to define the tumor region from an MRI slice. Then, the ITCN was used to label the defined tumor region into multiple subregions. Particularly, ITCN exploited a convolutional neural network (CNN) with deeper architecture and smaller kernel. The proposed approach was validated on multimodal brain tumor segmentation (BRATS 2015) datasets, which contain 220 high-grade glioma (HGG) and 54 low-grade glioma (LGG) cases. Dice similarity coefficient (DSC), positive predictive value (PPV), and sensitivity were used as evaluation metrics. Our experimental results indicated that our method could obtain the promising segmentation results and had a faster segmentation speed. More specifically, the proposed method obtained comparable and overall better DSC values (0.89, 0.77, and 0.80) on the combined (HGG + LGG) testing set, as compared to other methods reported in the literature. Additionally, the proposed approach was able to complete a segmentation task at a rate of 1.54 seconds per slice.

Intelligent medicine is eager to automatically generate radiology reports to ease the tedious work of radiologists. Previous researches mainly focused on the text generation with encoder-decoder structure, while CNN networks for visual features ignored the long-range dependencies correlated with textual information. Besides, few studies exploit cross-modal mappings to promote radiology report generation. To alleviate the above problems, we propose a novel end-to-end radiology report generation model dubbed Self-Supervised dual-Stream Network (S3-Net). Specifically, a Dual-Stream Visual Feature Extractor (DSVFE) composed of ResNet and SwinTransformer is proposed to capture more abundant and effective visual features, where the former focuses on local response and the latter explores long-range dependencies. Then, we introduced the Fusion Alignment Module (FAM) to fuse the dual-stream visual features and facilitate alignment between visual features and text features. Furthermore, the Self-Supervised Learning with Mask(SSLM) is introduced to further enhance the visual feature representation ability. Experimental results on two mainstream radiology reporting datasets (IU X-ray and MIMIC-CXR) show that our proposed approach outperforms previous models in terms of language generation metrics.

Medical imaging is widely used in hospital clinical workflows. Assisting physicians in diagnosis by automatically generating reports from radiological images is an unmet clinical demand and requires urgent attention. However, this task suffers from two significant problems: (1) visual and textual data biases, and (2) the Transformer decoder makes no distinction between visual and non-visual words. We propose a novel multi-task approach combining natural language processing with machine learning techniques to meet this clinical need, i.e., creating fluent and accurate radiology reports. We name our system as Multi-modal Adaptive Transformer (MATNet), which consists of three key modules. First, Multi-Modal Encoder (MME) explores the relationship between radiology images and clinical notes. Second, Disease Classifier (DC) classifies the states of each disease topic and provides state-aware disease embeddings to alleviate visual data bias. Last, Adaptive Decoder (AD) dynamically measures the contribution of source signals and target signals when generating the next word. Based on our evaluations using benchmark IU-XRay and MIMIC-CXR datasets, the proposed MATNet outperformed previous state-of-the-art models on language fluency and clinical accuracy metrics such as BLEU scores.

Hemorrhagic complication (HC) is the most severe complication of intravenous thrombolysis (IVT) in patients with acute ischemic stroke (AIS). This study aimed to build a machine learning (ML) prediction model and an application system for a personalized analysis of the risk of HC in patients undergoing IVT therapy. We included patients from Chongqing, Hainan and other centers, including Computed Tomography (CT) images, demographics, and other data, before the occurrence of HC. After feature engineering, a better feature subset was obtained, which was used to build a machine learning (ML) prediction model (Logistic Regression (LR), Random Forest (RF), Support Vector Machine (SVM), eXtreme Gradient Boosting (XGB)), and then evaluated with relevant indicators. Finally, a prediction model with better performance was obtained. Based on this, an application system was built using the Flask framework. A total of 517 patients were included, of which 332 were in the training cohort, 83 were in the internal validation cohort, and 102 were in the external validation cohort. After evaluation, the performance of the XGB model is better, with an AUC of 0.9454 and ACC of 0.8554 on the internal validation cohort, and 0.9142 and ACC of 0.8431 on the external validation cohort. A total of 18 features were used to construct the model, including hemoglobin and fasting blood sugar. Furthermore, the validity of the model is demonstrated through decision curves. Subsequently, a system prototype is developed to verify the test prediction effect. The clinical decision support system (CDSS) embedded with the XGB model based on clinical data and image features can better carry out personalized analysis of the risk of HC in intravenous injection patients.