Depression is a major psychological disorder with a growing impact worldwide. Traditional methods for detecting the risk of depression, predominantly reliant on psychiatric evaluations and self-assessment questionnaires, are often criticized for their inefficiency and lack of objectivity. Advancements in deep learning have paved the way for innovations in depression risk detection methods that fuse multimodal data. This paper introduces a novel framework, the Audio, Video, and Text Fusion-Three Branch Network (AVTF-TBN), designed to amalgamate auditory, visual, and textual cues for a comprehensive analysis of depression risk. Our approach encompasses three dedicated branches—Audio Branch, Video Branch, and Text Branch—each responsible for extracting salient features from the corresponding modality. These features are subsequently fused through a multimodal fusion (MMF) module, yielding a robust feature vector that feeds into a predictive modeling layer. To further our research, we devised an emotion elicitation paradigm based on two distinct tasks—reading and interviewing—implemented to gather a rich, sensor-based depression risk detection dataset. The sensory equipment, such as cameras, captures subtle facial expressions and vocal characteristics essential for our analysis. The research thoroughly investigates the data generated by varying emotional stimuli and evaluates the contribution of different tasks to emotion evocation. During the experiment, the AVTF-TBN model has the best performance when the data from the two tasks are simultaneously used for detection, where the F1 Score is 0.78, Precision is 0.76, and Recall is 0.81. Our experimental results confirm the validity of the paradigm and demonstrate the efficacy of the AVTF-TBN model in detecting depression risk, showcasing the crucial role of sensor-based data in mental health detection.

Background: Microvascular invasion (MVI) has a significant effect on the prognosis of hepatocellular carcinoma (HCC), but its preoperative identification is challenging. Radiomics features extracted from medical images, such as magnetic resonance (MR) images, can be used to predict MVI. In this study, we explored the effects of different imaging sequences, feature extraction and selection methods, and classifiers on the performance of HCC MVI predictive models. Methods: After screening against the inclusion criteria, 69 patients with HCC and preoperative gadoxetic acid-enhanced MR images were enrolled. In total, 167 features were extracted from the MR images of each sequence for each patient. Experiments were designed to investigate the effects of imaging sequence, number of gray levels (Ng), quantization algorithm, feature selection method, and classifiers on the performance of radiomics biomarkers in the prediction of HCC MVI. We trained and tested these models using leave-one-out cross-validation (LOOCV). Results: The radiomics model based on the images of the hepatobiliary phase (HBP) had better predictive performance than those based on the arterial phase (AP), portal venous phase (PVP), and pre-enhanced T1-weighted images [area under the receiver operating characteristic (ROC) curve (AUC) =0.792 vs. 0.641/0.634/0.620, P=0.041/0.021/0.010, respectively]. Compared with the equal-probability and Lloyd-Max algorithms, the radiomics features obtained using the Uniform quantization algorithm had a better performance (AUC =0.643/0.666 vs. 0.792, P=0.002/0.003, respectively). Among the values of 8, 16, 32, 64, and 128, the best predictive performance was achieved when the Ng was 64 (AUC =0.792 vs. 0.584/0.697/0.677/0.734, P<0.001/P=0.039/0.001/0.137, respectively). We used a two-stage feature selection method which combined the least absolute shrinkage and selection operator (LASSO) and recursive feature elimination (RFE) gradient boosting decision tree (GBDT), which achieved better stability than and outperformed LASSO, minimum redundancy maximum relevance (mRMR), and support vector machine (SVM)-RFE (stability =0.967 vs. 0.837/0.623/0.390, respectively; AUC =0.850 vs. 0.792/0.713/0.699, P=0.142/0.007/0.003, respectively). The model based on the radiomics features of HBP images using the GBDT classifier showed a better performance for the preoperative prediction of MVI compared with logistic regression (LR), SVM, and random forest (RF) classifiers (AUC =0.895 vs. 0.850/0.834/0.884, P=0.558/0.229/0.058, respectively). With the optimal combination of these factors, we established the best model, which had an AUC of 0.895, accuracy of 87.0%, specificity of 82.5%, and sensitivity of 93.1%. Conclusions: Imaging sequences, feature extraction and selection methods, and classifiers can have a considerable effect on the predictive performance of radiomics models for HCC MVI.

Adequate medical images are often indispensable in contemporary deep learning-based medical imaging studies, although the acquisition of certain image modalities may be limited due to several issues including high costs, patients issues, etc. However, thanks to recent advances in deep learning techniques, the above tough problem can be substantially alleviated by medical images synthesis, by which various modalities including T1 / T2 / DTI MRI images, PET images, cardiac ultrasound images, retinal images, etc., have already been synthesized. Unfortunately, the arterial spin labeling (ASL) image, which is an important fMRI indicator in dementia diseases diagnosis nowadays, has never been comprehensively investigated for the synthesis purpose yet. In this study, ASL images have been successfully synthesized from structural magnetic resonance images for the first time. Technically, a novel unbalanced deep discriminant learning-based model equipped with new ResNet sub-structures is proposed to realize the synthesis of ASL images from structural magnetic resonance images. Extensive experiments have been conducted. Comprehensive statistical analyses reveal that: 1) this newly introduced model is capable to synthesize ASL images that are similar towards real ones acquired by actual scanning; 2) synthesized ASL images obtained by the new model have demonstrated outstanding performance when undergoing rigorous tests of region-based and voxel-based corrections of partial volume effects, which are essential in ASL images processing; 3) it is also promising that the diagnosis performance of dementia diseases can be significantly improved with the help of synthesized ASL images obtained by the new model, based on a multi-modal MRI dataset containing 355 demented patients in this study.

Unilateral peripheral facial paralysis (UPFP) is a form of facial nerve paralysis and clinically classified according to conditions of facial symmetry. Prompt and precise assessment is crucial to neural rehabilitation of UPFP. The prevalent House-Brackmann (HB) grading system relies on subjective judgments with significant interobservation variation. Therefore, to explore an objective method for the UPFP assessment, clinical image sequences are captured using a web camera setup while 5 healthy and 27 UPFP subjects perform a group of predefined actions, including keeping expressionless, raising brows, closing eyes, bulging cheek, and showing teeth in turn. First, facial region is decided using Haar cascade classifier, and then landmark points are acquired by a supervised descent method. Second, these landmark points are used to generate a group of features reflecting the structural parameters of regions of eyebrows, eyes, nose, and mouth, respectively. Third, correlation coefficients are computed between the raw features HB scores. To reduce feature dimensions, only those with correlation coefficients larger than an empirically selected value, 0.35, are input into a support vector machine to generate a classifier. With the classifier, exact match (discrepancy = 0 between result from proposed method and HB scores) rate at 49.9%, and loose match (discrepancy = 1) rate at 87.97% are achieved on the experiment data. After sample augmentation, the final rate is increased to 90.01%, outperformed previous reports. In conclusion, it is demonstrated with an unobtrusive web camera setup, encouraging results have been generated with the proposed framework in this exploratory study.