Sleep staging is an indispensable indicator for measuring sleep quality and evaluating sleep disorders. Deep learning methods have been successfully applied to automatic sleep staging (ASS) based on EEG signals, achieving significant progress. However, previous studies have been limited to extracting local features of EEG signals while neglecting the importance of global features. To solve this problem, we propose a novel ASS model named LGSleepNet, which consists of asymmetric Siamese neural network (ASNN), Deep adaptive orthogonal fusion (DAOF) block, and weighted polynomial cross entropy (WPCE) loss function. Specifically, the ASNN is capable of simultaneously extracting local and global features from EEG signals, which provides diverse semantic representations for sleep staging. Moreover, a DAOF block is proposed to eliminate the information redundancy and semantic deviation among heterogeneous features by orthogonalization and adaptive fusion, which strengthens the correlation representation between local and global features. Ultimately, a weighted polynomial cross entropy (WPCE) loss function is designed to improve the decision-making ability of the classification head and alleviate the problem of sample imbalance. We evaluate the LGSleepNet on three publicly available datasets, namely Sleep-EDF-20, Sleep-EDF-78, and SVUH-UCD, which achieves macro F1-scores of 80.7%, 76.0%, and 75.1% and overall accuracy of 86.0%, 82.3%, and 76.3%, respectively. The experimental results indicate that the LGSleepNet performs at an advanced level compared to other state-of-the-art methods.

With the rapid growth of medical imaging data, radiologists must dedicate a significant amount of time to report writing. Automated generation of radiology reports not only alleviates the heavy workload of physicians but, more importantly, can reduce mistakes and oversights caused by insufficient experience. However, due to substantial data bias in medical data, prior studies using typical cross-entropy loss in encoder-decoder architectures often result in generalized descriptions of normal tissues and may overlook crucial clinical abnormalities. Therefore, we propose a clustering enhanced contrastive learning model named CECL to generate more diverse radiology reports, and it is worth noting that our method is end-to-end trainable. Specifically, an adaptive alignment fusion encoder-decoder network (AAF) is constructed by fusing the image features and text semantic features from the transformer decoder, eliminating information redundancy across different modalities. Moreover, a label-guided contrastive learning (LCL) module is proposed. In detail, clustering is performed on the fused features using Gaussian competition. Supervised contrastive learning is conducted based on the clustering results to enhance feature representation ability. We evaluate the CECL on two widely used publicly available datasets, IU X-ray and MIMIC-CXR, using NLG and CE metrics. The experimental results demonstrate that CECL can produce fluent reports with more descriptions of anomalies, outperforming other state-of-the-art methods with higher clinical correctness.

Multimodal sentiment analysis improves the understanding of human emotions by integrating multiple modalities. However, the language modality accessed by the actually deployed MSA systems is obtained through automatic speech recognition (ASR), which can induce text errors and lead to MSA performance degradation. Besides, multimodal heterogeneity and multimodal information imbalance further aggravate the negative impact of ASR errors. To this end, we propose a multi-granularity contrastive coordination framework (MGCCF) for achieving robust MSA under ASR errors. Specifically, an error sentiment awareness calibration (ESAC) unit is devised to learn cross-modal word-level correlation and contextual information, enabling the detection and calibration of erroneous sentiment word embedding. Furthermore, a multimodal contrastive coordination (MCC) paradigm is presented to achieve cross-modal multi-granularity alignment from both global semantic and local temporal aspects. Meanwhile, MCC ingeniously leverages non-verbal features to reinforce the ability of ESAC to correct anomalous sentiment information and refine the expression of language features. Eventually, a multimodal information fusion (MIF) approach is developed to deal with multimodal information imbalance, which interpretably quantifies the contributions of different modalities and achieves multimodal dynamic fusion. Comprehensive experiments on seven MSA datasets demonstrate that our MGCCF achieves remarkable performance and surpasses the current state-of-the-art MSA methods.