21 July 2023 in IEEE Transactions on Neural Networks and Learning Systems
Since labeled samples are typically scarce in real-world scenarios, self-supervised representation learning in time series is critical. Existing approaches mainly employ the contrastive learning framework, which automatically learns to understand similar and dissimilar data pairs. However, they are constrained by the request for cumbersome sampling policies and prior knowledge of constructing pairs. Also, few works have focused on effectively modeling temporal-spectral correlations to improve the capacity of representations. In this article, we propose the cross reconstruction transformer (CRT) to solve the aforementioned issues. CRT achieves time series representation learning through a cross-domain dropping-reconstruction task. Specifically, we obtain the frequency domain of the time series via the fast Fourier transform (FFT) and randomly drop certain patches in both time and frequency domains. Dropping is employed to maximally preserve the global context while masking leads to the distribution shift. Then a Transformer architecture is utilized to adequately discover the cross-domain correlations between temporal and spectral information through reconstructing data in both domains, which is called Dropped Temporal-Spectral Modeling. To discriminate the representations in global latent space, we propose instance discrimination constraint (IDC) to reduce the mutual information between different time series samples and sharpen the decision boundaries. Additionally, a specified curriculum learning (CL) strategy is employed to improve the robustness during the pretraining phase, which progressively increases the dropping ratio in the training process. We conduct extensive experiments to evaluate the effectiveness of the proposed method on multiple real-world datasets. Results show that CRT consistently achieves the best performance over existing methods by 2%–9%. The code is publicly available at https://github.com/BobZwr/Cross-Reconstruction-Transformer .
23 September 2022 in Biomedical Signal Processing and Control
16 November 2022 in BMC Medical Informatics and Decision Making
Background: Critical values are commonly used in clinical laboratory tests to define health-related conditions of varying degrees. Knowing the values, people can quickly become aware of health risks, and the health professionals can take immediate actions and save lives. Methods: In this paper, we propose a method that extends the concept of critical value to one of the most commonly used physiological signals in the clinical environment—Electrocardiogram (ECG). We first construct a mapping from common ECG diagnostic conclusions to critical values. After that, we build a 61-layer deep convolutional neural network named CardioV, which is characterized by an ordinal classifier. Results: We conduct experiments on a large public ECG dataset, and demonstrate that CardioV achieves a mean absolute error of 0.4984 and a ROC-AUC score of 0.8735. In addition, we find that the model performs better for extreme critical values and the younger age group, while gender does not affect the performance. The ablation study confirms that the ordinal classification mechanism suits for estimating the critical values which contain ranking information. Moreover, model interpretation techniques help us discover that CardioV focuses on the characteristic ECG locations during the critical value estimation process. Conclusions: As an ordinal classifier, CardioV performs well in estimating ECG critical values that can help people quickly identify different heart conditions. We obtain ROC-AUC scores above 0.8 for all four critical value categories, and find that the extreme values (0 (no risk) and 3 (high risk)) have better model performance than the other two (1 (low risk) and 2 (medium risk)). Results also show that gender does not affect the performance, and the older age group has worse performance than the younger age group. In addition, visualization techniques reveal that the model pays more attention to characteristic ECG locations.
9 December 2023 in Neural Computing and Applications