Wenjun Sun

Wenjun Sun

Auto-generated profile, based on publicly available information

Wenjun Sun

School of Instrument Science and Engineering, Southeast University, Nanjing, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
24 October 2023 in IEEE Transactions on Reliability
Verified
Journal Article
Closed access
WSWenjun Sun
Wenjun Sun
RYRuqiang Yan
Ruqiang Yan
RJRuibing Jin
Ruibing Jin
JXJiawen Xu
Jiawen Xu
Yuan YangYuan Yang
Yuan Yang
Zhenghua ChenZhenghua Chen
Zhenghua Chen
Transformer has shown impressive performance on global feature modeling in many applications. However, two drawbacks induced by its intrinsic architecture limit its application, especially in fault diagnosis. First, the quadratic complexity of its self-attention scheme extremely increases the computation cost, which poses a challenge to apply Transformer to a computationally limited platform like an industry system. In addition, the sequence-based modeling in the Transformer increases the training difficulty and requires a large-scale training dataset. This drawback becomes serious when Transformer is applied in fault diagnosis where only limited data is available. To mitigate these issues, we rethink this common approach and propose a new Transformer, which is more suitable for fault diagnosis. In this article, we first show that the attention module can be actually replaced with or even surpassed by a convolution layer under some conditions in mathematics and experiments. Then, we adopt the convolutions into the Transformer, where the computation burden issue is alleviated and the fault classification accuracy is significantly improved. Furthermore, to increase the computation efficiency, a lightweight Transformer called LiteFormer, is developed by utilizing the depth-wise convolutional layer. Extensive experiments are carried out on four datasets: Case Western Reserve University dataset; Paderborn University dataset; and two gearbox datasets of drivetrain dynamic simulator. Through our experiments, our LiteFormer not only reduces the computation cost in model training, but also sets new state-of-the-art results, surpassing other counterparts in both fault classification accuracy and model robustness.

22 December 2023 in IEEE Transactions on Instrumentation and Measurement
Verified
Journal Article
Closed access
WSWenjun Sun
Wenjun Sun
RYRuqiang Yan
Ruqiang Yan
RJRuibing Jin
Ruibing Jin
RZRui Zhao
Rui Zhao
Zhenghua ChenZhenghua Chen
Zhenghua Chen
Due to privacy issues, the data island problem of machine fault diagnosis widely exists in real industry. Federated Learning (FL) has received much attention as a decentralized machine-learning paradigm that learns a global model in the server by iteratively aggregating the local model parameters in a privacy-preserving scheme to address the data island problem. However, the fault data in industry is inevitably scarce and class imbalanced, so that the fault data in different clients are heterogeneous from each other in FL. The existing federated machine fault diagnosis methods are mostly based on the federated averaging (FedAvg) algorithm, which ignore the data heterogeneity issue for machine fault diagnosis. To address this issue for machine fault diagnosis, we propose a new model alignment method (called FedAlign) via data-free knowledge distillation, where a compact generator is trained in the server in a data-free manner to estimate the data feature distribution in a global view without accessing to the local data. Then the generator can produce the pseudo features to convey the distribution knowledge to both the server and the clients sides. Our proposed FedAlign utilizes the pseudo features to align the global model and the local models to finally address the inherent heterogeneity of local data for fault diagnosis in FL. Experiments performed on fault diagnosis datasets of non-iid settings indicate that our proposed method facilitates FL for machine fault diagnosis with favorable effectiveness and achieves significant performance gains compared with state-of-the-art methods.

28 August 2024 in IEEE Transactions on Industrial Informatics
Verified
Journal Article
Closed access
WSWenjun Sun
Wenjun Sun
RYRuqiang Yan
Ruqiang Yan
RJRuibing Jin
Ruibing Jin
RZRui Zhao
Rui Zhao
Zhenghua ChenZhenghua Chen
Zhenghua Chen
Federated learning (FL) has emerged as an effective machine-learning paradigm for collaborative machine fault diagnosis in a privacy-preserving scheme. However, due to the perception limitation and different annotation criteria of annotators, the data in clients may have noisy labels with varied noise levels, leading to degraded FL performances. Most existing methods in FL for tackling the label noise issue, assume that there is label noise in all clients and treat all clients with the same denoising training. However, these methods may result in sub-optimization and even training instability of local models, so that they cannot perform well on heterogeneous label noise across clients in FL. To address this issue, we propose a curriculum-based federated learning (called FedCNL) method to combat the heterogeneous label noise in FL settings. First, our proposed FedCNL exploits a noise modeling module to adaptively estimate the clean clients and noisy clients, and identify the clean samples and noisy samples in noisy clients in an unsupervised manner. Then, a multi-stage curriculum learning is designed by regarding the noise level as learning complexity, where the model learns from clean to noisy samples, gradually improving the performance of the global model. Moreover, a mixed loss correction method is explored in the curriculum stage to maximize the utilization of data with noisy labels. Experiments performed on fault datasets in non-identically and independently distributed settings indicate that our proposed method addresses the label noise issue for machine fault diagnosis in heterogeneous FL with favorable effectiveness, achieving state-of-the-art performances.

13 July 2022 in IEEE Open Journal of Instrumentation and Measurement
Verified
Journal Article
Open access
WSWenjun Sun
Wenjun Sun
HWHui Wang
Hui Wang
JXJiawen Xu
Jiawen Xu
Yuan YangYuan Yang
Yuan Yang
RYRuqiang Yan
Ruqiang Yan
To extract the global temporal correlations and local features together to enhance the accuracy for fault diagnosis, this paper proposes an effective convolutional Transformer (ECT), which can learn the global temporal correlations using Transformer and local features with convolution at the same time. The proposed method designs a multi-stage hierarchical structure of Transformer, which utilizes convolutional tokenization to distill dominating sequence features from raw vibration signals while increasing the dimension of token embedding across stages at the same time as that in CNNs. The spatial-reduction attention (SRA) and the linear dimension reduction projections are introduced respectively to Transformer at different stages to reduce the resource consumption of the model. Finally, the proposed method utilizes a sequence pooling strategy on the output of Transformer to eliminate the requirement of the class token and make the model accurate for classification. The specially designed structure makes the model flexible and effective for planetary gearbox fault diagnosis. Experiments performed on planetary gearbox fault simulators indicate that the ECT method has significant effectiveness and high accuracy compared with the state-of-the-art methods for planetary gearbox fault diagnosis.