Jian Zhao

Jian Zhao

Auto-generated profile, based on publicly available information

Jian Zhao

Regional medical center for National Institute of Respiratory Disease, Sir Run Run Shaw Hospital, Hangzhou, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 February 2024 in IEEE Internet of Things Journal
Verified
Journal Article
Closed access
NYNa Ying
Na Ying
YJYinhe Jiang
Yinhe Jiang
CGChunsheng Guo
Chunsheng Guo
DZDi Zhou
Di Zhou
JZJian Zhao
Jian Zhao
Driving can take up a substantial part of daily life and frequently trigger negative emotions like anger or anxiety, which have a significant adverse impact on driving safety as well as long-term human health. To identify driver emotions, thereby improving the safety and humanization of intelligent driving, we explore how to model the discriminative emotion features from both speech and facial expressions in this work. More specifically, an effective attention-based network for facial expression and a lightweight speech emotion network are proposed, separately. Then, audio and video features are combined at the feature level to construct our multimodal driver emotion recognition model. This paper proposes a new audio feature extractor that uses a multi-scale residual structure to extract spectrogram features. In terms of video, a set of frame sequences using Local Binary Pattern Histograms (LBPH) is obtained through preprocessing, which generates a fixed-dimensional feature representation. These features are then input into a fine-tuned ResNet18 model to analyze spatial information. This model is further augmented by integrating both a temporal attention module and a Gated Recurrent Unit (GRU), enhancing its capability to create a highly discriminative video representation. Additionally, we propose an Internet of Vehicles (IoV) platform, specifically designed for driver emotion recognition. The IoV platform consists of sensor layer, data acquisition and transport layer, server layer and data application layer. The IoV platform uses sensors to collect multimodal data from drivers, which can provide data support for the proposed multimodal driver emotion recognition algorithm. The performance of this proposed algorithm is evaluated on two multimodal emotional datasets, Ryerson Audio-Visual Dataset of Emotional Speech and Song (RAVDESS) and Surrey Audio-Visual Expressed Emotion (SAVEE), using a variety of performance indicators. Compared to other baseline methods, this proposed multimodal model achieves state-of-the-art results on the RAVDESS and SAVEE datasets, demonstrating superior recognition accuracy with rates of 0.93 and 0.99, respectively. Additionally, it exhibits precision scores of 0.93 on RAVDESS and 0.99 on SAVEE, along with exceptional specificity scores of 0.99 and 1.00, respectively.

12 April 2024 in IEEE Transactions on Fuzzy Systems
Verified
Journal Article
Closed access
NYNa Ying
Na Ying
HLHongyu Li
Hongyu Li
ZZZhi Zhang
Zhi Zhang
YZYong Zhou
Yong Zhou
HCHuahua Chen
Huahua Chen
MYMeng Yang
Meng Yang
Sleep apnea-hypopnea syndrome (SAHS) is a disease that seriously affects human sleep. Due to its strong concealment and significant harm, it has gradually attracted people's attention. Therefore, it is necessary to use the SAHS detection model to screen the disease as early as possible. However, most of the current SAHS detection models use convolutional neural network (CNN) architectures such as Resnet, which are ineffective in capturing images' details. Moreover, the Vision Transformer (ViT) model relies heavily on the number of SaO2 signal image data during training. Otherwise, it isn't easy to achieve better performance. In addition, due to the existence of fuzzy and uncertain information in the SaO2 signal image, the detection accuracy of previous models is low. Therefore, in this work, we propose a vision transformer enabled deep fuzzy neural network (ViTDFNN) to detect SAHS in an Internet of Medical Things platform. The ViTDFNN model can analyze the patient's vital signs and sleep status on time and identify potential SAHS conditions. The ViTDFNN model first uses ViT to learn the features of the SaO2 signal image so that global information and long-range dependencies can be learned. Subsequently, the ViTDFNN model uses VGG to extract the features of the SaO2 signal image to learn the local features in the image, which is convenient for distinguishing the texture, edge, and other details. Finally, the ViTDFNN model linearly fuses the two image features and inputs them into the deep fuzzy neural network and fully connected network to detect the patient's symptoms. The deep fuzzy neural network can convert each deterministic pixel into an uncertain pixel, thereby solving the fuzzy and uncertain information in the SaO2 signal image. In addition, the ViTDFNN model also uses the masked strategy to form a task of image reconstruction, which is used to reduce the dependence of the ViT model on the labeled SaO2 signal image. We verified the superiority of ViTDFNN by conducting extensive experiments on a real sleep dataset. The scores of the ViTDFNN model under the F1-Score, Recall and Accuracy metrics are 0.994, 0.992 and 0.994, respectively.