Minglu Zhao

Minglu Zhao

orcid

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 February 2024 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
MZMinglu Zhao
Minglu Zhao
WWWenmin Wang
Wenmin Wang
TCTongbao Chen
Tongbao Chen
RZRui Zhang
Rui Zhang
RLRuochen Li
Ruochen Li
Recent conditional and unconditional video generation tasks have been accomplished mainly based on generative adversarial network (GAN), diffusion, and autoregressive models. However, in some circumstances, using only one modality cannot provide enough semantic information. Therefore, in this paper, we propose text-audio to video (TA2V) generation, a new task for generating realistic videos from two different guided modalities, text and audio, which has not been explored much thus far. Compared to image generation, video generation is a harder task because of the complexity of processing higher-dimensional data and scarcer suitable datasets, especially for multimodal video generation. To overcome these limitations, (i) we propose the Text&Audio-guided-Video-Maker (TAgVM) model, which consists of two modules: a text-guided video generator and a text&audio-guided video modifier. (ii) This model uses a 3D VQ-GAN to compress high-dimension video data to a low-dimension discrete sequence, followed by an autoregressive model to guide text-conditional generation in the latent space. Then, we apply a text&audio-guided diffusion model to the generated video scenes, providing additional semantic details corresponding to the audio and text. (iii) We introduce a newly produced music performance video dataset, the University of Rochester Multimodal Music Performance with Video-Audio-Text (URMP-VAT), and a landscape dataset, Landscape with Video-Audio-Text (Landscape-VAT), both of which include three modalities (text, audio, and video) that are aligned with each other. The results demonstrate that our model can create videos with satisfactory quality and semantic information. The source code and datasets are available at https://github.com/Minglu58/TA2V.

28 June 2024 in PNAS Nexus
Verified
Journal Article
Open access
PDF file
RZRui Zhang
Rui Zhang
MZMinglu Zhao
Minglu Zhao
HWHengwei Wang
Hengwei Wang
HWHaimei Wang
Haimei Wang
HKHui Kong
Hui Kong
JXJianyin Xiong
Jianyin Xiong
Volatile organic compounds (VOCs) are ubiquitous in vehicle cabin environments, which can significantly impact the health of drivers and passengers, whereas quick and intelligent prediction methods are lacking. In this study, we firstly analyzed the variations of environmental parameters, VOC levels and potential sources inside a new car during 7 summer workdays, indicating that formaldehyde had the highest concentration and about one third of the measurements exceeded the standard limit for in-cabin air quality. Feature importance analysis reveals that the most important factor affecting in-cabin VOC emission behaviors is the material surface temperature rather than the air temperature. By introducing the attention mechanism and ensemble strategy, we present an LSTM-A-E deep learning model to predict the concentrations of 12 observed typical VOCs, together with other five deep learning models for comparison. By comparing the prediction–observation discrepancies and five evaluation metrics, the LSTM-A-E model demonstrates better performance, which is more consistent with field measurements. Extension of the developed model for predicting the 10-day VOC concentrations in a realistic residence further illustrates its excellent environmental adaptation. This study probes the not-well-explored in-cabin VOC dynamics via observation and deep learning approaches, facilitating rapid prediction and exposure assessment of VOCs in the vehicle micro-environment.

5 January 2025 in Expert Systems with Applications
Verified
Journal Article
Closed access
MZMinglu Zhao
Minglu Zhao
WWWenmin Wang
Wenmin Wang
RZRui Zhang
Rui Zhang
HJHaomei Jia
Haomei Jia
QCQi Chen
Qi Chen

11 June 2024 in Neurocomputing
Verified
Journal Article
Closed access
TCTongbao Chen
Tongbao Chen
WWWenmin Wang
Wenmin Wang
MZMinglu Zhao
Minglu Zhao
RLRuochen Li
Ruochen Li
ZJZhe Jiang
Zhe Jiang
CYCheng Yu
Cheng Yu

7 October 2024 in Biomedical Signal Processing and Control
Verified
Journal Article
Closed access
QCQi Chen
Qi Chen
WWWenmin Wang
Wenmin Wang
ZWZhibing Wang
Zhibing Wang
HJHaomei Jia
Haomei Jia
MZMinglu Zhao
Minglu Zhao

4 April 2024 in Neurocomputing
Verified
Journal Article
Closed access
RLRuochen Li
Ruochen Li
WWWenmin Wang
Wenmin Wang
HHHuanqiang Hu
Huanqiang Hu
TCTongbao Chen
Tongbao Chen
MZMinglu Zhao
Minglu Zhao