Yixuan Zhou

Yixuan Zhou

Auto-generated profile, based on publicly available information

Yixuan Zhou

Shenzhen International Graduate School, Tsinghua University, Shenzhen, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 October 2024
Verified
Conference Paper
Open access
YZYixuan Zhou
Yixuan Zhou
XQXiaoyu Qin
Xiaoyu Qin
ZJZeyu Jin
Zeyu Jin
SZShuoyi Zhou
Shuoyi Zhou
SLShun Lei
Shun Lei
JJJia Jia
Jia Jia
Recent AIGC systems possess the capability to generate digital multimedia content based on human language instructions, such as text, image and video. However, when it comes to speech, existing methods related to human instruction-to-speech generation exhibit two limitations. Firstly, they require the division of inputs into content prompt (transcript) and description prompt (style and speaker), instead of directly supporting human instruction. This division is less natural in form and does not align with other AIGC models. Secondly, the practice of utilizing an independent description prompt to model speech style, without considering the transcript content, restricts the ability to control speech at a fine-grained level. To address these limitations, we propose VoxInstruct, a novel unified multilingual codec language modeling framework that extends traditional text-to-speech tasks into a general human instruction-to-speech task. Our approach enhances the expressiveness of human instruction-guided speech generation and aligns the speech generation paradigm with other modalities. To enable the model to automatically extract the content of synthesized speech from raw text instructions, we introduce speech semantic tokens as an intermediate representation for instruction-to-content guidance. We also incorporate multiple Classifier-Free Guidance (CFG) strategies into our codec language model, which strengthens the generated speech following human instructions. Furthermore, our model architecture and training strategies allow for the simultaneous support of combining speech prompt and descriptive human instruction for expressive speech synthesis, which is a first-of-its-kind attempt. Codes, models and demos are at: https://github.com/thuhcsi/VoxInstruct.

2 August 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
SLShun Lei
Shun Lei
YZYixuan Zhou
Yixuan Zhou
LCLiyang Chen
Liyang Chen
ZWZhiyong Wu
Zhiyong Wu
XWXixin Wu
Xixin Wu
HMHelen Meng
Helen Meng
Expressive speech synthesis is crucial for many human-computer interaction scenarios, such as audiobooks, podcasts, and voice assistants. Previous works focus on predicting the style embeddings at one single scale from the information within the current sentence. Whereas, context information in neighboring sentences and multi-scale nature of style in human speech are neglected, making it challenging to convert multi-sentence text into natural and expressive speech. In this paper, we propose MSStyleTTS, a style modeling method for expressive speech synthesis, to capture and predict styles at different levels from a wider range of context rather than a sentence. Two sub-modules, including multi-scale style extractor and multi-scale style predictor, are trained together with a FastSpeech 2 based acoustic model. The predictor is designed to explore the hierarchical context information by considering structural relationships in context and predict style embeddings at global-level, sentence-level and subword-level. The extractor extracts multi-scale style embedding from the ground-truth speech and explicitly guides the style prediction. Evaluations on both in-domain and out-of-domain audiobook datasets demonstrate that the proposed method significantly outperforms the three baselines. In addition, we conduct the analysis of the context information and multi-scale style representations that have never been discussed before.

28 October 2024
Verified
Conference Paper
Open access
YXYaoxun Xu
Yaoxun Xu
YZYixuan Zhou
Yixuan Zhou
YCYunrui Cai
Yunrui Cai
JXJingran Xie
Jingran Xie
RYRunchuan Ye
Runchuan Ye
ZWZhiyong Wu
Zhiyong Wu
This paper addresses the challenges in MER 2024 by focusing on the Open Vocabulary (OV) task, which extends beyond traditional fixed label space for multimodal emotion recognition. The study emphasizes the use of Large Language Models (LLMs) to interpret and extract emotional information from multimodal inputs, complemented by speech transcription, speech emotion description, and video clues. The paper explores the integration of these features into a prompt fed into a pre-trained LLaMA3-8B model, utilizing prompt engineering to achieve satisfactory results without fine-tuning. This approach bridges the gap between speech, video and text data, leveraging the full potential of LLMs for open-ended emotion recognition tasks and introducing a solution to the field.

28 October 2024
Verified
Conference Paper
Open access
YCYunrui Cai
Yunrui Cai
RYRunchuan Ye
Runchuan Ye
JXJingran Xie
Jingran Xie
YZYixuan Zhou
Yixuan Zhou
YXYaoxun Xu
Yaoxun Xu
ZWZhiyong Wu
Zhiyong Wu
Multimodal emotion recognition (MER) plays a crucial role in user sentiment analysis and enhancing human-computer interaction experiences. However, in real-world scenarios, noise interference such as environmental noise and image blurring is widespread, impacting the model's ability to recognize emotions. Therefore, improving the robustness of MER models to handle complex and variable noise interference is a significant challenge. To address this challenge, we propose a robust representation learning method based on contrastive learning and Mixup data augmentation strategies to stabilize model performance on noisy data. Specifically, we first perform Mixup data augmentation in the representation space of each modality to broaden class decision boundaries and enhance model generalization. Meanwhile, through a contrastive learning strategy, we align the representations of clean and noisy data to improve model robustness. We evaluate our model's performance on the test set of the MER2024 MER-NOISE track. Our model achieves a weighted average F-score score of 82.71%, which is 3.09% higher than the MER2024 Baseline model. This result demonstrates that our method effectively enhances model robustness and generalization.

27 October 2025
Verified
Conference Paper
Open access
SZSongtao Zhou
Songtao Zhou
XQXiaoyu Qin
Xiaoyu Qin
YZYixuan Zhou
Yixuan Zhou
QWQixin Wang
Qixin Wang
ZJZeyu Jin
Zeyu Jin
JJJia Jia
Jia Jia

11 April 2023 in Journal of Surgery
Verified
Journal Article
Open access
PDF file
DLDingjie Liang
Dingjie Liang
MXMengmeng Xu
Mengmeng Xu
JZJianing Zhang
Jianing Zhang
THTao He
Tao He
YZYixuan Zhou
Yixuan Zhou
WQWeiqing Qian
Weiqing Qian
Chinese orthopedic surgeons have accumulated rich experience in clinical practice for thousands of years and formed a unique theoretical system and treatment methods, which have been passed down from generation to generation. With the introduction of western plastic surgery technology into China in the mid-19th century, orthopedics of traditional Chinese medicine had a great development. At this stage, traditional Chinese orthopedic surgeons accepted modern western surgical techniques and summed up eight new techniques for bone-setting (traditional Chinese medicine manual Reduction). The history of the development of CO (Chinese osteosynthesis) is briefly reviewed in this article. Instead of pursuing open reduction and internal fixation, CO insists on closed reduction to treat long bone fractures of the extremities. CO pursues functional alignment of fractures, relative fixation, and recovery of the function of the affected limb as the primary purpose. This treatment concept has been widely recognized by orthopedic surgeons in China. This article introduces eight kinds of reduction manipulations for treating long bone fractures under the guidance of the concept of CO, including Touching with Hands; Traction; Rotation, Flexion, and Extension; Lift-pressing and Squeezing; Swing and Percussing; Separating the bones; Folding and Rotation; Massage. It is important that manual Reduction should not be used for open fractures. Attention should be paid to avoid injury of blood vessels and nerves during reduction.