Wei Xue

Dr. Wei Xue


Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024
Verified
Conference Paper
Open access
PDF file
RYRuibin Yuan
Ruibin Yuan
HLHanfeng Lin
Hanfeng Lin
YWYi Wang
Yi Wang
ZTZeyue Tian
Zeyue Tian
SWShangda Wu
Shangda Wu
YGYike Guo
Yike Guo
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo. Findings of the Association for Computational Linguistics ACL 2024. 2024.

26 October 2023
Verified
Conference Paper
Open access
ZYZhen Ye
Zhen Ye
WXWei Xue
Wei Xue
XTXu Tan
Xu Tan
JCJie Chen
Jie Chen
QLQifeng Liu
Qifeng Liu
YGYike Guo
Yike Guo
Denoising diffusion probabilistic models (DDPMs) have shown promising performance for speech synthesis. However, a large number of iterative steps are required to achieve high sample quality, which restricts the inference speed. Maintaining sample quality while increasing sampling speed has become a challenging task. In this paper, we propose a Consistency Model-based Speech synthesis method, CoMoSpeech, which achieve speech synthesis through a single diffusion sampling step while achieving high audio quality. The consistency constraint is applied to distill a consistency model from a well-designed diffusion-based teacher model, which ultimately yields superior performances in the distilled CoMoSpeech. Our experiments show that by generating audio recordings by a single sampling step, the CoMoSpeech achieves an inference speed more than 150 times faster than real-time on a single NVIDIA A100 GPU, which is comparable to FastSpeech2, making diffusion-sampling based speech synthesis truly practical. Meanwhile, objective and subjective evaluations on text-to-speech and singing voice synthesis show that the proposed teacher models yield the best audio quality, and the one-step sampling based CoMoSpeech achieves the best inference speed with better or comparable audio quality to other conventional multi-step diffusion model baselines. Audio samples and codes are available at https://comospeech.github. https://comospeech.github.io/.

24 March 2024 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
DZDongmei Zhang
Dongmei Zhang
CLChang Li
Chang Li
RZRenrui Zhang
Renrui Zhang
SXShenghao Xie
Shenghao Xie
WXWei Xue
Wei Xue
SZShanghang Zhang
Shanghang Zhang
The superior performances of pre-trained foundation models in various visual tasks underscore their potential to enhance the 2D models' open-vocabulary ability. Existing methods explore analogous applications in the 3D space. However, most of them only center around knowledge extraction from singular foundation models, which limits the open-vocabulary ability of 3D models. We hypothesize that leveraging complementary pre-trained knowledge from various foundation models can improve knowledge transfer from 2D pre-trained visual language models to the 3D space. In this work, we propose FM-OV3D, a method of Foundation Model-based Cross-modal Knowledge Blending for Open-Vocabulary 3D Detection, which improves the open-vocabulary localization and recognition abilities of 3D model by blending knowledge from multiple pre-trained foundation models, achieving true open-vocabulary without facing constraints from original 3D datasets. Specifically, to learn the open-vocabulary 3D localization ability, we adopt the open-vocabulary localization knowledge of the Grounded-Segment-Anything model. For open-vocabulary 3D recognition ability, We leverage the knowledge of generative foundation models, including GPT-3 and Stable Diffusion models, and cross-modal discriminative models like CLIP. The experimental results on two popular benchmarks for open-vocabulary 3D object detection show that our model efficiently learns knowledge from multiple foundation models to enhance the open-vocabulary ability of the 3D model and successfully achieves state-of-the-art performance in open-vocabulary 3D object detection tasks. Code is released at https://github.com/dmzhang0425/FM-OV3D.git.

6 March 2026 in International Journal of Computer Vision
Verified
Journal Article
Open access
PDF file
XLXinyu Liu
Xinyu Liu
YHYingqing He
Yingqing He
LGLanqing Guo
Lanqing Guo
XLXiang Li
Xiang Li
BJBu Jin
Bu Jin
YGYike Guo
Yike Guo
The potential for higher-resolution image generation using pretrained diffusion models is immense. However, these models often struggle with object repetition and structural artifacts especially when scaling to 4K resolution and beyond. Our analysis reveals that causes the problem, a single prompt for the generation of multiple scales provides insufficient efficacy. To address this, we propose HiPrompt, a new tuning-free solution that tackles the above problems by introducing hierarchical prompts. The hierarchical prompts provide both global and local semantic guidance. Specifically, the global prompt captures overall scene semantics from user input, while local guidance comes from patch-wise descriptions generated by MLLMs to refine regional structures and textures. Furthermore, during inverse denoising, noise is decomposed into low- and high-frequency components, each conditioned on different prompt levels, facilitating prompt-guided denoising under hierarchical semantic guidance. It further allows the generation to focus more on local spatial regions and ensures the generated images maintain coherent local and global semantics, structures, and textures with high definition. Extensive experiments demonstrate that HiPrompt outperforms state-of-the-art works in higher-resolution image generation, significantly reducing object repetition and enhancing structural quality. The demo and code can be found on the project website: https://liuxinyv.github.io/HiPrompt/ .

28 October 2024
Verified
Conference Paper
Open access
ZYZhen Ye
Zhen Ye
ZJZeqian Ju
Zeqian Ju
HLHaohe Liu
Haohe Liu
XTXu Tan
Xu Tan
JCJianyi Chen
Jianyi Chen
YGYike Guo
Yike Guo

30 October 2024
Verified
Book Chapter
Closed access
PDF file
LLLujun Li
Lujun Li
ZWzimian wei
zimian wei
PDPeijie Dong
Peijie Dong
WLWenhan Luo
Wenhan Luo
WXWei Xue
Wei Xue
YGYike Guo
Yike Guo