Xuzhou Ye

Xuzhou Ye

Auto-generated profile, based on publicly available information

Xuzhou Ye

Beijing ByteDance Technology Co. Ltd.,Shanghai,China,201102

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 January 2024 in IEEE/CAA Journal of Automatica Sinica
Verified
Journal Article
Closed access
YZYin Zhu
Yin Zhu
QKQiuqiang Kong
Qiuqiang Kong
JSJunjie Shi
Junjie Shi
SLShilei Liu
Shilei Liu
XYXuzhou Ye
Xuzhou Ye
JZJunping Zhang
Junping Zhang
Binaural rendering is of great interest to virtual reality and immersive media. Although humans can naturally use their two ears to perceive the spatial information contained in sounds, it is a challenging task for machines to achieve binaural rendering since the description of a sound field often requires multiple channels and even the metadata of the sound sources. In addition, the perceived sound varies from person to person even in the same sound field. Previous methods generally rely on individual-dependent head-related transferred function (HRTF) datasets and optimization algorithms that act on HRTFs. In practical applications, there are two major drawbacks to existing methods. The first is a high personalization cost, as traditional methods achieve personalized needs by measuring HRTFs. The second is insufficient accuracy because the optimization goal of traditional methods is to retain another part of information that is more important in perception at the cost of discarding a part of the information. Therefore, it is desirable to develop novel techniques to achieve personalization and accuracy at a low cost. To this end, we focus on the binaural rendering of ambisonic and propose 1) channel-shared encoder and channel-compared attention integrated into neural networks and 2) a loss function quantifying interaural level differences to deal with spatial information. To verify the proposed method, we collect and release the first paired ambisonic-binaural dataset and introduce three metrics to evaluate the content information and spatial information accuracy of the end-to-end methods. Extensive experimental results on the collected dataset demonstrate the superior performance of the proposed method and the shortcomings of previous methods.

22 October 2022 in arXiv
Verified
Preprint
Open access
PDF file
QKQiuqiang Kong
Qiuqiang Kong
SLShilei Liu
Shilei Liu
JSJun-jie Shi
Jun-jie Shi
XYXuzhou Ye
Xuzhou Ye
YCYin Cao
Yin Cao
YWYuxuan Wang
Yuxuan Wang
Sound field decomposition predicts waveforms in arbitrary directions using signals from a limited number of microphones as inputs. Sound field decomposition is fundamental to downstream tasks, including source localization, source separation, and spatial audio reproduction. Conventional sound field decomposition methods such as Ambisonics have limited spatial decomposition resolution. This paper proposes a learning-based Neural Sound field Decomposition (NeSD) framework to allow sound field decomposition with fine spatial direction resolution, using recordings from microphone capsules of a few microphones at arbitrary positions. The inputs of a NeSD system include microphone signals, microphone positions, and queried directions. The outputs of a NeSD include the waveform and the presence probability of a queried position. We model the NeSD systems respectively with different neural networks, including fully connected, time delay, and recurrent neural networks. We show that the NeSD systems outperform conventional Ambisonics and DOANet methods in sound field decomposition and source localization on speech, music, and sound events datasets. Demos are available at https://www.youtube.com/watch?v=0GIr6doj3BQ.