Xinyi Tong

Xinyi Tong

Auto-generated profile, based on publicly available information

Xinyi Tong

Central Conservatory of Music Beijing China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 October 2024 in IEEE Transactions on Computational Social Systems
Verified
Journal Article
Closed access
XTXinyi Tong
Xinyi Tong
SCSitong Chen
Sitong Chen
PYPeiyang Yu
Peiyang Yu
NLNian Liu
Nian Liu
HQHui Qv
Hui Qv
SZSong-Chun Zhu
Song-Chun Zhu
Automatically generating video background music mitigates the inefficiency and time-consuming drawbacks of current manual video editing. Two key challenges hinder the expansion of the inception of video-to-music tasks. 1) Limited availability of high-quality video–music datasets and annotations. 2) Absence of music generation methods that consider actual musicality, which are controlled by interpretable factors based on music theory. In the article, we propose video echoed in harmony (VEH), a method for learning and sampling video-integrated chord progression sequences. Our approach adopts harmony, represented by chord progressions that are aligned with various music formats [musical instrument digital interface (MIDI), audio, and score], imitating chord precedence in human music composition. Visual-language models link visual features to chord progressions through genre labels and descriptive words in generated textualized videos. The two aforementioned features collectively obviate the necessity of extensive video–music paired data. Besides, an energy-based chord progression learning and sampling algorithm quantifies abstract harmony impressions to statistical features, serving as interpretable factors for the controllable music generation based on music theory. Experimental results demonstrate that the proposed method outperforms the state-of-the-art, producing a superior music alignment for the given video.


1 April 2026
Verified
Book Chapter
Closed access
PDF file
XTXinyi Tong
Xinyi Tong
TWTianle Wang
Tianle Wang
PYPeiyang Yu
Peiyang Yu
JCJishang Chen
Jishang Chen
SZSirui Zhang
Sirui Zhang
XJXin Jin
Xin Jin