Mengyi Shan

Mengyi Shan

Auto-generated profile, based on publicly available information

Mengyi Shan

University of Washington

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
21 November 2024
Verified
Book Chapter
Closed access
PDF file
MSMengyi Shan
Mengyi Shan
LDLu Dong
Lu Dong
YHYutao Han
Yutao Han
YYYuan Yao
Yuan Yao
TLTao Liu
Tao Liu
MHMitch Hill
Mitch Hill

14 February 2020 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
TTTJ Tsai
TJ Tsai
DYDaniel Yang
Daniel Yang
MSMengyi Shan
Mengyi Shan
TTThitaree Tanprasert
Thitaree Tanprasert
TJTeerapat Jenrungrot
Teerapat Jenrungrot
This article investigates a cross-modal retrieval problem in which a user would like to retrieve a passage of music from a MIDI file by taking a cell phone picture of several lines of sheet music. This problem is challenging for two reasons: it has a significant runtime constraint since it is a user-facing application, and there is very little relevant training data containing cell phone images of sheet music. To solve this problem, we introduce a novel feature representation called a bootleg score which encodes the position of noteheads relative to staff lines in sheet music. The MIDI representation can be converted into a bootleg score using deterministic rules of Western musical notation, and the sheet music image can be converted into a bootleg score using classical computer vision techniques for detecting simple geometrical shapes. Once the MIDI and cell phone image have been converted into bootleg scores, we can estimate the alignment using dynamic programming. The most notable characteristic of our system is that it has no trainable weights at all — only a set of about 40 hyperparameters. With a training set of just 400 images, we show that our system generalizes well to a much larger set of 1600 test images from 160 unseen musical scores. Our system achieves a test F measure score of 0.89, has an average runtime of 0.90 seconds, and outperforms baseline systems based on music object detection and sheet–audio alignment. We provide extensive experimental validation and analysis of our system.

26 March 2021 in Transactions of the International Society for Music Information Retrieval
Verified
Journal Article
Open access
PDF file
MSMengyi Shan
Mengyi Shan
TTTJ Tsai
TJ Tsai

10 December 2023
Verified
Conference Paper
Open access
MSMengyi Shan
Mengyi Shan
BCBrian Curless
Brian Curless
IKIra Kemelmacher-Shlizerman
Ira Kemelmacher-Shlizerman
SSSteve Seitz
Steve Seitz
We present a system that automatically brings street view imagery to life by populating it with naturally behaving, animated pedestrians and vehicles. Our approach is to remove existing people and vehicles from the input image, insert moving objects with proper scale, angle, motion and appearance, plan paths and traffic behavior, as well as render the scene with plausible occlusion and shadowing effects. The system achieves these by reconstructing the still image street scene, simulating crowd behavior, and rendering with consistent lighting, visibility, occlusions, and shadows. We demonstrate results on a diverse range of street scenes including regular still images and panoramas.