Music information retrieval (MIR) is developing these years rapidly. As the fundamental MIR tasks, automatic music transcription (AMT) and expressive analysis (EA) are gaining momentum in both Western and non-European music. However, the annotated datasets for non-Eurogenic instruments remain scarce in terms of quantity and feature diversity so that general evaluations and data-driven models on various tasks cannot be well explored. As one of the most popular traditional plucked string instruments in Asia, which is barely studied in the MIR community, pipa has lots of distinctive national and local characteristics including the fake nails, intrinsic pitch shift, rubato, as well as a high diversity of sophisticated playing techniques that greatly enhance the music expressiveness. Our work aims to systematically clarify a creation procedure of a pipa dataset with audio, musical notation and multiview video modalities for traditional Chinese solos. The use of 4-track string vibration signals captured by optical sensors paves a path for high quality annotations. Furthermore, a transcription and Expressiveness Annotation System (TEAS) was transparently implemented to ensure the scalability of dataset. Three expressive analysis approaches in this system were newly proposed and evaluated in this paper. Finally, two AMT models were investigated and a series of the existing and emerging MIR tasks enabled by this dataset were enumerated for the future exploration.
The musical signal produced by plucked instruments often exhibits non-stationarity due to variations in the pitch and amplitude, making pitch estimation a challenge. In this paper, we assess different transcription processes and algorithms applied to signals captured by optical sensors mounted on a pipa—a traditional Chinese plucked instrument—played using a range of techniques. The captured signal demonstrates a distinctive arched feature during plucking. This facilitates onset detection to avoid the impact of the spurious energy peaks within vibration areas that arise from pitch-shift playing techniques. Subsequently, we developed a novel time–frequency feature, known as continuous time-period mapping (CTPM), which contains pitch curves. The proposed process can also be applied to playing techniques that mix pitch shifts and tremolo. When evaluated on four renowned pipa music pieces of varying difficulty levels, our fully time-domain-based onset detectors outperformed four short-time methods, particularly during tremolo. Our zero-crossing-based pitch estimator achieved a performance comparable to short-time methods with a far better computational efficiency, demonstrating its suitability for use in a lightweight algorithm in future work.
The prosodic music signal physically exhibits non-linearity and non-stationarity driven by variations in pitch and amplitude, which pose a great challenge to pitch estimation under complex playing techniques. To address this issue, we revisit different single-pitch annotation algorithms and processes on the string displacement signal captured by optical sensors mounted on pipa, a Chinese traditional plucked string instrument with a high diversity of playing techniques. This signal demonstrates an arched form momentarily deviating from origin at plucks, which facilitates accurate boundary detection and avoids the impact of spurious energy peaks arose by pitch-shift playing techniques within vibration areas. Then we develop a novel amplitude-invariant sparse time-frequency representation termed by continuous time-period mapping (CTPM), which allows to extract pitch curves without signal shaping, even in cases where multiple oscillations occur within a single period. Playing techniques mixed of pitch-shift and tremolo are also covered by our proposed process. Evaluated on 4 renowned expressive pipa music pieces of varying difficulty levels, our proposed fully time-domain onset detectors outperform the 4 commonly-used short-time methods particularly during rapid-plucking and tremolo. Zero-crossing-based pitch estimator achieved an overwhelming computational efficiency and a performance comparable to short-time methods if provided the correct boundaries.