This paper is part of a special issue on Music Technology. We study the type recognition of traditional Chinese musical instrument audio in the common way. Using MEL spectrum characteristics as input, we train an 8‐layer convolutional neural network, and finally achieve 99.3% accuracy. After that, this paper mainly studies the performance skill recognition of Chinese traditional musical instruments. Firstly, for a single instrument, the features were extracted by using the pre‐trained ResNet model, and then the SVM algorithm was used to classify all the instruments with an accuracy of 99%. Then, in order to improve the generalization of the model, the paper proposes the performance skill recognition of the same kind of instruments. In this way, the regularity of the same playing technique of different instruments can be utilized. Finally, the recognition accuracy of the four kinds of instruments is as follows: 95.7% for blowing instruments, 82.2% for plucked‐string instruments, 88.3% for strings instruments, and 97.5% for percussion instruments. We open source the audio database of traditional Chinese musical instruments and the Python source code of the whole experiment for further research.
The development of artificial intelligence technology has made it possible to realize automatic evaluation systems for singing, and relevant research has been able to achieve accurate evaluations with respect to pitch and rhythm, but research on singing-voice timbre evaluation has remained at the level of theoretical analysis. Timbre is closely related to expression performance, breath control, emotional rendering, and other aspects of singing skills, and it has a crucial impact on the evaluation of song interpretation. The purpose of this research is to investigate the automatic evaluation method of singing-voice timbre. At the present stage, timbre research generally has problems such as a paucity of datasets, a single evaluation index, easy overfitting or a model’s failure to converge. Compared with the singing voice, the research on musical instruments is more mature, with more available data and richer evaluation dimensions. We constructed a deep network based on the CRNN model to perform timbre evaluation, and the test results showed that cross-media learning of timbre evaluation is feasible, which also indicates that humans have a consistent timbre perception with respect to musical instruments and vocals.
17 February 2022 in Cognitive Computation and Systems
The article presents the design and development of a virtual fretless Chinese stringed instrument App with the Duxianqin as an example, whose performance is expected to be no different from a real instrument. The digital simulation of fretless musical instruments is mainly divided into two parts: the simulation of the continuous pitch processing of the strings, and the simulation of the sound produced by plucking strings. The article returns to the theory of mechanics and wave theory and obtains the quantitative relationship between string frequency and its deformation and elongation. The Duxianqin selected in this article is a fretless instrument, which cannot be completely simulated by relying solely on sound source data. Playing and vocalization require real‐time synthesis through pitch processing, which has certain reference significance for the realization of other fretless instruments.