Automatic musical transcription of the Tatar folk song: comparative analysis of AI-powered programs

Abstract

This article is relevant due to the loss of the carriers of folk music that needs to be recorded in digital audio formats and requires music transcription for the subsequent creation of collections for the purposes of scientific research by ethnomusicologists. The study aims at determining the need to use software for the automatic music transcription of audio recordings of folk music. The main research method is the comparative analysis of the music transcription of the Tatar Kryashen songs performed by people and three AI-powered programs (Celemony Melodyne, AudioScore Ultimate and Cubase). Then we compared the scores we prepared and the visual data of three programs: wave, spectral, “piano roll” and traditional music scores. According to five evaluation parameters (the accuracy of displaying a melody, rhythm, key, time signature and subjective assessment), the Cubase program was recognized as the most user-friendly. It is still controversial whether to use artificial intelligence for the music transcription of folk songs since music researchers decide for themselves. The undoubted benefit of the automatic music transcription of folk music is the rapid analysis of audio recordings, the ability to create more music notations in a shorter time, assist in the analysis of fragments that are difficult to hear by ear and restore damaged audio recordings.

Keywords

• automatic music transcription
• computational ethnomusicology
• Tatar folk song

The possibilities of artificial intelligence in automatic musical transcription of the Tatar folk song

Abstract

This article is relevant due to the loss of the carriers of folk music that needs to be recorded in digital audio formats and requires music transcription for the subsequent creation of collections for the purposes of scientific research by ethnomusicologists. The study aims at determining the need to use software for the automatic music transcription of audio recordings of folk music. The main research method is the comparative analysis of the music transcription of the Tatar Kryashen songs performed by people and three AI-powered programs (Celemony Melodyne, AudioScore Ultimate and Cubase). Then we compared the scores we prepared and the visual data of three programs: wave, spectral, “piano roll” and traditional music scores. According to five evaluation parameters (the accuracy of displaying a melody, rhythm, key, time signature and subjective assessment), the Cubase program was recognized as the most user-friendly. It is still controversial whether to use artificial intelligence for the music transcription of folk songs since music researchers decide for themselves. The undoubted benefit of the automatic music transcription of folk music is the rapid analysis of audio recordings, the ability to create more music notations in a shorter time, assist in the analysis of fragments that are difficult to hear by ear and restore damaged audio recordings.

Keywords

• automatic music transcription
• computational ethnomusicology
• Tatar folk song

Thanks

This paper is performed as part of the implementation of the Development Program of the Kazan State Institute of Culture and аs part of the implementation of the Kazan Federal University Strategic Academic Leadership Program.

References

1. Arakelyan, A.E. (2011). Est li u nekrasovskikh kazakov mnogogolosie? (opyt issledovaniya tekhnicheskimi sredstvami) [Is there the chorus of the Nekrasov Cossacks? (the experience of research through technical means)]. Yuzhno-Rossiiskii Muzykalnyi Almanakh, 2(9), 3-7.
2. Borodovskaja, L.Z. (2021). Istorija razvitija informatizacii discipliny “Sbor i rasshifrovka muzykalnogo folklore” [The historical development of discipline “The collection and transcription of music folklore”]. In: L.Z. Borodovskaja (Compl.), Mnogogrannyj mir tradicionnoj kultury i narodnogo hudozhestvennogo tvorchestva: Proceedings of the All-Russian scientific conference within the All-Russian competition AR/ VR “Hackathon in the sphere of culture”, October 12, 2020, Kazan, Russia (pp. 251- 254). Kazan: Kazan State Institute of Culture.
3. Briot, J. P. (2021). From artificial neural networks to deep learning for music generation: history, concepts and trends. Neural Computing and Applications, 33(1), 39–65. https://doi.org/10.1007/s00521-020- 05399-0
4. Celemony. (n.d.). Melodyne 5. https:// www.celemony.com/en/melodyne/new-in- melodyne-5
5. Danshina, M.V., Filippovich, A.Yu. (2014). Metodika avtomatizirovannoi rasshifrovki znamennykh pesnopenii [The methods of automatic transcription of Znamenny Chants]. Vestnik Moskovskogo gosudarstvennogo tekhnicheskogo universiteta im. N.E. Baumana. Seriya “Priborostroenie”, 4(97), 55-69.
6. Dyganova, E.A., Yavgildina, Z.M., Shirieva, N.V. (2017). The competition training method in the formation of professional competence of the future music teacher. Man in India, 97(20), 403-414.
7. Gorbunova, I.B. (2016). Metodicheskie aspekty tolkovaniya funktsionalno- logicheskikh zakonomernostei muzyki i muzykalno-kompyuternye tekhnologii: sistemy muzykalnoi notatsii [The methodological aspects of explaining functional-logical regularities of music and music computational technologies: music notation systems]. Obshchestvo: Sotsiologiya, psikhologiya, pedagogika, 10, 69-77.
8. Grebosz-Haring, K., & Weichbold, M. (2020). Contemporary art music and its audiences: Age, gender, and social class profile. Musicae Scientiae, 24(1), 60–77. https://doi. org/10.1177/1029864918774082

9. Gubaidullin, konvertatsiya selkupskikh instrumentalnykh naigryshei [The digitalization of analog recordings of instrumental folk tunes]. Vestnik Tomskogo gosudarstvennogo universiteta. Kulturologiya i iskusstvovedenie, 1(21), 99-105.
10. Heil, L. (2017). Teaching Improvisation through Melody and Blues-Based Harmony: A Comprehensive and Sequential Approach. Music Educators Journal, 104(1), 40–46. https://doi.org/10.1177/0027432117711484
11. ISMIR (2022). International Society for Music Information. Retrieval web: https://ismir. net/
12. Kharuto, A. traditsionnykh Vozmozhnosti kompyuternogo analiza [Tone series in traditional musical cultures: Possibilities of computer-assisted analysis]. In: Muzykalyкetnografiyadaғy tұlғa, dәstүr, mәdeniet:AleksandrZataevich150zhyldyғyna arnalғan Khalyқaralyқ ғylymi-praktikalyқ konferentsiyanyӊ materialdary [Personality, tradition, culture in musical ethnography: Proceedings of the International scientific- practical conference dedicated to the 150th anniversary of Alexander Zataevich] (pp. 77- 82). Almaty: Kazakh National Conservatory named after Kurmangazy.
13. Konev, A.A., Onishchenko, A.A., Kostyuchenko, E.Yu., Yakimuk, A.Yu. (2015). Avtomaticheskoe raspoznavanie muzykalnykh not [The automatic detection of music scores]. Nauchnyi vestnik Novosibirskogo gosudarstvennogo tekhnicheskogo universiteta, 3, 32-47.
14. Kroher, N., Gómez, E. (2016). Automatic transcription of flamenco singing from polyphonic music recordings. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 24(5), 901-913.
15. Liu, L., & Benetos, E. (2021). From audio to music notation. In: E.R. Miranda (Ed.), Handbook of artificial intelligence for music (pp. 693-714). Cham: Springer.
16. Miranda, E.R. (Ed.). (2021). Handbook of artificial intelligence for music: Foundations, advanced approaches, and developments for creativity. Cham: Springer Nature.
17. Neuratron. (2020). AudioScore 2020. https:// www.neuratron.com/audioscore.htm
18. Pavlov, D. N. (2021). Methods for development of creative expression of college students in music composition. Bulletin of Nizhnevartovsk State University, (1 (53)), 66–73. https:// doi.org/10.36906/2311-4444/21-1/09
19. Rao, B.T., Chinnam, S., Kanth, P.L., Gargi, M. (2012). Automatic Melakarta Raaga Identification System: Carnatic Music. International Journal of Advanced Research in Artificial Intelligence, 1(4), 43-48. Steinberg. (n.d.). Cubase. https://www. steinberg.net/cubase/
20. Sviyazova, E.R. (2019). Problema avtomaticheskoi muzykalnoi transkriptsii [The issue of automatic music transcription]. Molodezhnyi vestnik Ufimskogo gosudarstvennogo aviatsionnogo tekhnicheskogo universiteta, 1, 159-162.
21. Turchet, L., Fischione, C., Essl, G., Keller, D., & Barthet, M. (2018). Internet of Musical Things: Vision and Challenges. IEEE Access, 6, 61994–62017. https://doi.org/10.1109/ ACCESS.2018.2872625
22. Yakimuk, A.Yu. (2016). Algoritmy analiza chastoty osnovnogo tona vokalnogo ispolneniya [The algorithms of analyzing the basic frequency of vocal performance]. In: Nauchnaya sessiya TUSUR–2016: Proceedings of the International scientific conference of students, postgraduate students and young scientists (pp. 245-248). V-Spektr.
23. Yakimuk, A.Yu. (2020). Otsenka raboty programmnogo kompleksa po raspoznavaniyu not [Assessing the functioning of programs deciphering music scores]. In: I.A. Kurzina, G.A. Voronova (Eds.), Perspektivy razvitiya fundamentalnykh nauk (pp. 141-143). Tomsk: Tomsk State University of Control Systems and Radioelectronics.
24. Yunusova, V.N., Kharuto, A.V. (2020). Kompyuternaya etnomuzykologiya: Zadachi, metody, rezultaty [Computational ethnomusicology: Tasks, methods, results]. Muzykalnaya akademiya, 3, 162-177.
25. Zhang, L. (2020). The genre of folk song settings: the ways of interaction of traditional and professional musical art. Aspects of Historical Musicology, 21(21), 113–123. https://doi.org/10.34064/khnum2-21.07