Real-time chord identification application: Enabling lifelong music education through seamless integration of audio processing and machine learning

Year 2024, Volume: 9 Issue: 2, 405 - 414, 31.12.2024

Nihan Özbaltan

https://doi.org/10.31811/ojomus.1580523

Abstract

Lifelong music education is critical need for all with a particular focus on adult learners and seniors. One of the difficulties in music education is identifying chords accurately. This is a preliminary study to develop a chord identification application using Artificial Intelligence (AI) technologies. I seek to answer the key research question of how audio processing algorithms and deep learning models can be used to provide real-time, accurate and user-friendly chord recognition that meets the diverse needs of adult learners and senior citizens. Our overall goal is to create an application that not only assists with chord identification, but also fosters a lifelong love of music and learning. My methodology is based on the principles of adult and senior education initiatives and includes the following key steps: using ready-made datasets for audio processing and feature extraction, transforming waveforms into mel spectrograms, and preparing and extending the datasets where necessary. I then train and optimise deep learning models, such as various convolutional neural network (CNN) architectures, to achieve high accuracy in chord recognition. By using advanced technologies and adhering to the principles of lifelong learning, our research aims to enhance the musical journey of individuals throughout their lives, contributing to both personal enrichment and cognitive well-being.

Keywords

lifelong music education, audio processing, machine learning, adult learners, artificial intelligence

Gerçek zamanlı akor tanımlama uygulaması: Ses işleme ve makine öğreniminin kusursuz entegrasyonuyla ömür boyu müzik eğitimini mümkün kılıyor

Abstract

Ömür boyu müzik eğitimi, özellikle yetişkin öğrenciler ve yaşlılar olmak üzere herkes için kritik bir ihtiyaçtır. Müzik eğitimindeki zorluklardan biri akorları doğru bir şekilde belirlemektir. Bu, Yapay Zeka (AI) teknolojilerini kullanarak bir akor tanımlama uygulaması geliştirmek için yapılan bir ön çalışmadır. Ses işleme algoritmalarının ve derin öğrenme modellerinin yetişkin öğrencilerin ve yaşlı vatandaşların çeşitli ihtiyaçlarını karşılayan gerçek zamanlı, doğru ve kullanıcı dostu akor tanıma sağlamak için nasıl kullanılabileceği konusundaki temel araştırma sorusunu yanıtlamayı amaçlıyorum. Genel hedefimiz yalnızca akor tanımlamaya yardımcı olmakla kalmayıp aynı zamanda müzik ve öğrenmeye yönelik ömür boyu bir sevgiyi de teşvik eden bir uygulama yaratmaktır. Metodolojim yetişkin ve yaşlı eğitim girişimlerinin ilkelerine dayanmaktadır ve aşağıdaki temel adımları içerir: ses işleme ve özellik çıkarma için hazır veri kümelerini kullanma, dalga formlarını mel spektrogramlarına dönüştürme ve gerektiğinde veri kümelerini hazırlama ve genişletme. Daha sonra akor tanımada yüksek doğruluk elde etmek için çeşitli evrişimli sinir ağı (CNN) mimarileri gibi derin öğrenme modellerini eğitiyor ve optimize ediyorum. İleri teknolojileri kullanarak ve ömür boyu öğrenme ilkelerine bağlı kalarak, araştırmamız bireylerin yaşamları boyunca müzik yolculuğunu geliştirmeyi, hem kişisel zenginleşmeye hem de bilişsel refaha katkıda bulunmayı hedefliyor.

Keywords

yaşam boyu müzik eğitimi, ses işleme, makine öğrenimi, yetişkin öğrenciler, yapay zeka

References

• Acoustic Guitar Notes. (n.d.). https://www.kaggle.com/datasets/mohammedalkooheji/guitar-notes-dataset/data
• Boon, İ. E. T. (2024). Self-regulated learning skills in instrument education: A qualitative study. International Journal of Education and Literacy Studies, 12(1), 106-114. https://doi.org/10.7575/aiac.ijels.v.12n.1p.106
• Bowles, C. L. (1991). Self- expressed adult music education interests and music experiences. Journal of Research in Music Education, 39(3), 191-205. https://doi.org/10.2307/3344719
• Canavar, S., & Titrek, O. (2024). The impact of COVID-19 pandemic on school administrators’ psychology, family life, and work life (İznik sample). In Proceedings of the International Conference on Education Studies (pp. 76-96). https://doi.org/10.2991/978-94-6463-380-1_8
• Chollet, F. (2015) Keras. GitHub. https://github.com/fchollet/keras
• Choo, S. H., & Choi, J. H. (2023). The status and satisfaction level of instrumental music education on adult music education. Korean Journal of Research in Music Education, 52(3), 109-130. https://doi.org/10.30775/KMES.52.3.109
• Dascălu, M.-I., Coman, M., Postelnicu, R., & Nichifor, C. (2014). Learning to play a musical instrument in adulthood: Challenges and computer-mediated solutions. Procedia - Social and Behavioral Sciences, 142, 23-28. https://doi.org/10.1016/j.sbspro.2014.07.639
• Dong, M. (2018). Convolutional neural network achieves human-level accuracy in music genre classification. arXiv. http://arxiv.org/abs/1802.09697
• Dutta, A., Sil, D., Chandra, A., & Palit, S. (2022). CNN based musical instrument identification using time-frequency localized features. Internet Technology Letters, 5(1), e191. https://doi.org/10.1002/itl2.191
• François, C., Grau-Sánchez, J., Duarte, E., & Rodriguez-Fornells, A. (2015). Musical training as an alternative and effective method for neuro-education and neuro-rehabilitation. Frontiers in Psychology, 6, 475. https://doi.org/10.3389/fpsyg.2015.00475
• Giri, G. A. V. M., & Radhitya, M. L. (2024). Musical instrument classification using audio features and convolutional neural network. Journal of Applied Informatics and Computing, 8(1), 226-234.
• Gvozdevskaia, G. A. (2021). Methods of maintaining the mental activity of students when learning to play musical instruments online (on the example of learning to play the piano). Musical Art and Education, 9(3), 66-80. https://doi.org/10.31862/2309-1428-2021-9-3-66-80
• Hrybyk, A., & Kim, Y. (2010). Combined audio and video analysis for guitar chord identification. In Proceedings of the 11th International Society for Music Information Retrieval Conference, ISMIR 2010.
• Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv. http://arxiv.org/abs/1502.03167
• Konecki, M. (2023). Adaptive drum kit learning system: Impact on students’ learning outcomes. International Journal of Information and Education Technology, 13(10), 767-773. https://doi.org/10.18178/ijiet.2023.13.10.1959
• Kristian, Y., Zaman, L., Tenoyo, M., & Jodhinata, A. (2024). Advancing guitar chord recognition: A visual method based on deep convolutional neural networks and deep transfer learning. ECTI Transactions on Computer and Information Technology (ECTI-CIT), 18(2), 235-249.
• Lippolis, M., Müllensiefen, D., Frieler, K., Matarrelli, B., Vuust, P., Cassibba, R., & Brattico, E. (2022). Learning to play a musical instrument in middle school is associated with superior audiovisual working memory and fluid intelligence: A cross-sectional behavioral study. Frontiers in Psychology, 13, 982704. https://doi.org/10.3389/fpsyg.2022.982704
• Liu, X., & Dai, Y. (2023). Virtual computer systems in AI-powered music analysis: A comparative study for genre classification and musicological investigations. Journal of Information Systems Engineering and Management, 8(4), 23395. https://doi.org/10.55267/iadt.07.14016
• Ma, R., Deeprasert, J., & Jiang, S. (2024). Toward lifelong learning: Modelling willingness of chinese older adults learning music via social media. Theory and Practice, 2024(5), 3067-3081. https://doi.org/10.53555/kuey.v30i5.3388
• Mangla, P., Arora, S., & Bhatia, M. P. S. (2022). Intelligent audio analysis techniques for identification of music in smart devices. Internet Technology Letters, 5(2), e268. https://doi.org/10.1002/itl2.268
• Mavaddati, S. (2024). A voice activity detection algorithm using deep learning in the time-frequency domain. Neural Computing and Applications, 1-15. https://doi.org/10.1007/s00521-024-10795-x
• McFee, B., Raffel, C., Liang, D., Ellis, D. P. W., McVicar, M., Battenberg, E., & Nieto O. (2015). librosa: Audio and music signal analysis in Python. In Proceedings of the 14th Python in Science Conference (pp. 18-24). https://doi.org/10.25080/Majora-7b98e3ed-003
• Menglibekovich, B. M. (2024). Benefits of learning an instrument: Exploring the advantages. Journal of Education, Ethics and Value, 3(5), 140-143.
• Mukherjee, H., Dhar, A., Ghosh, M., Obaidullah, S. M., Santosh, K. C., Phadikar, S., & Roy, K. (2020). Music chord inversion shape identification with LSTM-RNN. Procedia Computer Science, 167, 265-274. https://doi.org/10.1016/j.procs.2020.03.327
• Mukherjee, H., Dhar, A., Paul, B., Obaidullah, S. M., Santosh, K. C., Phadikar, S., & Roy, K. (2020). Deep learning-based music chord family identification. In S. Santosh (Ed.), Advances in intelligent systems and computing (Vol. 1034, pp. 169-178). Springer. https://doi.org/10.1007/978-981-15-1084-7_18
• Mushtaq, Z., & Su, S. F. (2020). Environmental sound classification using a regularized deep convolutional neural network with data augmentation. Applied Acoustics, 167, 107389. https://doi.org/10.1016/j.apacoust.2020.107389
• Nataliia, S. (2019). Lifelong learning: Music adult education. Continuing Professional Education Theory and Practice, 1, 17-22. https://doi.org/10.28925/1609-8595.2019.1.1722
• Preda-Uliţă, A. (2016). Improving children’s executive functions by learning to play a musical instrument. Bulletin of the Transilvania University of Brașov, Series VIII: Performing Arts, 9(2), 37-47.
• Rao, Z., & Feng, C. (2023). Automatic identification of chords in noisy music using temporal correlation support vector machine. IAENG International Journal of Computer Science, 50(2), 179-185.
• Roden, I., Friedrich, E. K., Etzler, S., Frankenberg, E., Kreutz, G., & Bongard, S. (2021). Development and preliminary validation of the Emotions While Learning an Instrument Scale (ELIS). PLoS ONE, 16(8), e0255019. https://doi.org/10.1371/journal.pone.0255019
• Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929-1958.
• Stevens, S. S., Volkmann, J., & Newman, E. B. (1937). A scale for the measurement of the psychological magnitude pitch. Journal of the Acoustical Society of America, 8, 185-190. https://doi.org/10.1121/1.1915893
• Tsugawa, S. (2022). Intergenerational music teaching and learning among preservice music teachers and senior adult musicians. Research Studies in Music Education, 44(1), 52-69. https://doi.org/10.1177/1321103X20977541
• Upitis, R., Abrami, P. C., & Brook, J. (2012). Learning to play a musical instrument with a digital portfolio tool. Journal of Instructional Pedagogies, 9, 1-12.
• Yan, H. (2022). Design of online music education system based on artificial intelligence and multiuser detection algorithm. Computational Intelligence and Neuroscience, 2022(1), 1-12. https://doi.org/10.1155/2022/9083436
• Zaman, K., Sah, M., Direkoglu, C., & Unoki, M. (2023). A survey of audio classification using deep learning. IEEE Access, 11, 106620-106649. https://doi.org/10.1109/ACCESS.2023.3318015