The effectiveness of the haptic glove for piano education

Abstract

Educational technology plays an important role in learning processes; provides the environment, practice and materials that teachers need for cognitive, affective, and psychomotor areas. Haptic technologies offer significant opportunities especially in the field of psychomotor learning. With this research, it is aimed to explore the effectiveness of a haptic glove for piano education. A haptic glove was used in the right hand and vibrations were sent to the learner. Within the scope of the study, an enriched pattern was used which combined both qualitative and quantitative methods. In order to compare the skills of the students such as playing the notes correctly and with correct fingers, an equal-time sample model was used. In the qualitative method, however, process focused interviews were conducted with the students. The interviews and the implementation process were carried out with video recordings. Students from the Department of Computer Education and Instructional Technology and Music Education participated in the study voluntarily. According to the results of the study, there was no difference between the implementation of haptic glove or piano with respect to the criteria such as playing the notes correctly and playing with the correct fingers. However, it was determined that the glove could be a good exercise tool and finger opener for opening the fingers before the piano playing and could help to better understand the finger numbers and according to opinions haptic glove had effects such as facilitating learning, saving time, assisting with studies and solving the song in less than the given time.

Keywords

• human-computer interface
• haptic learning
• haptic gloves
• piano education

Piyano eğitiminde haptik eldivenin etkililiği

Abstract

Eğitim teknolojisi öğrenme süreçlerinde önemli bir rol oynamakta bilişsel, duyuşsal, psikomotor alanlar için öğretmenlerin ihtiyaç duydukları ortam, uygulama ve materyalleri sağlamaktadır. Haptik teknolojiler, özellikle psikomotor öğrenme alanında önemli olanaklar sunmaktadır. Bu araştırmayla, piyano çalmayı yeni öğrenenler için günlük etkinlikler sırasında pasif haptik öğrenme yoluyla dokunsal arayüz entegre edilmiş bir eldivenin piyano eğitimi için etkililiğinin sınanması amaçlanmıştır. Bu amaç doğrultusunda sağ elde kullanılan ve öğrenene birim zamanda tek nota tuşlanacak şekilde titreşimler yollayan bir eldiven kullanılmıştır. Araştırma kapsamında nitel ve nicel yöntemlerin bir arada kullanıldığı zenginleştirilmiş desen kullanılmıştır. Öğrencilerin notaları ve nota sürelerini doğru çalabilme ve doğru parmaklarla çalabilme gibi becerilerini karşılaştırmak amacıyla eşit – zaman örneklemli model kullanılmıştır. Bununla birlikte nitel yöntemde ise öğrenciler ile sürece yönelik görüşmeler yapılmıştır. Görüşmeler ve uygulama süreci video kayıtları ile gerçekleştirilmiştir. Çalışmaya Bilgisayar ve Öğretim Teknolojileri Eğitimi (BÖTE) ve Müzik Eğitimi bölümünden öğrenciler gönüllü olarak katılmıştır. Çalışmanın sonucunda katılımcıların notaları doğru çalabilme ve doğru parmaklarla çalabilme gibi ölçütlerine göre haptik eldiven veya piyano ile uygulama arasında bir farklılık bulunamamıştır. Ancak, eldivenin piyano çalmadan önce parmakları açmada iyi bir alıştırma aracı ve parmak açıcı olabileceği ve parmak numaralarının daha iyi kavranmasında yardımcı olabileceği ve görüşlere göre eldivenin öğrenmeyi kolaylaştırma, zamandan tasarruf etme, çalışmalara yardımcı olma ve verilen süreden daha kısa sürede şarkıyı çözme gibi etkilerinin olduğu belirlenmiştir.

Keywords

• insan bilgisayar etkileşimi
• haptik öğrenme
• haptik eldiven
• piyano eğitimi

References

1. Akkuş, R. (2000). Instrument study behaviours. The Journal of Buca Faculty of Education, 12, 107-111.
2. Ashimori, K., & Igarashi, H. (2017, June). Skill assist system for musical instruments by skilled players force feedback. In 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE) (pp. 2008-2013). IEEE.
3. Ashimori, K., & Igarashi, H. (2018a, March). Complemental Learning Assist for Musical Instruments by Haptic Presentation. In 2018 IEEE 15th International Workshop on Advanced Motion Control (AMC) (pp. 175-180). IEEE.
4. Ashimori, K., & Igarashi, H. (2018b, July). Complementary Learning Assist System: Guitar Performance Assist by Haptic Presentation. In International Conference on Human-Computer Interaction (pp. 11-16). Springer, Cham.
5. Aveni, T. J., Seim, C., & Starner, T. (2019, July). A preliminary apparatus and teaching structure for passive tactile training of stenography. In 2019 IEEE World Haptics Conference (WHC) (pp. 383-388). IEEE.
6. Aydıner Uygun, M. & Kılınçer, Ö. (2018). Examination of strategies professional music education Students use while practicing and learning instrumental music. Mehmet Akif Ersoy University Journal of Education Faculty, 47, 317-339.
7. Barfield, W. (2010). The use of haptic display technology in education. Themes in Science and Technology Education, 2 (1-2), 11-30.
8. Baumann, A., & Grinstein, G. (2008). Haptic Sound: Expressive Control of Song Playback Using Haptics. International Community for Auditory Display. Proceedings of the 14th International Conference on Auditory Display, Paris, France.

9. Bircan, K. E. (2018). Music teacher candidates the strategies of musical learning investigation from different variables. [Unpublished Master of Science Thesis]. Ondokuz Mayis University, Graduate School of Educational Sciences, Samsun.
10. Büyüköztürk, Ş., Çakmak, E. K., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2011). Scientific research methods. Pegem Akademi, Ankara.
11. Civelek, T., Ucar, E., Ustunel, H., & Aydın, M. K. (2014). Effects of a Haptic Augmented Simulation on K-12 Students' Achievement and their Attitudes towards Physics. Eurasia Journal of Mathematics, Science & Technology Education, 10 (6).
12. El Saddik, A., Orozco, M., Eid, M., & Cha, J. (2011). Haptics technologies: bringing touch to multimedia. Springer Science & Business Media.
13. Ercan, N. (2006). The importance of practicing in piano education and some suggestions. Mehmet Akif Ersoy University Journal of Education Faculty, 7 (12), 104-108.
14. Grindlay, G. C. (2007). The impact of haptic guidance on musical motor learning. [Doctoral dissertation, Massachusetts Institute of Technology.]
15. Grindlay, G. C. (2008, March). Haptic guidance benefits musical motor learning. In 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (pp. 397-404). IEEE.
16. Hallam, S. (1997). What do we know about practicing? Towards a model synthesising the research literature. Does practice make perfect? Current theory and research on instrumental music practice, 1, 179-231.
17. Hamza-Lup, F. G., & Stanescu, I. A. (2010). The haptic paradigm in education: Challenges and case studies. The Internet and Higher Education, 13(1-2), 78-81.
18. Hsiao, C. P., Li, R., Yan, X., & Do, E. Y. L. (2015, January). Tactile teacher: Sensing finger tapping in piano playing. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 257-260). ACM.
19. Huanhuan, L. (2009). Computer assisted music instrument tutoring applied to violin practice [Doctoral dissertation, National University of Singapore].
20. Huang, K., Do, E. Y. L., & Starner, T. (2008). PianoTouch: A wearable haptic piano instruction system for passive learning of piano skills. In Wearable Computers, 2008. ISWC 2008. 12th IEEE International Symposium (pp. 41-44). IEEE.
21. Huang, K., Starner, T., Do, E., Weinberg, G., Kohlsdorf, D., Ahlrichs, C., & Leibrandt, R. (2010). Mobile music touch: mobile tactile stimulation for passive learning. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 791–800). Atlanta, GA: Special Interest Group on Computer-Human Interaction.
22. Huberman, A. M., & Miles, M. B. (1994). Data management and analysis methods, in N. Denzin and Y. Lincoln (eds) Handbook of Qualitative Research, pp. 428-444. London: Sage.
23. Jain, A. (2011). Mobile Tactile Stimulation for Passive Haptic Learning of Simple Melodies. In Partial Fulfillment of the Requirements for the Degree B.S. Computer Science with Research Option in the College of Computing Georgia Institute of Technology.
24. Karasar, N. (2007). Scientific research methods. Nobel Yayınevi, Ankara.
25. Kılınçer, Ö. (2013). Evaluation of the learning strategies used in piano class in terms of different variables. [Unpublished Doctoral Dissertation], Erciyes University, Institute of Fine Arts, Kayseri.
26. Kılınçer, Ö. & Aydıner Uygun, M. (2017). The examination of strategies of amateur and professional musicians’ use while practicing and learning music in the instruments. International Journal of Eurasia Social Sciences, 8, 28, 523-534.
27. Kohlsdorf, D., & Starner, T. (2010, October). Mobile music touch: The effect of primary tasks on passively learning piano sequences. In International Symposium on Wearable Computers (ISWC) 2010 (pp. 1-8). IEEE.
28. Luzhnica, G., Veas, E., & Seim, C. (2018, October). Passive haptic learning for vibrotactile skin reading. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (pp. 40-43). ACM.
29. Meneses, J. (2014). Haptic Keyboard Aid. A Major Qualifying Project Report. In partial fulfillment of the requirements for the Degree of Bachelor of Science. The Faculty of the Worcester Polytechnic Institute.
30. Nicolau, H., Guerreiro, J., Guerreiro, T., & Carriço, L. (2013, October). UbiBraille: designing and evaluating a vibrotactile Braille-reading device. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (p. 23). ACM.
31. Nielsen, J. (2012, June 03). How many test users in a usability study? Nielsen Norman Group. https://www.nngroup.com/articles/how-many-test-users
32. Pala, F. K. & Mihci Turker, P. (2019). Developing a haptic glove for basic piano education. World Journal on Educational Technology: Current Issues. 11(1), 038–047.
33. Pescara, E., Polly, T., Schankin, A., & Beigl, M. (2019, September). Reevaluating passive haptic learning of morse code. In Proceedings of the 23rd International Symposium on Wearable Computers (pp. 186-194). ACM.
34. Rodríguez, J. L., Velázquez, R., Del-Valle-Soto, C., Gutiérrez, S., Varona, J., & Enríquez-Zarate, J. (2019). Active and passive haptic perception of shape: passive haptics can support navigation. Electronics, 8(3), 355.
35. Seim, C., Chandler, J., DesPortes, K., Dhingra, S., Park, M., & Starner, T. (2014, September). Passive haptic learning of Braille typing. In Proceedings of the 2014 ACM International Symposium on Wearable Computers (pp. 111-118). ACM.
36. Seim, C., Estes, T., & Starner, T. (2015, June). Towards passive haptic learning of piano songs. In World Haptics Conference (WHC), 2015 IEEE (pp. 445-450). IEEE.
37. Seim, C., Reynolds-Haertle, S., Srinivas, S., & Starner, T. (2016, September). Tactile taps teach rhythmic text entry: passive haptic learning of morse code. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (pp. 164-171). ACM.
38. Takegawa, T., Terada, T., and Tsukamoto, M. A. (2012, September). Piano Learning Support System considering Rhythm. Proceedings of the 38th International Computer Music Conference (ICMC), Ljubljana, Slovenia, 2012, p. 325-332.
39. Yıldırım A. ve Şimşek, H. (2006). Qualitative Research Methods. Seçkin Yayınevi, Ankara.
40. Xia, G., Jacobsen, C., Chen, Q., Yang, X., & Dannenberg, R. (2018, June). ShIFT: A Semi-haptic Interface for Flute Tutoring. In Proceedings of the international conference on new interfaces for musical expression, Blacksburg, VA, USA, 2018.
41. Zhang, Y., Li, Y., Chin, D., & Xia, G. (2019, June). Adaptive Multimodal Music Learning via Interactive-haptic Instrument. In Proceedings of the international conference on new interfaces for musical expression, Porto Alegre, Brazil, 2019, p. 140–145.