Exploring Musical Affordance of a Virtual Reality Malay Gamelan Through Qualitative Study

Abstract

Many virtual gamelans are designed to give users alternative access to learn about the gamelan instruments. As the gamelan instruments are physically big and heavy, there is typically limited access to ordinary people who want to learn about the instruments. In recent years, there have been many efforts to virtualize gamelan instruments using various advanced technologies to attract the younger generations and make them accessible. New musical technologies often afford novel interactions that are unattainable with their physical counterparts. This paper investigates the affordance of a virtual reality (VR) gamelan instrument called Air Bonang. Using the qualitative methods under the User-Centered Design (UCD) approach, the affordance of Air Bonang is explored through a ‘breaching experiment’ with the Malay gamelan community. Findings showed that the affordance of the VR gamelan instrument is contextual to three aspects: traditional, exploratory, and collective. The traditional context was based on how Air Bonang was played like the actual bonang, while the exploratory context presents a novel virtual interaction with the instrument. The collective context was afforded through ensemble playing of the instrument. This encompasses how the VR instrument is used in different musical settings regarding its purpose, practicality, relevancy, and reliability.

Keywords

• Musical affordance
• Virtual gamelan
• Malay gamelan
• Virtual musical instrument

References

1. Gibson, J. (1977). The theory of affordances. Hilldale, USA. https://doi.org/10.1075/lsse.2.03bli
2. Chong, I., & Proctor, R. W. (2020). On the Evolution of a Radical Concept: Affordances According to Gibson and Their Subsequent Use and Development. Perspectives on Psychological Science, 15(1), 117–132. https://doi.org/10.1177/1745691619868207
3. Norman, D. A. (1999). Affordance, conventions, and design. Interactions, 38–43. https://doi.org/10.1145/301153.301168
4. IxDF, I. D. F. (2016). What Are Affordances. https://www.interaction-design.org/literature/topics/affordances#:~:text=Psychologist James Gibson coined “affordance,based on users’’ physical capabilities.&text=Human-computer interaction (HCI),actions which users consider possible.”
5. Choi, I. (2022). An Introduction to Musical Interactions. Multimodal Technologies and Interaction, 6(1), 4. https://doi.org/10.3390/mti6010004
6. Çamcı, A., Vilaplana, M., & Wang, R. (2020). Exploring the Affordances of VR for Musical Interaction Design with VIMEs. Proceedings of the International Conference on New Interfaces for Musical Expression, 144–150.
7. Wakefield, G., Palumbo, M., & Zonta, A. (2020). Affordances and Constraints of Modular Synthesis in Virtual Reality. Proceedings of the International Conference on New Interfaces for Musical Expression, 653–657.
8. Huovinen, E., & Rautanen, H. (2020). Interaction affordances in traditional instruments and tablet computers: A study of children’s musical group creativity. Research Studies in Music Education, 42(1), 94–112. https://doi.org/10.1177/1321103X18809510

9. Hassan, A. K., Teuku Iskandar, T. U. I., & Che Adeni, M. F. (2022). Gamelan Melayu: Asas Permainan. Akademi Seni Budaya dan Warisan Kebangsaan.
10. Teuku Iskandar, T. U. I., & Salleh, M. (2021). Malay Gamelan: Playing Techniques of the Keromong and Gambang. International Journal of Creative Industries, 3(8), 01–12. https://doi.org/10.35631/ijcrei.38001
11. Tanaka, A., Altavilla, A., & Spowage, N. (2012). Gestural musical affordances. Proceedings of the 9th Sound and Music Computing Conference, SMC 2012.
12. Godøy, R. I., & Leman, M. (2010). Musical Gestures: Sound, Movement, and Meaning. Routledge.
13. Schnell, N., Bevilacqua, F., Robaszkiewicz, S., & Schwarz, D. (2015). Collective Sound Checks: Exploring intertwined sonic and social affordances of mobile web applications. TEI 2015 - Proceedings of the 9th International Conference on Tangible, Embedded, and Embodied Interaction, 685–690. https://doi.org/10.1145/2677199.2688808
14. Hinojosa, L. P. C. (2020). Electronic Khipu : Thinking in Experimental Sound from an Ancestral Andean Interface. Computer Music Journal, 44(2), 39–54. https://doi.org/10.1162/COMJ
15. Ham, J., Woessner, U., Kieferle, J., & Harvey, L. (2019). Affordances and Performance Opportunities of a Virtual Drumming. ECAADe, 2, 441–448.
16. Saffian, K. A., Norowi, N. M., Abdullah, L. N., Sulaiman, P. S., & Musib, A. F. (2022). Playing Gamelan Bonang in the Air: User Requirements for Designing a Digital Musical Instrument for the Malay Bonang. Malaysian Journal of Music, 11(1), 68–83. https://doi.org/10.37134/mjm.vol11.1.5.2022
17. Shah, S. M., & Poheng, J. J. (2021). The Changing Landscape of the Malay Gamelan and its Implications on Music Education in Malaysia. Environment-Behaviour Proceedings Journal, 6(SI5), 137–141. https://doi.org/10.21834/ebpj.v6isi5.2939
18. Nilsson, T., Crabtree, A., Fischer, J., & Koleva, B. (2019). Breaching the future: understanding human challenges of autonomous systems for the home. Personal and Ubiquitous Computing, 287–307. https://doi.org/10.1007/s00779-019-01210-7
19. Bryan-Kinns, N., & Reed, C. N. (2023). A Guide to Evaluating the Experience of Media and Arts Technology. In A. L. Brooks (Ed.), Creating Digitally. Shifting Boundaries: Arts and Technologies - Contemporarory Applications and Concepts. Springer.
20. Creswell, J., & Piano Clark, V. L. (2018). Designing and Conducting Mixed Methods Research.
21. Leedy, P., & Ormrod, J. (2016). Practical Research: Planning and Design. Pearson.
22. Crabtree, A. (2004). Design in the absence of practice: Breaching experiments. DIS2004 - Designing Interactive Systems: Across the Spectrum, 59–68.
23. Deacon, T., Stockman, T., & Barthet, M. (2017). User experience in an interactive music virtual reality system: An exploratory study. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): Vol. 10525 LNCS. https://doi.org/10.1007/978-3-319-67738-5_12
24. Marvasti, A. (2014). Analysing Observation. In The SAGE Handbook of Qualitative Data Analysis (pp. 354–367). SAGE.
25. Chan, C. S. C. (2013). Beyond Rhythm: Exploring Infinite Possibilities in Music, Movement and Choreography-Hands Percussion Malaysia. Malaysian Music Journal, 2(2), 104–117.
26. Serafin, S., Erkut, C., Kojs, J., Nilsson, N. C., & Nordahl, R. (2016). Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions. Computer Music Journal, 40(3), 22–40. https://doi.org/10.1162/COMJ
27. Zellerbach, K. C., & Roberts, C. (2022). A Framework for the Design and Analysis of Mixed Reality Musical Instruments. Proceedings of the International Conference on New Interfaces for Musical Expression, 1–21. https://doi.org/10.21428/92fbeb44.b2a44bc9
28. Demos, A. P., & Palmer, C. (2023). Social and nonlinear dynamics unite: musical group synchrony. Trends in Cognitive Sciences, 1–11. https://doi.org/10.1016/j.tics.2023.05.005
29. Turchet, L. (2023). Musical Metaverse: vision, opportunities, and challenges. Personal and Ubiquitous Computing, 0123456789. https://doi.org/10.1007/s00779-023-01708-1
30. Syukur, A., Andono, P. N., & Syarif, A. M. (2024). Gamelan Land: A Multiplayer Virtual Reality Game based on a Social Presence Approach. Journal of Metaverse, 4(1), 1–10. https://doi.org/10.57019/jmv.1334412
31. Syukur, A., Andono, P. N., Hastuti, K., & Syarif, A. M. (2023). Immersive and Challenging Experiences through A Virtual Reality Musical Instruments Game: An Approach to Gamelan Preservation.
32. Aryadana, I. P. S., Wiranatha, A. A. K. A. C., & Dharmadi, I. P. A. (2019). Aplikasi Virtual Reality Gamelan Gong Kebyar Tradisional Bali Berbasis Android. Jurnal Ilmiah Merpati (Menara Penelitian Akademika Teknologi Informasi), 7(2), 162. https://doi.org/10.24843/jim.2019.v07.i02.p08
33. Dwipayana, K., Wirawan, I. M. A., & Sindu, I. geed P. (2019). Go-Byar Based on Virtual Reality for the Learning Media of Gamelan. 25(2), 229–236. https://doi.org/10.21831/jptk.v25i2.26182