Immersive and Challenging Experiences through A Virtual Reality Musical Instruments Game: An Approach to Gamelan Preservation

Abstract

This study proposes the preservation of traditional Javanese musical instruments known as Gamelan using an immersive and challenging approach in a virtual reality game format. In an effort to achieve a challenging experience, the gamification is designed by observing musical instrument games selected based on their popularity in the Google Play and the App Store. Oculus Quest 2 which is a stand-alone Head Mounted Displays that provides a high-quality immersive display and more flexibility for users to move is chosen to present the game. Overall, the research target can be achieved based on the evaluation using Mean Opinion Scores. The degree of immersion, presence, likeable, challenging, and the value of cultural preservation contained in the game reaches a range of values between good and excellent. Meanwhile, cybersickness still seems to be a chore for developers to make Head Mounted Displays devices more comfortable.

Keywords

• Virtual reality
• Virtual musical instrument
• Cultural heritage preservation
• Gamelan

References

1. R. Zender, A.H. Knoth, M.H. Fischer, and Ul. Lucke, “Potentials of virtual reality as an instrument for research and education”, i-com, 18(1), pp. 3-15, 2019. Doi: 10.1515/icom-2018-0042.
2. J. Granzow, and A. Camci, “Recreating a rare instrument using VR and fabrication: A hyperreal instrument case study”. Forum Acusticum, Lyon, France, pp. 655-658, 2020. Doi:10.48465/fa.2020.0847. hal-03234058.
3. Q. Jia, “Application of Chinese traditional musical instruments”, Scientific and Social Research, 3(1), pp. 80-84, 2021. Doi: 10.36922/ssr.v3i1.1063.
4. S. Weech, S. Kenny, and M. Barnett-Cowan, “Presence and cybersickness in virtual reality are negatively related: A review”, Front. Psychol. 10(158), 2019. Doi: 10.3389/fpsyg.2019.00158.
5. Z. Lv, “Virtual reality in the context of Internet of Things”, Neural Comput & Applic, 32, pp. 9593–9602, 2020. Doi: 10.1007/s00521-019-04472-7.
6. J. Radianti, T.A. Majchrzak, J. Fromm, and I. Wohlgenannt, “A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda”, Computers & Education, 147, 103778, 2020. Doi: 10.1016/j.compedu.2019.103778.
7. J. Brett, T. Gladwell, T, N. Xu, P. Amelidis, T. Davis, and C. Gatzidis, “Developing games for the purposes of rote learning for keyboard and piano”, IEEE Conference on Games (CoG), pp. 724-727, 2020. Doi: 10.1109/CoG47356.2020.9231779.
8. S. Serafin, C. Erkut, J. Kojs, N.C. Nilsson, and R. Nordahl, “Virtual reality musical instruments: State of the art, design principles, and future directions”, Computer Music Journal, vol. 40(3), pp. 22–40, 2016. Doi:10.1162/COMJ a 00372.

9. A. Çamcı, and R. Hamilton, “Virtual 3D environments as composition and performance spaces”, Journal of New Music Research, 49(1), pp. 104-113, 2020. Doi: 10.1080/09298215.2019.1703013.
10. K. Okasaki, and M. Makino, “A VR-based piano self-training portable system on standalone HMD”, Proc. SPIE 11766, International Workshop on Advanced Imaging Technology (IWAIT) 2021, 117662I, 2021. Doi: 10.1117/12.2591031.
11. K. Brunnström, E. Dima, T. Qureshi, M. Johanson, M. Andersson, and M. Sjöström, “Latency impact on quality of experience in a virtual reality simulator for remote control of machines”, Signal Processing: Image Communication, 89 (2020) 116005, 2020. Doi: 10.1016/j.image.2020.116005.
12. S. Cottrell, and J. Howell, “Reproducing musical instrument components from manufacturers technical drawings using 3D printing: Boosey & Hawkes as a case study”, Journal of New Music Research, 48(5), pp. 449-457, 2019. Doi: 10.1080/09298215.2019.1642362.
13. Y.P. Yuda, and M.N.L. Azis, “3D modeling the gamelan of saron as a documentation of cultural heritage preservation efforts”, Journal of Physics: Conference Series, 1375, 2019. Doi: 10.1088/1742-6596/1375/1/012036.
14. A.Z. Fanani, K. Hastuti, A.M. Syarif, and A.R. Mulyana, “Rule-based interactive learning application model on how to play music instruments”, International Journal of Emerging Technologies in Learning, 15(15), pp. 52-63, 2020. Doi: 10.3991/ijet.v15i15.11486.
15. F. Permana, H. Tolle, F. Utaminingrum, and R. Dermawi, “Development of augmented reality (AR) based Gamelan simulation with leap motion control”, International Journal of Interactive Mobile Technologies, 13(12), pp. 120–135, 2019. Doi: 10.3991/ijim.v13i12.9270
16. C.F. Setiyawan, and M. Shobri, “Blocking Gamelan instruments frequency in virtual reality”, Proc. SPIE 11515, International Workshop on Advanced Imaging Technology, 1151526, 2020. Doi: 10.1117/12.2566939.
17. A. Z. Fanani, K. Hastuti, A.M. Syarif, P.W. Harsanto, “Challenges in developing virtual reality, augmented reality and mixed-reality applications: Case Studies on a 3D-based tangible cultural heritage conservation”, International Journal of Advanced Computer Science and Applications, vol. 12, issue 11, pp. 219-227, 2021. Doi:10.14569/IJACSA.2021.0121126.
18. G. Makransky, and G.B. Petersen, G.B, “”The cognitive affective model of immersive learning (camil): A theoretical research-based model of learning in immersive virtual reality”, Educational Psychology Review, vol. 33, pp. 937–958, 2021. Doi: 10.1007/s10648-020-09586-2.
19. A.M. Syarif, A. Azhari, S. Suprapto, and K. Hastuti, “Human and computation-based music representation for Gamelan music”, Malaysian Journal of Music, vol. 9, pp. 82-100, 2020. Doi: 10.37134/mjm.vol9.7.2020.
20. https://www.qualcomm.com/research/extended-reality (accessed on January 5, 2022).
21. H. Park, J. Kim, S. Bang, and W. Woo, “The effect of applying film-induced tourism to virtual reality tours of cultural heritage sites”, 3rd Digital Heritage International Congress held jointly with 24th International Conference on Virtual Systems & Multimedia, San Francisco, CA, USA, pp. 1-4, 2018, Doi: 10.1109/DigitalHeritage.2018.8810089.
22. R. Zamora-Musa, J. Vélez, and H. Paez-Logreira, “Evaluating learnability in a 3D heritage tour”, Presence: Virtual and Augmented Reality, 26(4), pp. 366-377, 2017. Doi: 10.1162/PRES_a_00305.
23. https://sumi.uxp.ie/about/whatis.html (accessed on January 9, 2022).
24. D. Pal, C. Arpnikanondt, S. Funilkul, and V. Varadarajan, “User experience with smart voice assistants: The accent perspective”, 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), pp. 1-6, 2019. Doi: 10.1109/ICCCNT45670.2019.8944754.
25. G. Meynet, J. Digne, and G. Lavoué, “PC-MSDM: A quality metric for 3D point clouds”, 11th International Conference on Quality of Multimedia Experience (QoMEX), pp. 1-3, 2019. Doi: 10.1109/QoMEX.2019.8743313.
26. P.C. Madhusudana, and R. Soundararajan, “Subjective and objective quality assessment of stitched images for virtual reality”, IEEE Transactions on Image Processing, 28(11), pp. 5620-5635, 2019. Doi: 10.1109/TIP.2019.2921858.
27. S. Katsigiannis, J. Scovell, N. Ramzan, L. Janowski, P. Corriveau, M.A. Saad, and G. Van Wallendael, “Interpreting MOS scores, when can users see a difference? understanding user experience differences for photo quality”, Quality and User Experience, 3(6), 2018, Doi: 10.1007/s41233-018-0019-8.