A brain-computer interface with gamification in the Metaverse

Öz

This study contributes to our understanding of the Metaverse by presenting a case study of the implementation of brain-computer interface supported game-based engagement in a Virtual Environment (VE). In VE, individuals can communicate with anyone, anywhere, anytime, without any limits. This situation will increase the barrier-free living standards of disabled people in a more accessible environment. A virtual world of well-being awaits these individuals, primarily through gamified applications thanks to Brain-Computer Interfaces. Virtual environments in the Metaverse can be infinitely large, but the user's movement in a virtual reality (VR) environment is constrained by the natural environment. Locomotion has become a popular motion interface as it allows for full exploration of VE. In this study, the teleport method from locomotion methods was used. To teleport, the user selects the intended location using brain signals before being instantly transported to that location. Brain signals are decomposed into alpha, beta, and gamma bands. The features of each band signal in Time, frequency, and time-frequency domains are extracted. In this proposed method, the highest performance of binary classification was obtained in the frequency domain and the Alpha band. Signals in the alpha band were tested in the domains Time, Frequency, and Time-Frequency. Teleport operations are faster with Time and more stable with the frequency domain. However, the Hilbert-Huang Transform (HHT) method used in the Time-Frequency domain could not respond adequately to real-time applications. All these analyses were experienced in the Erzurum Virtual Tour case study, which was prepared to promote cultural heritage with the gamification method.

Anahtar Kelimeler

• Metaverse
• locomotion
• teleport
• EEG
• gamification
• BCI

Kaynakça

1. P. Milgram, H. Takemura, A. Utsumi, and F. Kishino, “Augmented reality: A class of displays on the reality-virtuality continuum,” in Telemanipulator and telepresence technologies, 1995, vol. 2351, pp. 282–292.
2. G. C. Burdea and P. Coiffet, Virtual reality technology. John Wiley & Sons, 2003.
3. C. Boletsis, “The new era of virtual reality locomotion: A systematic literature review of techniques and a proposed typology,” Multimodal Technol. Interact., vol. 1, no. 4, pp. 1–17, 2017, doi: 10.3390/mti1040024.
4. E. Bozgeyikli, A. Raij, S. Katkoori, and R. Dubey, “Point & Teleport locomotion technique for virtual reality,” CHI Play 2016 - Proc. 2016 Annu. Symp. Comput. Interact. Play, pp. 205–216, 2016, doi: 10.1145/2967934.2968105.
5. M. J. Habgood, D. Wilson, D. Moore, and S. Arapont, “Hci lessons from playstation VR.,” in Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play, 2017, pp. 125–135.
6. S. H. Pyo, H. S. Lee, B. M. Phu, S. J. Park, and J. W. Yoon, “Development of an Fast-Omnidirectional Treadmill (F-ODT) for Immersive Locomotion Interface,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 760–766. doi: 10.1109/ICRA.2018.8460669.
7. M. Palaus, E. M. Marron, R. Viejo-Sobera, and D. Redolar-Ripoll, “Neural basis of video gaming: A systematic review,” Front. Hum. Neurosci., vol. 11, p. 248, 2017.
8. S. Mystakidis et al., “Design, Development, and Evaluation of a Virtual Reality Serious Game for School Fire Preparedness Training,” Education Sciences , vol. 12, no. 4. 2022. doi: 10.3390/educsci12040281.

9. C. Prandi, P. Salomoni, and S. Mirri, “Gamification in Crowdsourcing Applications.” 2019.
10. J. R. Wolpaw and E. W. Wolpaw, “Brain-computer interfaces: something new under the sun,” Brain-computer interfaces Princ. Pract., vol. 14, 2012.
11. S. Ghosh, “Brain Computer Interface: Definition, Tools and Applications,” 2020. https://aithority.com/machine-learning/neural-networks/brain-computer-interface-definition-tools-and-applications/
12. E. Niedermeyer and F. H. L. da Silva, Electroencephalography: basic principles, clinical applications, and related fields. Lippincott Williams & Wilkins, 2005.
13. M. Teplan and others, “Fundamentals of EEG measurement,” Meas. Sci. Rev., vol. 2, no. 2, pp. 1–11, 2002.
14. B. He and L. Ding, “Electrophysiological mapping and neuroimaging,” in Neural engineering, Springer, 2013, pp. 499–543.
15. J. J. Newson and T. C. Thiagarajan, “EEG frequency bands in psychiatric disorders: a review of resting state studies,” Front. Hum. Neurosci., vol. 12, p. 521, 2019.
16. M. S. Özerdem and Ö. Emhan, “Yukarı-Aşağı imleç hareketlerine ilişkin EEG kayıtlarında en etkin kanalın belirlenmesi,” Dicle Üniversitesi Mühendislik Fakültesi Mühendislik Derg., vol. 8, no. 3, pp. 587–597, 2017.
17. S. Baceviciute, T. Terkildsen, and G. Makransky, “Remediating learning from non-immersive to immersive media: Using EEG to investigate the effects of environmental embeddedness on reading in Virtual Reality,” Comput. \& Educ., vol. 164, p. 104122, 2021.
18. P. Batres-Mendoza et al., “Quaternion-Based Signal Analysis for Motor Imagery Classification from Electroencephalographic Signals,” Sensors (Basel)., vol. 16, no. 3, p. 336, Mar. 2016, doi: 10.3390/s16030336.
19. B. Hjorth, “EEG analysis based on time domain properties,” Electroencephalogr. Clin. Neurophysiol., vol. 29, no. 3, pp. 306–310, 1970, doi: https://doi.org/10.1016/0013-4694(70)90143-4.
20. M. Ahn, H. Cho, S. Ahn, and S. C. Jun, “High Theta and Low Alpha Powers May Be Indicative of BCI-Illiteracy in Motor Imagery,” PLoS One, vol. 8, no. 11, p. e80886, Nov. 2013, [Online]. Available: https://doi.org/10.1371/journal.pone.0080886
21. J. Gruenwald, C. Kapeller, C. Guger, H. Ogawa, K. Kamada, and J. Scharinger, “Comparison of Alpha/Beta and high-gamma band for motor-imagery based BCI control: A qualitative study,” in 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2017, pp. 2308–2311. doi: 10.1109/SMC.2017.8122965.
22. M. Şeker and M. S. Özerdem, “İyi – kötü koku uyartılarının EEG aktivitesine etkisinin Welch metodu ile incelenmesi,” Dicle Üniversitesi Mühendislik Fakültesi Mühendislik Derg., vol. 8, no. 3, pp. 547–553, 2017.
23. E. Langbehn, P. Lubos, and F. Steinicke, “Evaluation of locomotion techniques for room-scale vr: Joystick, teleportation, and redirected walking,” in Proceedings of the Virtual Reality International Conference-Laval Virtual, 2018, pp. 1–9.