Bir Bulmaca Oyununda VR ve Masaüstü Oyun Kullanıcı Deneyimi Kıyaslaması: “Keep Talking and Nobody Explodes”

Yıl 2020, , 180 - 204, 17.10.2020

Mehmet Berkman , Güven Çatak , Mıstık Eremektar

https://doi.org/10.5824/ajite.2020.03.008.x

Öz

Çağdaş oyun üretim süreci Entegre Geliştirme Ortamı (IDE) uygulamalarına dayandığından, geliştiricilerin oyunlarının hem VR hem de masaüstü platformları için birden çok sürümünü oluşturmaları daha kolaydır. Bu drum, araştırmacılara da, sanal gerçeklik uygulamalarının kullanıcı deneyimini karşılaştırmalı olarak değerlendirme fırsatı sağlamaktadır. Bu çalışmada, oyun başarısı ve oynanış süresi objektif ölçümlerinin yanında, oyun kullanıcısı deneyimini değerlendirmek için birçok öznel ölçümü de kullanarak bir bulmaca oyununu masaüstü ve VR'yi karşılaştırması yaparak değerlendirdik. GUESS ölçeğinin (Oyun Kullanıcı Deneyimi Memnuniyeti Ölçeği) seçilen boyutlarına ek olarak, mevcudiyeti ölçmek için MEC-SPQ (Ölçüm Etkileri Koşulları - Mekansal Mevcudiyet Anketi) kullanıldı. Ayrıca, VR ve masaüstü oyun ortamlarında yürütülen aynı görevin algılanan görev karmaşıklığını kıyaslamak için NASA-TLX (NASA Görev Yük Endeksi) ölçeğinden yararlanıldı. Sonuçlar, VR ve masaüstü ortamı karşılaştırıldığında, oyuncu performansının objektif ölçümlerinde önemli bir fark olmadığını ortaya koydu. Oyun Kullanıcı Deneyimi Memnuniyeti Ölçeği, VR ve masaüstü deneyimlerinin ortalama puanları arasında anlamlı bir fark görülmedi. MEC-SPQ'nun mekansal mevcudiyetle ilgili boyutları olan Olası Eylemler ve Öz-Konum boyutları için VR ortamında anlamlı biçimde daha yüksek skorlar gözlendi. NASA-TLX ağırlıklı skorlar fiziksel yük boyutunda VR ve hayal kırıklığı boyutunda masaüstü için anlamlı olarak daha yüksektir. Sonuçlar, VR'de deneyimlenen bulmaca tabanlı bir oyunun, oyun kullanıcı deneyimi açısından daha yüksek bir memnuniyet düzeyine yol açmadığını, ancak mekansal mevcudiyet duygusunu tetiklediğini göstermektedir. Farklı kontrol şemaları nedeniyle, oyuncular kafaya takılan ekran tabanlı oyunun daha fazla fiziksel görev yükü gerektirdiğini düşünmektedirler. Bununla birlikte, oyun süresi ve oyun başarı oranı önemli ölçüde farklı değildir. Masaüstü oyunlarındaki başarısızlık, deneyim oyunculara daha tanıdık geldiği için daha yüksek hayal kırıklığına yol açmıştır. Sonuçlar daha önceki çalışmalarla kısmen uyumlu olduğu için, farklı sürükleyici teknolojilerin oyun kullanıcı deneyimi üzerindeki etkisine ilişkin kesin bir sonuç çıkarmak mümkün değildir. Oyun türünden ziyade oyun tasarımı bileşenlerine odaklanan daha tutarlı bir metodoloji yoluyla daha fazla çalışmaya ihtiyaç vardır.

Anahtar Kelimeler

Sanal Gerçeklik, Oyun Kullanıcı Deneyimi, Bulmaca Oyunu, Görev Yükü, Mevcudiyet, Bilgisayar Oyunları

Comparison of VR and Desktop Game User Experience in a Puzzle Game: “Keep Talking and Nobody Explodes”

Öz

Since the contemporary game production process is based on the Integrated Development Environment (IDE) applications, it is easier for developers to create multiple versions of their game for both VR and desktop platforms. This provided a great opportunity for researchers to conduct comparative studies to explore the user experience of the relatively novel virtual reality applications In this study, we evaluated a puzzle game through a within-subjects experiment design using objective measures of game success and gameplay duration, as well as plenty of subjective measures in order to assess game user experience, comparing desktop and VR. In addition to selected dimensions of GUESS (Game User Experience Satisfaction Scale), we employed MEC-SPQ (Measurement Effects Conditions - Spatial Presence Questionnaire) to measure presence. Furthermore, we employed NASA-TLX (NASA Task Load Index) to compare the perceived task complexity of the same task executed in VR and desktop gaming environments. Results revealed that there is not a significant difference in objective measures of player performance, comparing the VR and desktop gameplay. The Game User Experience Satisfaction Scale did not reveal any significant difference between the mean scores of VR and desktop experiences. The spatial presence related dimensions of MEC-SPQ revealed significantly higher scores of VR, for Possible Actions and Self Location dimensions. NASA-TLX weighted scores were significantly higher for VR in physical load and for desktop in frustration. Our results show that a puzzle-based game experienced in VR does not lead to a higher level of satisfaction in terms of game user experience but triggers a sense of spatial presence. Due to the different control schemes, players perceive that HMD based gameplay demands more physical task load. However, the gameplay duration and game success rate are not significantly different. The failure in desktop gameplay might have led to higher frustration, since the experience seems more familiar to players. Since the results are partially concordant with previous studies, it is not possible to make a strict conclusion on the effect caused by different immersive technologies on game user experience. Further studies are required through a more consistent methodology with a focus on game design components rather than game genre.

Anahtar Kelimeler

Virtual Reality, Game User Experience, Puzzle Game, Task Load, Presence, Video Games

Kaynakça

• Abeele, V. Vanden, Spiel, K., Nacke, L., Johnson, D., & Gerling, K. (2020). Development and validation of the player experience inventory: A scale to measure player experiences at the level of functional and psychosocial consequences. International Journal of Human Computer Studies. https://doi.org/10.1016/j.ijhcs.2019.102370
• Aksayim, A., & Berkman, M. İ. (2020). Effect of Physical Activity on VR Experience: An Experimental Study. In S. Richir (Ed.), Laval Virtual ConVRgence (VRIC) Virtual Reality International Conference - VRIC 2020. Retrieved from https://ijvr.eu/article/view/3316
• Anton, D., Kurillo, G., & Bajcsy, R. (2018). User experience and interaction performance in 2D/3D telecollaboration. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2017.12.055
• Aoki, H., Oman, C. M., Buckland, D. A., & Natapoff, A. (2008). Desktop-VR system for preflight 3D navigation training. Acta Astronautica. https://doi.org/10.1016/j.actaastro.2007.11.001
• Arns, L., Cook, D., & Cruz-Neira, C. (1999). Benefits of statistical visualization in an immersive environment. Proceedings - Virtual Reality Annual International Symposium.
• Bacim, F., Ragan, E., Scerbo, S., Polys, N. F., Setareh, M., & Jones, B. D. (2013). The effects of display fidelity, visual complexity, and task scope on spatial understanding of 3D graphs. Proceedings - Graphics Interface.
• Berkman, M. İ., Bostan, B., & Yalçın, B. (2020). Controllers in VR Game User Experience: Perceived User Performance on a VR Puzzle Game. In Contemporary Topics in Computer Graphics and Games: Selected Papers from the Eurasia Graphics Conference Series.
• Berkman, M. I., & Akan, E. (2019). Presence and Immersion in Virtual Reality. In Encyclopedia of Computer Graphics and Games (pp. 1–10). https://doi.org/10.1007/978-3-319-08234-9_162-1
• Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability Evaluation in Industry.
• Buttussi, F., & Chittaro, L. (2018). Effects of Different Types of Virtual Reality Display on Presence and Learning in a Safety Training Scenario. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2017.2653117
• Carroll, M., Osborne, E., & Yildirim, C. (2019). Effects of VR gaming and game genre on player experience. 2019 IEEE Games, Entertainment, Media Conference, GEM 2019. https://doi.org/10.1109/GEM.2019.8811554 Chirico, A., Cipresso, P., Yaden, D. B., Biassoni, F., Riva, G., & Gaggioli, A. (2017). Effectiveness of Immersive Videos in Inducing Awe: An Experimental Study. Scientific Reports. https://doi.org/10.1038/s41598-017-01242-0
• Chirico, A., Ferrise, F., Cordella, L., & Gaggioli, A. (2018). Designing awe in virtual reality: An experimental study. Frontiers in Psychology, 8(JAN), 2351. https://doi.org/10.3389/fpsyg.2017.02351
• Christensen, J. V., Mathiesen, M., Poulsen, J. H., Ustrup, E. E., & Kraus, M. (2018). Player experience in a VR and non-VR multiplayer game. ACM International Conference Proceeding Series. https://doi.org/10.1145/3234253.3234297
• Clarke, R. I., Lee, J. H., & Clark, N. (2017). Why Video Game Genres Fail: A Classificatory Analysis. Games and Culture. https://doi.org/10.1177/1555412015591900
• Elmqvist, N., Tudoreanu, M. E., & Tsigas, P. (2008). Evaluating motion constraints for 3D wayfinding in immersive and desktop virtual environments. Proceeding of the Twenty-Sixth Annual CHI Conference on Human Factors in Computing Systems - CHI ’08, 1769. https://doi.org/10.1145/1357054.1357330
• Hart, S. G. (2006). NASA-task Load Index (NASA-TLX). Proceedings of the Human Factors and Ergonomics Society.
• Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology. https://doi.org/10.1016/S0166-4115(08)62386-9
• Hartmann, T., Wirth, W., Schramm, H., Klimmt, C., Vorderer, P., Gysbers, A., … Sacau, A. M. (2015). The spatial presence experience scale (SPES): A short self-report measure for diverse media settings. Journal of Media Psychology, 28(1), 1–15. https://doi.org/10.1027/1864-1105/a000137
• Hunicke, R., Leblanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. AAAI Workshop - Technical Report.
• IJsselsteijn, W. A., de Kort, Y. A. W., & Poels, K. (2007). Game Experience Questionnaire: development of a self-report measure to assess the psychological impact of digital games — Eindhoven University of Technology research portal. PRESENCE 2007.
• Johnsen, K., & Lok, B. (2008). An evaluation of immersive displays for virtual human experiences. Proceedings - IEEE Virtual Reality. https://doi.org/10.1109/VR.2008.4480764
• Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G. (1993). Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology, 3(3), 203–220. https://doi.org/10.1207/s15327108ijap0303_3 Kim, D., & Ko, Y. J. (2019). The impact of virtual reality (VR) technology on sport spectators’ flow experience and satisfaction. Computers in Human Behavior, 93, 346–356. https://doi.org/10.1016/j.chb.2018.12.040
• Kim, K., Rosenthal, M. Z., Zielinski, D. J., & Brady, R. (2014). Effects of virtual environment platforms on emotional responses. Computer Methods and Programs in Biomedicine. https://doi.org/10.1016/j.cmpb.2013.12.024
• Kober, S. E., Kurzmann, J., & Neuper, C. (2012). Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study. International Journal of Psychophysiology, 83(3), 365–374. https://doi.org/10.1016/j.ijpsycho.2011.12.003
• Kulshreshth, A., Schild, J., & LaViola, J. J. (2012). Evaluating user performance in 3D stereo and motion enabled video games. Foundations of Digital Games 2012, FDG 2012 - Conference Program. https://doi.org/10.1145/2282338.2282350
• Laha, B., Sensharma, K., Schiffbauer, J. D., & Bowman, D. A. (2012). Effects of immersion on visual analysis of volume data. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2012.42
• Laugwitz, B., Held, T., & Schrepp, M. (2008). Construction and evaluation of a user experience questionnaire. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). https://doi.org/10.1007/978-3-540-89350-9-6
• Lessiter, J., Freeman, J., Keogh, E., & Davidoff, J. (2001). A cross-media presence questionnaire: The ITC-sense of presence inventory. Presence: Teleoperators and Virtual Environments, 10(3), 282–297. https://doi.org/10.1162/105474601300343612
• Li, H., & Giudice, N. A. (2013). The effects of immersion and body-based rotation on learning multi-level indoor virtual environments. Proceedings of the 5th ACM SIGSPATIAL International Workshop on Indoor Spatial Awareness, ISA 2013. https://doi.org/10.1145/2533810.2533811
• Litwiller, T., & LaViola, J. J. (2011). Evaluating the benefits of 3d stereo in modern video games. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/1978942.1979286
• Lugrin, J.-L., Cavazza, M., Charles, F., Le Renard, M., Freeman, J., & Lessiter, J. (2013). Immersive FPS games. Proceedings of the 2013 ACM International Workshop on Immersive Media Experiences - ImmersiveMe ’13, 7–12. https://doi.org/10.1145/2512142.2512146
• Mania, K., & Chalmers, A. (2001). The effects of levels of immersion on memory and presence in virtual environments: A reality centered approach. Cyberpsychology and Behavior. https://doi.org/10.1089/109493101300117938 Mania, K., & Chalmers, A. (2002). The Effects of Levels of Immersion on Memory and Presence in Virtual Environments: A Reality Centered Approach. CyberPsychology & Behavior, 4(2), 247–264. https://doi.org/10.1089/109493101300117938
• Martel, E., Su, F., Gerroir, J., Hassan, A., Girouard, A., & Muldner, K. (2015). Diving Head-First into Virtual Reality—Evaluating HMD Control Schemes for VR Games. Proceedings of the 10th International Conference on the Foundations of Digital Games - FDG 2015. Retrieved from https://pdfs.semanticscholar.org/2c7d/b9069822f2599cdb466d51f17b28fd2baad6.pdf?_ga=2.187769836.550852407.1589226699-1224180204.1587306111
• McMahan, R. P., Bowman, D. A., Zielinski, D. J., & Brady, R. B. (2012). Evaluating display fidelity and interaction fidelity in a virtual reality game. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2012.43
• Mizell, D. W., Jones, S. P., Slater, M., & Spanlang, B. (2002). Comparing immersive virtual reality with other display modes for visualizing complex 3D geometry. In University College London, ….
• Monteiro, D., Liang, H. N., Xu, W., Brucker, M., Nanjappan, V., & Yue, Y. (2018). Evaluating enjoyment, presence, and emulator sickness in VR games based on first- and third-person viewing perspectives. Computer Animation and Virtual Worlds. https://doi.org/10.1002/cav.1830
• Moroney, W. F., Biers, D. W., Eggemeier, F. T., & Mitchell, J. A. (2003). A comparison of two scoring procedures with the NASA task load index in a simulated flight task. https://doi.org/10.1109/naecon.1992.220513
• Pallavicini, F., Ferrari, A., Zini, A., Garcea, G., Zanacchi, A., Barone, G., & Mantovani, F. (2018). What distinguishes a traditional gaming experience from one in virtual reality? An exploratory study. Advances in Intelligent Systems and Computing. https://doi.org/10.1007/978-3-319-60639-2_23
• Pallavicini, F., Pepe, A., & Minissi, M. E. (2019). Gaming in Virtual Reality: What Changes in Terms of Usability, Emotional Response and Sense of Presence Compared to Non-Immersive Video Games? Simulation & Gaming, 50(2), 136–159. https://doi.org/10.1177/1046878119831420
• Patrick, E., Cosgrove, D., Slavkovic, A., Rode, J. A., Verratti, T., & Chiselko, G. (2000). Using a large projection screen as an alternative to head-mounted displays for virtual environments. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/332040.332479
• Pausch, R., Proffitt, D., & Williams, G. (1997). Quantifying immersion in virtual reality. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997. https://doi.org/10.1145/258734.258744
• Pestaluky, A., Kane, B., & Fetter, B. (2015). Keep Talking and Nobody Explodes. Retrieved from https://www.keeptalkinggame.com/
• Phan, M. H., Keebler, J. R., & Chaparro, B. S. (2016). The Development and Validation of the Game User Experience Satisfaction Scale (GUESS). Human Factors. https://doi.org/10.1177/0018720816669646
• Qi, W., Taylor, R. M., Healey, C. G., & Martens, J. B. (2006). A comparison of immersive HMD, fish tank VR and fish tank with haptics displays for volume visualization. Proceedings - APGV 2006: Symposium on Applied Perception in Graphics and Visualization. https://doi.org/10.1145/1140491.1140502
• Ragan, E. D., Kopper, R., Schuchardt, P., & Bowman, D. A. (2013). Studying the effects of stereo, head tracking, and field of regard on a small-scale spatial judgment task. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2012.163
• Riddle, K. (2010). Remembering Past Media Use: Toward the Development of a Lifetime Television Exposure Scale. Communication Methods and Measures, 4(3), 241–255. https://doi.org/10.1080/19312458.2010.505500
• Roettl, J., & Terlutter, R. (2018). The same video game in 2D, 3D or virtual reality – How does technology impact game evaluation and brand placements? PLoS ONE. https://doi.org/10.1371/journal.pone.0200724
• Ruddle, R. A., Payne, S. J., & Jones, D. M. (1999). Navigating large-scale virtual environments: What differences occur between helmet-mounted and desk-top displays? Presence: Teleoperators and Virtual Environments. https://doi.org/10.1162/105474699566143
• Ruddle, R. A., & Péruch, P. (2004). Effects of proprioceptive feedback and environmental characteristics on spatial learning in virtual environments. International Journal of Human Computer Studies. https://doi.org/10.1016/j.ijhcs.2003.10.001
• Sauro, J., & Lewis, J. R. (2012). Quantifying the User Experience. In Quantifying the User Experience. https://doi.org/10.1016/C2010-0-65192-3
• Schild, J., LaViola, J. J., & Masuch, M. (2012). Understanding user experience in stereoscopic 3D games. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/2207676.2207690
• Shelstad, W. J., Smith, D. C., & Chaparro, B. S. (2017). Gaming on the rift: How virtual reality affects game user satisfaction. Proceedings of the Human Factors and Ergonomics Society. https://doi.org/10.1177/1541931213602001
• Slater, M., Linakis, V., Usoh, M., & Kooper, R. (1995). Immersion , Presence , and Performance in Virtual Environments : An Experiment with Tri-Dimensional Chess. Virtual Reality. https://doi.org/10.1.1.34.6594
• Slobounov, S. M., Ray, W., Johnson, B., Slobounov, E., & Newell, K. M. (2015). Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. International Journal of Psychophysiology. https://doi.org/10.1016/j.ijpsycho.2014.11.003
• Sousa Santos, B., Dias, P., Pimentel, A., Baggerman, J. W., Ferreira, C., Silva, S., & Madeira, J. (2009). Head-mounted display versus desktop for 3D navigation in virtual reality: A user study. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-008-0223-2
• Swindells, C., Po, B. A., Hajshirmohammadi, I., Corrie, B., Dill, J. C., Fisher, B. D., & Booth, K. S. (2004). Comparing CAVE, wall, and desktop displays for navigation and wayfinding in complex 3D models. Proceedings of Computer Graphics International Conference, CGI. https://doi.org/10.1109/CGI.2004.1309243
• Tan, C. T., Leong, T. W., Shen, S., Dubravs, C., & Si, C. (2015). Exploring gameplay experiences on the oculus rift. CHI PLAY 2015 - Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play. https://doi.org/10.1145/2793107.2793117
• Usoh, M., Catena, E., Arman, S., & Slater, M. (2000). Using presence questionnaires in reality. Presence: Teleoperators and Virtual Environments. https://doi.org/10.1162/105474600566989
• Ventura, S., Brivio, E., Riva, G., & Baños, R. M. (2019). Immersive Versus Non-immersive Experience: Exploring the Feasibility of Memory Assessment Through 360° Technology. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2019.02509
• Vorderer, P., Wirth, W., Gouveia, F. R., Biocca, F., Saari, T., Jäncke, F., … Jäncke, P. (2004). MEC Spatial Presence Questionnaire (MECSPQ): Short Documentation and Instructions for Application. Retrieved from https://www.researchgate.net/profile/Feliz_Gouveia/publication/318531435_MEC_spatial_presence_questionnaire_MEC-SPQ_Short_documentation_and_instructions_for_application/links/598041b5458515687b4fa65d/MEC-spatial-presence-questionnaire-MEC-SPQ-Short-docume
• Waller, D., Hunt, E., & Knapp, D. (1998). The transfer of spatial knowledge in virtual environment training. Presence: Teleoperators and Virtual Environments. https://doi.org/10.1162/105474698565631
• Weidner, F., Hoesch, A., Poeschl, S., & Broll, W. (2017). Comparing VR and non-VR driving simulations: An experimental user study. Proceedings - IEEE Virtual Reality. https://doi.org/10.1109/VR.2017.7892286 Wilson, R., & Mayhorn, C. B. (2019). Examining the Role of Video in Sports Media Viewing. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. https://doi.org/10.1177/1071181319631424
• Winn, W., Windschitl, M., Fruland, R., & Lee, Y. (2002). When Does Immersion ina a virtual Environment Helps Students construct Understanding? ICLS 2002.
• Wirth, W., Vorderer, P., Hartmann, T., Klimmt, C., & Schramm, H. (2003). Constructing Presence : Towards a two-level model of the formation of Spatial Presence experiences. In Communication (Vol. 2003). Retrieved from https://www.researchgate.net/publication/318531733_Constructing_Presence_Towards_a_two-level_model_of_the_formation_of_Spatial_Presence
• Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments. https://doi.org/10.1162/105474698565686
• Yildirim, Ç., Bostan, B., & Berkman, M. İ. (2019). Impact of different immersive techniques on the perceived sense of presence measured via subjective scales. Entertainment Computing. https://doi.org/10.1016/j.entcom.2019.100308
• Yildirim, C., Carroll, M., Hufnal, D., Johnson, T., & Pericles, S. (2018). Video Game User Experience: To VR, or Not to VR? 2018 IEEE Games, Entertainment, Media Conference (GEM), 1–9. https://doi.org/10.1109/GEM.2018.8516542
• Zanbaka, C. A., Lok, B. C., Babu, S. V., Ulinski, A. C., & Hodges, L. F. (2005). Comparison of path visualizations and cognitive measures relative to travel technique in a virtual environment. IEEE Transactions on Visualization and Computer Graphics, 11(6), 694–705. https://doi.org/10.1109/TVCG.2005.92