Embodiment in Virtual Reality and Augmented Reality Games: An Investigation on User Interface Haptic Controllers

Abstract

Undoubtedly, virtual reality technologies stand out as one of the most rapidly expanding domains in the present era. Apart from features like clarity of vision, viewing angle and time distortion that are improving daily, user interfaces are another crucial part of this technology that is not talked about much but must be. This study investigates and summarizes the literature on wearable technology and haptic controllers, which serve as the player’s interface between virtual and augmented reality. The controllers and devices we investigate, try to achieve a holistic approach in the experience thus it appears to be a rich soil for future studies. Although many technological advancements and entirely new ones are anticipated, a thorough analysis of the phenomena of interaction, participation, and integration, as well as embodiment and presence, should shed light on the question of its scope, depth, assessment, and limitations.

Keywords

• Embodiment
• Virtual Reality
• Augmented Reality
• Gaming
• User Interface
• Haptic Controllers
• Human Computer Interaction

Thanks

To my family, advisor Prof. Dr. Rıfat Şahiner and editors.

References

1. A. Gregersen, “Genre, technology and embodied interaction: The evolution of digital game genres and motion gaming” Journal of media and communication research 27(51):16, 2011.
2. J. Juul, “The Game, the Player, the World: Looking for a Heart of Gameness”, DIGRA International Conference: Level Up, pp.30-45, 2010.
3. P. Skalski, R. Tamborini, A, Shelton, M Buncher, Michael and P. Lindmark, “Mapping the road to fun: Natural video game controllers, presence, and game enjoyment. Sage Journals, 2011.
4. S. De Castell, J, Jenson, J., “The Entrepreneurial Gamer: Regendering the Order of Play”. Games and Culture. 13 (7), 2018.
5. P. Dourish, Where the action is: The Foundations of embodies interaction. The MIT Press Online, 2001.
6. D. Hufnal, E. Osborne, T. Johnson, and C. Yildirim, “The Impact of Controller Type on Video Game User Experience in Virtual Reality”, 2019 IEEE Games, Entertainment, Media Conference (GEM), New Haven, CT, USA, 2019, pp.1-9, 2019.
7. P. Lankoski, “Embodiment in Character-based Video Games. Academic- Mindtrek’16, October 17-18, 2016, Tampere, Finland (c) ACM 2016.
8. M. Bernatchez, and R. Jean-Marc, “A Study on the Impact of Spatial Reference Frames on Human Performance in Virtual Reality User Interfaces”. Journal of Multimedia, 3(5), pp. 2600-2605., 2007.

9. S. M. LaValle, “Virtual Reality”, Cambridge University Press (E-book), 2019.
10. M. R. Sudha, K. Sriraghav, S. Abisheck, J. S. Gracia and S. Manisha, “Approaches and Applications of Virtual Reality and Gesture Recognition: A Review”. International Journal of Ambient Computing and Intelligence, 8(4), 1–18, 2017.
11. D. Bachmann, F. Weichert and G. Rinkenauer “Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller”. Sensors, 18(7), 2194, 2018.
12. J. Perret, E. Benjamin and V. Poorten, V. “Touching Virtual Reality: a Review of Haptic Gloves Conference”: ACTUATOR 18. Berlin 66, 2015.
13. Y. Li, J. Huang, F. Tian, H. A. Wang and G. Z. Dai, “Gesture Interaction in Virtual Reality”. Virtual Reality & Intelligent Hardware, 1(1), 84–112, 2019.
14. C. Boletsis, J. E. Cedergren, “VR Locomotion in the New Era of Virtual Reality: An Empirical Comparison of Prevalent Techniques”. Advances in Human-Computer Interaction Volume, 2019.
15. T. Nováček and M. Jirina, “Synchronizing Data from Multiple Optical Sensors for Precise Hand Tracking”, Journal of Visual Communication and Image Representation, 2022.
16. Nintendo UK: 2023. Online, Access Link: www. nintendo.co.uk/
17. T. Leyvand, C. Meekhof, Y. Wei, J Sun and B. Guo, “Kinect Identity: Technology and Experience, Computer”, 44, pp. 94-96, 2011.
18. Metacritic, 2023, Online. Access Link: www. metacritic.com/
19. H. Tjaden, U. Schwanecke, F. A. Stein and E. Schömer “High-Speed and Robust Monocular Tracking”, Proceedings of the 10th International Conference on Computer Vision Theory and Applications VISIGRAPP - (Volume 1), 2015.
20. M. Merleau-Ponty, “Phenomenology of Perception” 1945, Trans: Colin Smith, Tylor & Francis e-Library, Routhledge, London, 2005.
21. Microsoft, 2023. Online, Access Link: www. micro soft.com
22. R. Cong and R. Winters, “How Does the Xbox Kinect Work”, Access Link: www.jameco.com/
23. H. Sarbolandi, D. Lefloch, A. Kolb, “Kinect range sensing: Structured light versus Time of Flight Kinect”. Computer Vision and Image Understanding. Vol: 139 October 2015, Sf: 1-20 Access Link: https://doi.org/ 10.1016/j.cviu.2015.05.006. 36-54.
24. ART – Advanced Realtime Tracking. Access Link: https://ar-tracking.com/
25. OptiTrack, Access Link: https://optitrack.com/
26. WorldViz: Virtual Reality for Training and Research. Access Link: www.worldviz.com/
27. Epic games. Access Link: www.epicgames.com
28. Unity Technologies: Real-Time Development Program. Access Link: https://unity.com/
29. “Oculus quest” on XinReality. Online, Access Link: https://xinreality.com/wiki/Oculus_Quest
30. J. Durbin “Community Download: Was Constellation Tracking a Long-Term Mistake for Oculus?” Access Link: www.uploadvr.com
31. Play Station VR. 2021-2023, Online. Access Link: www.playstation.com/ps-vr/bundles/
32. Valve. Access Link: www.valvesoftware.com
33. S. Machkovech on Samred. Access Link: www. samred.com/
34. ArsElectronica, 2013. Online, Access link: www.ars. electronica.art/news/en/
35. Facebook, 2023. Access Link: www. facebook.com
36. Meta AI: Access Link: https://ai.meta.com/
37. B. Lang, “Lyrobotix Merges Ultrasonic and Lighthouse-like Tech for Portable Positional VR Tracking”, Access Link: www.roadtovr.com/lyrobotix -merges-ultrasonic-and-lighthouse-like-tech-for-porta le-positional-vr-tracking, 2023.
38. N. Murchinson and W. R. Proctor, “Spatial Compatibility Effects with Unimanual and Bimanual Wheel-Rotation Responses: An Homage to Guiard (1983)”, Journal of Motor Behavior. Vol: 45(5).pp. 441-454. Access Link: /doi.org/ 10.1080 /0022 28 95 . 2013.823906
39. M. Karam, “A framework for research and design of gesture-based human-computer interactions”. University of Southampton [PhD Thesis], 2006.
40. T. Massie J. K. and Salisbury, “The PHANTOM Haptic Interface: A Device for Probing Virtual Objects”. Proceedings of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, (55). Pp. 295–300., 1994.
41. T. Endo, H. Kawasaki, T. Mouri, Y. Ishigure, H. Shimomura, M. Matsumura and K. Koketsu, “Five-Fingered Haptic Interface Robot: HIRO III”. IEEE Trans Haptics. 2011,4(1):14-27, 2011.
42. D. Bachmann, F. Weicher and G. Rinkenauer “Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller” Sensors, Basel. 2018 Jul 7;18(7):2194. Access Link: doi: 10.3390/s18072194.
43. D. K. Y. Chen, J. B. Chossat and P. B. Shull “HaptiVec: Presenting Haptic Feedback Vectors in Handheld Controllers using Embedded Tactile Pin Arrays” CHI’19: Proceedings of the 2019: CHI Conference on Human Factors in Computing Systems May 2019 Paper No.: 171 pp. 1–11, 2019.
44. J. M. Walker, N. Zemiti, P. Poignet, and A. M. Okamura, “Holdable Haptic Device for 4-DOF Motion Guidance”. 2019 IEEE World Haptics Conference (WHC). pp: 109–114, 2019.
45. Y. Guiard, “Asymmetric Division of Labor in Human Skilled Bimanual Action”, Journal of Motor Behavior, 19, 486–517, 2013. Online. Access link: https:// doi.org/10.1080/00222895.1987.10735426
46. I. Choi, E. Ofer, H. Benko, M. Sinclair and C. Holz, “CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality”, CHI’18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems April 2018 Paper No.: 654 Sf: 1–13 Access Link: https://dl.acm.org/doi/10.1145/3173574.3174228.
47. E. Whitmire, H. Benko, C. Holz, E. Ofek, M. Sinclair, “Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfig- urable Virtual Reality Controller” CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. April 2018.
48. A. Rahimi, H. Patel, H. Ajmal and S. Haghani, “The Design and Implementation of a VR Gun Controller with Haptic Feedback”, 2018 IEEE International Conference on Consumer Electronics, 2019.
49. A. Rahimi and S. Haghani, “A VR gun controller with Recoil Adjustability”, IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 2020, pp. 1-20, 2020.
50. M. Bernatchez and R. Jean-Marc, “Impact of Spatial Reference Frames on Human Performance in Virtual Reality User Interfaces”. Journal of Multimedia, 3(5), pp. 2600-2605, 2017.
51. HTC Vive, 2023. Access link: www.vive.com/us/
52. Steam. Access Link: www.steampowered.com
53. M. Lin, “Etee: The Button-Free VR Controller” Access Link: www.kickstarter.com/projects/tg0/etee-complete-control-in-3d 2023
54. W. Greenwald, “The Best VR Headsets for 2023” Access Link: www.au.pcmag.com/virtual-reality-1/42 713/the-best-vr-headset Access Date: 20.11.2023.
55. P. Temoche, E. Ramirez and Omaira Rodríguez, “A Low-cost Data Glove for Virtual Reality” Conference: XI International Congress of Numerical Methods in Enginnering and Applied Sciences (CIMENICS), 2012.
56. T. Mazuryk and M. Gervautz, “Virtual Reality – History, Applications, Technology and Future”, 1999. www.cg.tuwien.ac.at/research/publications/1996/mazuryk-1996-VRH/TR-186-2-96- 06Paper.pdf
57. A. Lasko-Harvill, “Identity and Mask in Virtual Reality.” Discourse, vol. 14, no. 2, 1992, pp: 222–34. JSTOR, http://www.jstor.org/stable/41389227
58. N. J. Jarque-Bou, M. Vergara, J. L. Sancho-Bru, V. Gracia-Ibáñez and A. Roda-Sales, "Hand Kinematics Characterization While Performing Activities of Daily Living Through Kinematics Reduction," in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 28, no. 7, pp. 1556-1565, July 2020, Access Link: doi: 10.1109/TNSRE.2020.2998642.
59. MANUS Meta VR, 2013. Access Link: www.manus-meta.com/
60. M. Ilves, Y. Gizatdinova, V. Surakka and E. Vankka “Head movement and facial expressions as game input”, Entertainment Computing 5 (3), pp:147-156, 2014.
61. D. A. Bowman, D. Koller, L. F. and Hodges, “Travel in immersive virtual environ- ments: an evaluation of viewpoint motion control techniques” Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality, Albuquerque, NM, USA, pp. 45-52,
62. E. Martel and K. Muldner, “Controlling VR Games: Control Schemes’ and the Player Experience. Entertainment Computing. Access Link: DOI: 10.1016/ jentcom.2017.04.004, 2017.
63. “Haptic” on Venturebeat, 2023. Access Link: www. venturebeat.com/
64. Highfidelty, 2022. Online, Access Link: www.high fidelity.com/
65. N. Koolonavich, “High Fidelity and NeoSensory Announce Exoskin Haptic Jacket”. Access Link: https://www.vrfocus.com/2018/04/high-fidelity-and-neosensory-announce-exoskin. 2013, 2023.
66. NeoSensory. Access Link: www. neosensory.com
67. Open Critic. Access Link: www.opencritic.com
68. VRScout, 2018. Access Link: www.vrscout.com
69. A. Delazio, K. Nakagaki, R. L. Klatzky, S. E. Hudson, J. F. Lehman, and Alanson P. Sample, “Force Jacket: Pneumatically Actuated Jacket for Embodied Haptic Experiences”. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, Paper 320, 1–12. Access Link: https://doi.org/ 10.1145/3173574.317389
70. A. A. Maimani and A. A. Roudaut, “Frozen Suit: Designing a Changeable Stiffness Suit and Its Application to Haptic Games”. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 2440-2448, 2017. Access Link: https://doi.org/10.1145/3025453.3025655
71. P. Pita, “List of Full Body Virtual Reality Haptic Suits”. 2017. Online, Access Link: www.virtualreality times.com/2017/02/28/list-of-full-body-virtual-reality haptic-suits.
72. Noitom: Motion Capture Systems, 2020. Access Link: https://www.noitom.com/
73. H. Noma and T. Miyasato, “A New Approach for Canceling Turning Motion in the Locomotion Interface, ATLAS”. Proceedings of the ASME 1999 International Mechanical Engineering Congress and Exposition. Dynamic Systems and Control. Nashville, Tennessee, USA. November 14–19. pp. 405-406.
74. J. L. Campos and H. H. Bülthoff, “Multimodal Integration during Self-Motion in Virtual Reality”. The Neural Bases of Multisensory Processes. (Eds: Murray MM, Wallace MT). Boca Raton (FL): CRC Press/Taylor & Francis; 2012. In: 30. PMID: 22593878. 2029. Access Link: www.pubmed. ncbi. nlm.nih.gov/22593878/
75. E. Guizzo “CyberWalk: Giant Omni-Directional Treadmill to Explore Virtual Worlds” Access Link: https://spectrum.ieee.org/automaton/robotics/robotics-software/cyberwalk- giant-omnidirectional-treadmill-to-explore-virtual-worlds.
76. “CyberWalk Projekt”. Max Planck Gesselschaft.: www.mpg.de/552995/pressemitteilung20080411
77. Infinadeck. (2023). Infinadeck VR Treadmill Gets Vive Tracker Enhancement. Access Link: www.infinadeck.com/blog/infinadeck-vr-treadmillget vive-tracker-enhancement
78. KATVR, 2023. Access Link: www.kat-vr.com
79. “VRBike by Holodia”: Access Link: www. holodia. com
80. C. Boletsis and J. E. Cedergren “VR Locomotion in the New Era of Virtual Reality: An Empirical Comparison of Prevalent Techniques”. Advances in Human-Computer Interaction Volume 2019. Access Link: www.hindawi.com/journals/ahci/2019/7420781/
81. J. W. Lin, K. W. Chen and Y. P. Hung “Visualizing the Keyboard in Virtual Reality for Enhancing Immersive Experience”, ACM SIGGRAPH 2017 Posters, 2017.
82. D. Cohen, X. Tang and C. W. Fu, “Grab AR: Occlusion-aware Grabbing Virtual Objects in AR”, UIST’ 20 ACM ISBN: 978-1-4503-6708, 2020.