Evaluating the effectiveness of a collaborative robotic maintenance training platform in extended reality (XR): A comprehensive validation study

Year 2026, Volume: 9 Issue: 1, 123 - 145, 31.01.2026

Batur Alp Akdoğn , Yunus Emre Esen , Kübra Yayan , Uğur Yayan

https://doi.org/10.31681/jetol.1772552

Abstract

Maintenance training for industrial robotic systems involves challenges such as high costs, safety risks, and hardware limitations. This study presents the validation outcomes of the Multiplayer No-Code Mixed Reality Editor solution developed to provide immersive, interactive, and scalable training. The platform features a no-code scenario creation tool that allows users to design detailed training scenarios without programming knowledge, offering a more hands-on experience than traditional formats. A two-phase research design was adopted, involving 25 undergraduate students (Phase 1) and 6 experienced robotics experts (Phase 2). Participant performance was evaluated using task completion times, pre/post-tests, the System Usability Scale (SUS), and the NASA Task Load Index (NASA- TLX). The findings showed significant advantages over traditional methods. Results indicated high usability (average SUS score of 78) and a balanced cognitive workload (average NASA-TLX score of 4.3). XR- trained experts completed virtual scenarios up to 76.8% faster and real- world tasks 28.7% faster than their traditionally trained counterparts, reporting higher clarity and applicability. These results prove that editör is an effective, efficient, and user-friendly training solution for both novice and expert users in industrial robotic maintenance.

Keywords

Extended Reality (XR) , Robotic Maintenance , Training Effectiveness , Validation , System Usability Scale (SUS) , NASA-TLX

References

• Abidi, M. H., Al-Ahmari, A., Ahmad, A., Ameen, W., & Alkhalefah, H. (2022). A systematic review of the applications of augmented and virtual reality in maintenance. Virtual Reality, 26(4), 1435–1453.
• Barbalho, Y. G. S., Silva, C. M. D. S., Fernandes, C. S., & Funghetto, S. S. (2024). Designing and validating Health Unit in Focus: A serious game for enhancing undergraduate education on older adults’ health. Applied Sciences.
• Blattgerste, J., Behrends, J., & Pfeiffer, T. (2023). TrainAR: An open-source visual scripting-based authoring tool for procedural handheld augmented reality trainings. Information, 14(4), 219.
• Brooke, J. (1996). SUS: A quick and dirty usability scale. In Usability Evaluation in Industry (pp. 189–194).
• Carvalho, S. A., Conceição, E. S., & Marques, I. C. P. (2025). The impact of virtual reality on employee training and learning in organisations: A systematic literature review. Applied Sciences, 15(19), 10459.
• Choi, S., Yoon, J., & Lee, J. (2022). The effects of cognitive load on user experience in virtual reality training environments. Journal of Educational Technology & Society, 25(3), 112–125.
• Coupry, C., Richard, P., Bigaud, D., Noblecourt, S., & Baudry, D. (2023). The value of extended reality techniques to improve remote collaborative maintenance operations: A user study. In Proceedings of the International Conference on Construction Applications of Virtual Reality (pp. 23–33).
• Dengel, A., Iqbal, M. Z., Grafe, S., & Mangina, E. (2022). A review on augmented reality authoring toolkits for education. Frontiers in Virtual Reality, 3, 798032.
• Grassini, S., Laumann, K., & Rasmussen Skogstad, M. (2020). The use of virtual reality alone does not promote training performance (but sense of presence does). Frontiers in Psychology, 11, 1743.
• Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Human Mental Workload (pp. 139–183).
• Monetti, F. M., de Giorgio, A., Yu, H., Maffei, A., & Romero, M. (2022). An experimental study of the impact of virtual reality training on manufacturing operators on industrial robotic tasks. Procedia CIRP, 106, 33–38.
• Papanastasiou, G., Drigas, A., & Skianis, C. (2022). Virtual and augmented reality in industrial and manufacturing training: A systematic literature review. Education and Information Technologies, 27(8), 11489–11516.
• Söderlund, H., Zamola, S., Boström, J., Li, D., Mugur, P., Cao, H., & Johansson, B. (2024). The creation of a multi-user virtual training environment for operator training in VR. In Sustainable Production Through Advanced Manufacturing, Intelligent Automation and Work Integrated Learning (pp. 173–180).
• Stary, C. (2023). Skill transfer in virtual reality training: A meta-analysis. Computers & Education, 205, 104892.
• Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.
• Wang, P., Wu, P., Wang, J., Chi, H. L., & Wang, X. (2024). A critical review of the use of virtual and augmented reality in construction safety. Automation in Construction, 157, 105120