Evaluating the effectiveness of a collaborative robotic maintenance training platform in extended reality (XR): A comprehensive validation study

Abstract

Maintenance training for industrial robotic systems involves challenges such as high costs, safety risks, and hardware limitations. This study presents the validation outcomes of the Multiplayer No-Code Mixed Reality Editor solution developed to provide immersive, interactive, and scalable training. The platform features a no-code scenario creation tool that allows users to design detailed training scenarios without programming knowledge, offering a more hands-on experience than traditional formats. A two-phase research design was adopted, involving 25 undergraduate students (Phase 1) and 6 experienced robotics experts (Phase 2). Participant performance was evaluated using task completion times, pre/post-tests, the System Usability Scale (SUS), and the NASA Task Load Index (NASA- TLX). The findings showed significant advantages over traditional methods. Results indicated high usability (average SUS score of 78) and a balanced cognitive workload (average NASA-TLX score of 4.3). XR- trained experts completed virtual scenarios up to 76.8% faster and real- world tasks 28.7% faster than their traditionally trained counterparts, reporting higher clarity and applicability. These results prove that editör is an effective, efficient, and user-friendly training solution for both novice and expert users in industrial robotic maintenance.

Keywords

• Extended Reality (XR)
• Robotic Maintenance
• Training Effectiveness
• Validation
• System Usability Scale (SUS)
• NASA-TLX

References

1. Abidi, M. H., Al-Ahmari, A., Ahmad, A., Ameen, W., & Alkhalefah, H. (2022). A systematic review of the applications of augmented and virtual reality in maintenance. Virtual Reality, 26(4), 1435–1453.
2. Barbalho, Y. G. S., Silva, C. M. D. S., Fernandes, C. S., & Funghetto, S. S. (2024). Designing and validating Health Unit in Focus: A serious game for enhancing undergraduate education on older adults’ health. Applied Sciences.
3. Blattgerste, J., Behrends, J., & Pfeiffer, T. (2023). TrainAR: An open-source visual scripting-based authoring tool for procedural handheld augmented reality trainings. Information, 14(4), 219.
4. Brooke, J. (1996). SUS: A quick and dirty usability scale. In Usability Evaluation in Industry (pp. 189–194).
5. Carvalho, S. A., Conceição, E. S., & Marques, I. C. P. (2025). The impact of virtual reality on employee training and learning in organisations: A systematic literature review. Applied Sciences, 15(19), 10459.
6. Choi, S., Yoon, J., & Lee, J. (2022). The effects of cognitive load on user experience in virtual reality training environments. Journal of Educational Technology & Society, 25(3), 112–125.
7. Coupry, C., Richard, P., Bigaud, D., Noblecourt, S., & Baudry, D. (2023). The value of extended reality techniques to improve remote collaborative maintenance operations: A user study. In Proceedings of the International Conference on Construction Applications of Virtual Reality (pp. 23–33).
8. Dengel, A., Iqbal, M. Z., Grafe, S., & Mangina, E. (2022). A review on augmented reality authoring toolkits for education. Frontiers in Virtual Reality, 3, 798032.

9. Grassini, S., Laumann, K., & Rasmussen Skogstad, M. (2020). The use of virtual reality alone does not promote training performance (but sense of presence does). Frontiers in Psychology, 11, 1743.
10. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Human Mental Workload (pp. 139–183).
11. Monetti, F. M., de Giorgio, A., Yu, H., Maffei, A., & Romero, M. (2022). An experimental study of the impact of virtual reality training on manufacturing operators on industrial robotic tasks. Procedia CIRP, 106, 33–38.
12. Papanastasiou, G., Drigas, A., & Skianis, C. (2022). Virtual and augmented reality in industrial and manufacturing training: A systematic literature review. Education and Information Technologies, 27(8), 11489–11516.
13. Söderlund, H., Zamola, S., Boström, J., Li, D., Mugur, P., Cao, H., & Johansson, B. (2024). The creation of a multi-user virtual training environment for operator training in VR. In Sustainable Production Through Advanced Manufacturing, Intelligent Automation and Work Integrated Learning (pp. 173–180).
14. Stary, C. (2023). Skill transfer in virtual reality training: A meta-analysis. Computers & Education, 205, 104892.
15. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.
16. Wang, P., Wu, P., Wang, J., Chi, H. L., & Wang, X. (2024). A critical review of the use of virtual and augmented reality in construction safety. Automation in Construction, 157, 105120