Industrial Augmented Reality - A Framework for Defining Requirements

Abstract

Augmented Reality (AR) boasts a wide array of applications throughout the entire product lifecycle; however, its adoption in industrial settings is often impeded by factors such as high setup costs, poor system integration, and limited modality. Despite these challenges, current Industrial Augmented Reality (IAR) applications exhibit a significant overlap in components used for information gathering and visualization. This paper describes the current state of modern IAR architectures and introduces a novel framework for the definition of requirements specific to IAR applications. This draws upon the principles of human-cantered design to provide a structured approach for integrating IAR more effectively in industrial contexts. Finally, a case-study shows how the framework can be used to help implement an assistance system for assembly of a simple product.

Keywords

• Augmented Reality
• Industrial Augmented Reality
• Requirements Engineering
• Human Machine Interaction
• System Design

References

1. [1] P. Fite-Georgel, “Is there a reality in industrial augmented reality?” in International symposium on mixed and augmented reality, in Sci-ence and technology proceedings. IEEE, Oct. 2011.
2. [2] S. Büttner, M. Prilla, and C. Röcker, “Augmented reality training for industrial assembly work - are projection-based AR assistive systems an appropriate tool for assembly training?” in 2020 CHI conference on human factors in computing systems, New York, NY, USA: As-sociation for Computing Machinery, 2020, pp. 1–12. doi: 10.1145/3313831.3376720.
3. [3] M. Funk, A. Bächler, L. Bächler, T. Kosch, T. Heidenreich, and A. Schmidt, “Working with augmented reality?” in 10th international conference on PErvasive technologies related to assistive environments, New York, NY, USA: Association for Computing Machinery, Jun. 2017, pp. 222–229.
4. [4] E. Bottani and G. Vignali, “Augmented reality technology in the manufacturing industry: A review of the last decade,” IISE Transac-tions, vol. 51, no. 3, pp. 284–310, Feb. 2019.
5. [5] V. Bilous, R. Porsch, and K. Spanoudakis, “Augmented reality: Increasing availability and its implication for SMEs,” The Future of Smart Production for SMEs, pp. 349–356, Oct. 2022, doi: 10.1007/978-3-031-15428-7_31.
6. [6] W. Fang, L. Chen, T. Zhang, C. Chen, Z. Teng, and L. Wang, “Head-mounted display augmented reality in manufacturing: A systematic review,” Robotics and Computer-Integrated Manufacturing, vol. 83, p. 102567, Oct. 2023, doi: 10.1016/j.rcim.2023.102567.
7. [7] T. Masood and J. Egger, “Adopting augmented reality in the age of industrial digitalisation,” Computers in Industry, vol. 115, p. 103112, 2020.
8. [8] “Ergonomie der mensch-system-interaktion – teil 210: Menschzentrierte gestaltung interaktiver systeme,” Deutsches Instut für Normung e.V., DIN EN ISO 9241-210, 2020.

9. [9] M. Runji, Y.-J. Lee, and C.-H. Chu, “Systematic literature review on augmented reality-based maintenance applications in manufacturing centered on operator needs,” International Journal of Precision Engineering and Manufacturing-Green Technology, vol. 10, no. 2, pp. 567–585, Mar. 2023, doi: 10.1007/s40684-022-00444-w.
10. [10] T. Kosch, Y. Abdelrahman, M. Funk, and A. Schmidt, “One size does not fit all,” Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Com-puters, Sep. 2017, doi: 10.1145/3123024.3124395.
11. [11] A. Syberfeldt, M. Holm, O. Danielsson, L. Wang, and R. L. Brewster, “Support systems on the industrial shop-floors of the future – operators’ perspective on augmented reality,” Procedia CIRP, vol. 44, pp. 108–113, 2016.
12. [12] M. Eswaran, A. K. Gulivindala, A. K. Inkulu, and M. V. A. Raju Bahubalendruni, “Augmented reality-based guidance in product assembly and maintenance/repair perspective: A state of the art review on challenges and opportunities,” Expert Systems with Applica-tions, vol. 213, p. 118983, Mar. 2023, doi: 10.1016/j.eswa.2022.118983.
13. [13] G. Stübl, G. Ebenhofer, H. Bauer, and A. Pichler, “Lessons learned from industrial augmented reality applications,” Procedia Computer Science, vol. 200, pp. 1218–1226, 2022, doi: 10.1016/j.procs.2022.01.322.
14. [14] D. Röltgen and R. Dumitrescu, “Classification of industrial augmented reality use cases,” Procedia CIRP, vol. 91, pp. 93–100, 2020, doi: 10.1016/j.procir.2020.01.137.
15. [15] J. L. Siewert, M. Neges, and D. Gerhard, “Ein klassifizierungssystem für industrielle augmented reality anwendungen,” pp. 401–416.
16. [16] M. Bauer et al., “Design of a component-based augmented reality framework,” in Proceedings IEEE and ACM international symposium on augmented reality, in ISAR-01. IEEE Comput. Soc, 2001. doi: 10.1109/isar.2001.970514.
17. [17] M. Schumann, P. Klimant, C. Kollatsch, and V. Wittstock, “Modulares framework zur entwicklung von augmented-reality-anwendungen,” Feb. 2013.
18. [18] P. Fraga-Lamas, T. M. Fernandez-Carames, O. Blanco-Novoa, and M. A. Vilar-Montesinos, “A review on industrial augmented reality systems for the industry 4.0 shipyard,” IEEE Access, vol. 6, pp. 13358–13375, 2018.
19. [19] T. Kuster et al., “A distributed architecture for modular and dynamic augmented reality processes,” in 2019 IEEE 17th international conference on industrial informatics (INDIN), IEEE, Jul. 2019. doi: 10.1109/indin41052.2019.8972101.
20. [20] E. Laviola, M. Gattullo, A. Evangelista, M. Fiorentino, and A. E. Uva, “In-situ or side-by-side? A user study on augmented reality maintenance instructions in blind areas,” Computers in Industry, vol. 144, p. 103795, Jan. 2023, doi: 10.1016/j.compind.2022.103795.
21. [21] M. Gattullo, A. Evangelista, A. E. Uva, M. Fiorentino, and J. L. Gabbard, “What, how, and why are visual assets used in industrial augmented reality? A systematic review and classification in maintenance, assembly, and training (from 1997 to 2019),” IEEE Trans-actions on Visualization and Computer Graphics, vol. 28, no. 2, pp. 1443–1456, Feb. 2022, doi: 10.1109/tvcg.2020.3014614.
22. [22] D. Gerhard, M. Neges, J. L. Siewert, and M. Wolf, “Towards universal industrial augmented reality: Implementing a modular IAR system to support assembly processes,” Multimodal Technologies and Interaction, vol. 7, no. 7, 2023, doi: 10.3390/mti7070065.
23. [23] M. Quandt, A. A. Alla, L. Meyer, and M. Freitag, “Success factors for the development of augmented reality-based assistance systems for maintenance services,” in 7. WGP-jahreskongress, R. H. Schmitt and G. Schuh, Eds., Apprimus Verlag, 2017
24. [24] M. Quandt, B. Knoke, C. Gorldt, M. Freitag, and K.-D. Thoben, “General requirements for industrial augmented reality applications,” Procedia CIRP, vol. 72, pp. 1130–1135, 2018, doi: 10.1016/j.procir.2018.03.061.
25. [25] M. Lorenz, S. Knopp, and P. Klimant, “Industrial augmented reality: Requirements for an augmented reality maintenance worker sup-port system,” in Adjunct proceedings of the 2018 ieee international symposium on mixed and augmented reality (ismar), 2018, pp. 151–153. doi: 10.1109/ISMAR-Adjunct.2018.00055.
26. [26] S. Aquino, M. Rapaccini, F. Adrodegari, and G. Pezzotta, “Augmented reality for industrial services provision: The factors influencing a successful adoption in manufacturing companies,” Journal of Manufacturing Technology Management, vol. 34, no. 4, pp. 601–620, Feb. 2023, doi: 10.1108/jmtm-02-2022-0077.
27. [27] M. Gattullo, G. W. Scurati, M. Fiorentino, A. E. Uva, F. Ferrise, and M. Bordegoni, “Towards augmented reality manuals for industry 4.0: A methodology,” Robotics and Computer-Integrated Manufacturing, vol. 56, pp. 276–286, Apr. 2019, doi: 10.1016/j.rcim.2018.10.001.
28. [28] M. Quandt, T. Beinke, and M. Freitag, “User-centered evaluation of an augmented reality-based assistance system for maintenance,” Procedia CIRP, vol. 93, pp. 921–926, 2020, doi: 10.1016/j.procir.2020.03.053.
29. [29] A. Dey, M. Billinghurst, R. W. Lindeman, and J. E. Swan, “A systematic review of 10 years of augmented reality usability studies: 2005 to 2014,” Frontiers in Robotics and AI, vol. 5, Apr. 2018, doi: 10.3389/frobt.2018.00037.
30. [30] A. Syberfeldt, O. Danielsson, and P. Gustavsson, “Augmented reality smart glasses in the smart factory: Product evaluation guidelines and review of available products,” IEEE Access, vol. 5, pp. 9118–9130, 2017, doi: 10.1109/access.2017.2703952.
31. [31] K. Tainaka, Y. Fujimoto, T. Sawabe, M. Kanbara, and H. Kato, “Selection framework of visualization methods in designing AR indus-trial task-support systems,” Computers in Industry, vol. 145, p. 103828, Feb. 2023, doi: 10.1016/j.compind.2022.103828.
32. [32] T. Bosch, G. van Rhijn, F. Krause, R. Könemann, E. S. Wilschut, and M. de Looze, “Spatial augmented reality,” in Proceedings of the 13th ACM international conference on PErvasive technologies related to assistive environments, New York, NY, USA: Association for Computing Machinery, Jun. 2020. doi: 10.1145/3389189.3397975.
33. [33] M. Jost, A. Luxenburger, S. Knoch, and J. Alexandersson, “PARTAS: A personalizable augmented reality based task adaption system for workers with cognitive disabilities,” Proceedings of the 15th International Conference on PErvasive Technologies Related to Assis-tive Environments, Jun. 2022, doi: 10.1145/3529190.3529208.
34. [34] M. Gattullo, G. W. Scurati, A. Evangelista, F. Ferrise, M. Fiorentino, and A. E. Uva, “Informing the use of visual assets in industrial augmented reality,” 2019, pp. 106–117. doi: 10.1007/978-3-030-31154-4_10.
35. [35] M. Funk, T. Kosch, and A. Schmidt, “Interactive worker assistance,” Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Sep. 2016.