An Integration Model to Facilitate Occupational Safety Inspection through Augmented Reality and Artificial Intelligence for Working at High Locations in Buildings

Abstract

Accidents at the construction site, especially falls from height, are the leading cause of both fatal and non-fatal injuries. In the construction industry, digital technologies such as Building information modeling (BIM), Extended Reality (XR) and Artificial Intelligence (AI) have been identified as valuable tools to increase construction productivity, efficiency and safety. In this research, an integration model to facilitate occupational safety inspection through augmented reality and artificial intelligence for working at high locations in buildings is proposed. The business process model and system application model integration are shown in relation to the theoretical framework. A Structural Equation Model was developed to evaluate the proposed model and test the reliability, validity and contribution of the hypotheses. Research findings confirm the positive effect and importance of integration of technologies used in the proposed model on occupational safety auditing. The proposed model digitizes the occupational safety information of teams working at high locations with analytical capabilities and optimizes the decision-making process.

Keywords

• Occupational safety inspection
• augmented reality
• artificial intelligence
• integration model

Yapılarda Yüksekte Çalışma İş Güvenliği Denetimini Kolaylaştırmak İçin Genişletilmiş Gerçeklik ve Yapay Zekânın Entegrasyonu Modeli

Abstract

İnşaat sahasında gerçekleşen kazalar özellikle yükseklikten düşmeler hem ölümcül hem de ölümcül olmayan yaralanmaların önde gelen nedenidir. İnşaat sektöründe Yapı bilgi modellemesi (YBM), Genişletilmiş Gerçeklik (GG) ve Yapay Zekâ (YZ) gibi dijital teknolojiler, yapım üretkenliğini, verimliliğini ve güvenliğini artırmak için değerli araçlar olarak tanımlanmıştır. Bu araştırmada, yapım işlerinde yüksekte çalışma iş güvenliği denetimini kolaylaştırmak için Genişletilmiş Gerçeklik ve Yapay Zekânın entegrasyonu modeli önerilmektedir. Teorik çerçeveye ilişkin olarak iş süreci modeli ve sistem uygulama model entegrasyonu gösterilmektedir. Önerilen modelin değerlendirilmesi, hipotezlerin güvenilirliğini, geçerliliğini ve katkısının test edilmesi için bir Yapısal Eşitlik Model geliştirilmiştir. Araştırma bulguları, önerilen modelde kullanılan teknolojilerin entegrasyonun iş güvenliği denetimine olan olumlu etkisini ve önemini doğrulamaktadır. Önerilen model yüksek lokasyonda çalışan ekiplerin iş güvenliği bilgilerini analitik yeteneklerle dijitalleştirir ve karar verme sürecini optimize eder.

Keywords

• Yüksekte çalışma
• iş güvenliği denetimi
• genişletilmiş gerçeklik
• yapay zekâ
• entegrasyon modeli

References

1. Zhou, Z., Goh, Y.M. and Li, Q.,Overview and analysis of safety management studies in the construction industry. Safety Science, 72:337–350, 2015.
2. Zhang, S.Teizer,J. Lee, J., Eastman, C.M., Venugopal, M. Building Information Modeling (BIM) and Safety: Automatic Safety Checking of Construction Models and Schedules. Automation in Construction, 29:183–195, 2013.
3. Guo, H. Yu, Y., Skitmore, M. Visualization technology-based construction safety management: A review. Automation in Construction, 73:135–144, 2017.
4. Melzner J, Zhang S, Teizer J, Bargstädt H.J. A case study on automated safety compliance checking to assist fall protection design and planning in building information models. Construction Management and Economics, 31(6):661–674, 2013.
5. Zhang S, Sulankivi K, Kiviniemi M, Romo I, Eastman CM, Teizer J. BIM-based fall hazard identification and prevention in construction safety planning. Safety Science, 72:31–45, 2015.
6. Zhou, W., Zhao, T., Liu, W., & Tang, J. Tower crane safety on construction sites: A complex sociotechnical system perspective. Safety Science, 109(June):95–108, 2018.
7. Snowden, D. J., & Boone, M. E. A Leader’s Framework for Decision Making -Harvard Business Review. Harvard Business Review, pages 1–8, 2007.
8. Zhou Z, Irizarry J, Li Q.Applying advanced technology to improve safety management in the construction industry: a literature review. Construction Management and Economics, 31(6):606–622, 2013.

9. Kiani, A., Salman, A., Riaz, Z. (2014).Real-time environmental monitoring, visualization and notification system for construction H&S management. Journal of Information Technology in Construction, 19 (September 2013):72–91, 2014.
10. Hammad, A., Setayeshgar, S., Zhang, C., Asen, Y. Automatic generation of dynamic virtual fences as part of BIM-based prevention program for construction safety. Proceedings – Winter Simulation Conference, (December), 2012.
11. Park, C.S., Kim, H.J. A framework for construction safety management and visualization system. Automation in Construction, 33:95–103, 2013.
12. Nancy Leveson, N. A new accident model for engineering safer systems. Safety Science, 42(4):237–270, 2004.
13. Rasmussen, J. Risk management in a dynamic society: A modelling problem. Safety Science, 27(2-3):183–213, 1997.
14. Bureau of Labor Statistics. (2017). Survey of occupational injuries and illnesses chart data. https://www.bls.gov/iif/soii-chart-data-2017.htm
15. Kincl, L. D., Bhattacharya, A., Succop, P. A., Clark, C. S. Postural sway measurements: A potential safety monitoring technique for workers wearing personal protective equipment. Applied Occupational and Environmental Hygiene, 17(4), 256–266, 2002.
16. Huppert, D., Grill, E., Brandt, T. Down on heights? One in three has visual height intolerance. Journal of Neurology, 260(2), 597–604, 2013.
17. Salassa, J. R., Zapala, D. A. Love and fear of heights: The pathophysiology and psychology of height imbalance. Wilderness & Environmental Medicine, 20(4), 378–382, 2009.
18. Brandt, T., Arnold, F., Bles, W., & Kapteyn, T. S. The mechanism of physiological height vertigo: I. Theoretical approach and psychophysics. Acta Oto-Laryngologica, 89(3–6), 513–523, 1980.
19. Abed, H.R., Hatem, W.A., Jasim, N.A. Adopting BIM technology in fall prevention plans. Civil Eng. J. 5 (10), 2270–2281, 2019.
20. Goh, Y.M., Guo, B.H.W., FPSWizard: A web-based CBR-RBR system for supporting the design of active fall protection systems. Autom. Constr. 85, 40–50, 2018.
21. Karakhan, A., Gambatese, J., Rajendran, S., Application of choosing by advantages decision-making system to select fall-protection measures. In: Proc. 24th Ann.Conf of the Int’l. Group for Lean Construction, Boston, MA, USA, pp. 33–42, 2016.
22. Jokkaw, N., Suteecharuwat, P., Weerawetwat, P., Measurement of Construction Workers’ Feeling by Virtual Environment (VE) Technology for Guardrail Design in High-Rise Building Construction Projects. Eng. J. 21 (5), 161–177, 2017.
23. Abd Rahman, N., Goh, K.C., Goh, H.H., Omar, M.F.,Toh, T.C., Zin, M.,Asuhaimi, A.,MohdJaini, Z.,Yunus, R.,Rahmat, S.N. Accidents preventive practice for high-rise construction. In Proceedings of the MATEC Web of Conferences, Cape Town, SouthAfrica, 1–3 February 2016; EDP Sciences: Les Ulis, France, Volume 47, pp. 1–6.2016.
24. Im, H.-J.; Kwon, Y.-J.; Kim, S.-G.; Kim, Y.-K.; Ju, Y.-S.; Lee, H.-P. The characteristics of fatal occupational injuries in Korea’s construction industry, 1997–2004. Saf. Sci., 47, 1159–1162,2009.
25. Rubio-Romero, J.C.; Gámez, M.C.R.; Carrillo-Castrillo, J.A. Analysis of the safety conditions of scaffolding on construction sites. Saf. Sci., 55, 160–164, 2013
26. Hu, K.; Rahmandad, H., Smith-Jackson, T, Winchester,W. Factors influencing the risk of falls in the construction industry: A review of the evidence. Constr. Manag. Econ., 29, 397–416,2011.
27. Forteza, F.J.; Carretero-Gomez, J.M.; Sese, A. Occupational risks, accidents on sites and economic performance of construction firms. Saf. Sci. 2017, 94, 61–76.
28. Feng, Y.; Zhang, S.; Wu, P. Factors influencing workplace accident costs of building projects. Saf. Sci., 72, 97–104,2015.
29. Zhou, Z.; Goh, Y.M.; Li, Q. Overview and analysis of safety management studies in the construction industry. Saf. Sci., 72, 337–350,2015.
30. Amiri, M.; Ardeshir, A.; Zarandi, M.H.F. Fuzzy probabilistic expert system for occupational hazard assessment in construction. Saf. Sci., 93, 16–28, 2017.
31. Andersen, L.P.; Nørdam, L.; Joensson, T.; Kines, P.; Nielsen, K.J. Social identity, safety climate and self-reported accidents among construction workers. Constr. Manag. Econ., 36, 22–31,2018
32. Bilir, S., Gürcanli, G.E. (2018). A method for determination of accident probability in construction industry. Teknik Dergi, 29(4), 8537-8561.
33. Mistikoglu, G.; Gerek, I.H.; Erdis, E.; Usmen, P.E.M.; Cakan, H.; Kazan, E.E. Decision tree analysis of construction fall accidents involving roofers. Expert Syst. Appl., 42, 2256–2263, 2015.
34. Volk, R.; Stengel, J.; Schultmann, F. Building Information Modeling (BIM) for existing buildings—Literature review and future needs. Autom. Constr., 38, 109–127. 2014.
35. Scheffer, M.; Mattern, H.; König, M. BIM Project Management. In Building Information Modeling; Springer: Berlin/Heidelberg, Germany, pp. 235–249, 2018.
36. Akinlolu, M.; Haupt, T.C.; Edwards, D.J.; Simpeh, F. A bibliometric review of the status and emerging research trends in construction safety management technologies. Int. J. Constr. Manag., 1–13,2020.
37. Manzoor, B.; Othman, I.; Durdyev, S.; Ismail, S.; Wahab, M.H. Influence of Artificial Intelligence in Civil Engineering toward Sustainable Development-A Systematic Literature Review. Appl. Syst. Innov., 4, 52, 2021.
38. Alizadehsalehi, S.; Yitmen, I. The impact of field data capturing technologies on automated construction project progress monitoring. Procedia Eng. 2016, 161, 97–103.
39. Melzner, J.; Teizer, J.; Zhang, S.; Bargstaedt, H.-J. Object-oriented safety planning of building construction by using Building Information Modeling. Bauingenieur, 88, 471–479, 2013.
40. Martinez, J.G.; Gheisari, M.; Alarcón, L.F. UAV integration in current construction safety planning and monitoring processes: Case study of a high-rise building construction project in Chile. J. Manag. Eng., 36, 5020005,2020.
41. Skibniewski, M.J. Research trends in information technology applications in construction safety engineering and management. Front. Eng. Manag. 2015, 1, 246–259.
42. Bhagwat, K.; Delhi, V.S.K. Review of construction safety performance measurement methods and practices: A science mapping approach. Int. J. Constr. Manag., 1–15,2021.
43. Manzoor, B.; Othman, I.; Manzoor, M. Evaluating the critical safety factors causing accidents in high-rise building projects. Ain Shams Eng. J. 2021.
44. Sunindijo, R.Y.; Zou, P.X.W. Political skill for developing construction safety climate. J. Constr. Eng. Manag. 2012, 138, 605–612.
45. Xu, Q.; Chong, H.-Y.; Liao, P.-C. Collaborative information integration for construction safety monitoring. Autom. Constr., 102, 120–134, 2019.
46. Getuli, V.; Capone, P.; Bruttini, A.; Isaac, S. BIM-based immersive Virtual Reality for construction workspace planning: A safety-oriented approach. Autom. Constr., 114, 103160,2020.
47. Ji, Y.; Leite, F. Automated tower crane planning: Leveraging 4-dimensional BIM and rule-based checking. Autom. Constr., 93, 78–90,2018.
48. Braun, A.; Tuttas, S.; Borrmann, A.; Stilla, U. Improving progress monitoring by fusing point clouds, semantic data and computer vision. Autom. Constr., 116, 103210, 2020.
49. Thinakaran, V.; Othman, I. Factors of Safety Misconduct Affecting Safety Performance of Tall Building Construction Site. In Proceedings of the International Conference on Civil, Offshore and Environmental Engineering, Kuching, Malaysia, 13–15 June 2021; Springer: Singapore, pp. 620–630, 2021.
50. Newaz, M. T., Ershadi, M., Carothers, L., Jefferies, M., & Davis, P. A review and assessment of technologies for addressing the risk of falling from height on construction sites. Safety science, 147, 105618, 2022.
51. Winge, S., Albrechtsen, E., Mostue, B. A. Causal factors and connections in construction accidents. Safety science, 112, 130-141, 2019.
52. Le, Q.T.; Pedro, A.; Park, C.S. A Social Virtual Reality Based Construction Safety Education System for Experiential Learning.J. Intell. Robot. Syst., 79, 487–506,2015.
53. Kolar, Z.; Chen, H.; Luo, X. Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images.Autom. Constr., 89, 58–70, 2018.
54. Fang, Q.; Li, H.; Luo, X.; Ding, L.; Luo, H.; Li, C. Computer vision aided inspection on falling prevention measures for steeple jacksin an aerial environment. Autom. Constr., 93, 148–164,2018.
55. Rey-Merchán, M. D. C., Gómez-de-Gabriel, J. M., Fernández-Madrigal, J. A., & López-Arquillos, A. Improving the prevention of fall from height on construction sites through the combination of technologies. International journal of occupational safety and ergonomics, 28(1), 590-599, 2022.
56. Alizadehsalehi, S., Yitmen, I., Celik, T., & Arditi, D. (2020). The effectiveness of an integrated BIM/UAV model in managing safety on construction sites. International journal of occupational safety and ergonomics, 26(4), 829-844.
57. Sidani, A., Martins, J. P., & Soeiro, A. BIM approaches for enhanced health and safety status in construction-protocol for a systematic review: Protocol. International Journal of Occupational and Environmental Safety, 6(1), 1-8, (2022).
58. Harichandran, A., Johansen, K. W., Jacobsen, E. L., &Teizer, J. A conceptual framework for construction safety training using dynamic virtual reality games and digital twins. In ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction, Vol. 38, pp. 621-628, IAARC Publications, 2021.
59. Ogunseiju, O. R., Olayiwola, J., Akanmu, A. A., Nnaji, C. Digital twin-driven framework for improving self-management of ergonomic risks. Smart and Sustainable Built Environment, 10(3), 403-419, 2021.
60. Kaarlela, T., Pieskä, S., Pitkäaho, T. Digital twin and virtual reality for safety training. In 2020 11th IEEE International Conference on Cognitive Infocommunications (CogInfoCom) (pp. 000115-000120). IEEE, 2020.
61. Choi, S. H., Park, K. B., Roh, D. H., Lee, J. Y., Mohammed, M., Ghasemi, Y., &Jeong, H. An integrated mixed reality system for safety-aware human-robot collaboration using deep learning and digital twin generation. Robotics and Computer-Integrated Manufacturing, 73, 102258,2022.
62. Ramos-Hurtado, J.; Muñoz-La Rivera, F.; Mora-Serrano, J.; Deraemaeker, A.; Valero, I. Proposal for the Deployment of an Augmented Reality Tool for Construction Safety Inspection. Buildings, 12(4), 500, 2022.
63. Wu, S., Hou, L., Zhang, G. K., & Chen, H. Real-time mixed reality-based visual warning for construction workforce safety. Automation in Construction, 139, 104252, 2022.
64. Wolf, M., Teizer, J., Wolf, B., Bükrü, S., & Solberg, A. Investigating hazard recognition in augmented virtuality for personalized feedback in construction safety education and training. Advanced Engineering Informatics, 51, 101469, 2022.
65. Recal, F., Demirel, T. Comparison of machine learning methods in predicting binary and multi-class occupational accident severity. Journal of Intelligent & Fuzzy Systems, 40(6), 10981-10998, 2021.
66. Kazar, G., Comu, S. Developing a virtual safety training tool for scaffolding and formwork activities. Teknik Dergi, 33(2), 11729-11748, 2022.
67. Teizer, J., Johansen, K. W., & Schultz, C. The Concept of Digital Twin for Construction Safety. In Construction Research Congress 2022, pp. 1156-1165, 2022.
68. Guo, X., Ji, J., Khan, F., Ding, L., Tong, Q., A novel fuzzy dynamic Bayesian network for dynamic risk assessment and uncertainty propagation quantification in uncertainty environment. Saf. Sci. 141, 105285, 2021.
69. Liu, M., Tang, P., Liao, P.-C., Xu, L., Propagation mechanics from workplace hazards to human errors with dissipative structure theory. Saf. Sci. 126, 104661, 2020.
70. Liu, M., Xu, L., Liao, P.-C., Character-based hazard warning mechanics: A network of networks approach. Adv. Eng. Inf. 47, 101240, 2021.
71. Zhang, W., Zhu, S., Zhang, X., Zhao, T., Identification of critical causes of construction accidents in China using a model based on system thinking and case analysis. Saf. Sci. 121, 606–618, 2020.
72. Da Rocha Leao, B.B., Barkokebas Jr, B., Barkok´ebas, B., Zlatar, T., Risk management of falls from height by using the bim platform: a systematic review. Int. J. Develop. Res. 9 (11), 31267–31273, 2019.
73. Kim, Y., Jung, H., Koo, B., Kim, J., Kim, T., Nam, Y., Detection of pre-impact falls from heights using an inertial measurement unit sensor. Sensors 20 (18), 5388, 2020.
74. Darko, A., Chan, A.P., Yang, Y., Tetteh, M.O., Building information modeling (BIM)-based modular integrated construction risk management–Critical survey and future needs. Comput. Ind. 123, 103327, 2020.
75. Chen, H., Luo, X., Zheng, Z., Ke, J., A proactive workers’ safety risk evaluation framework based on position and posture data fusion. Autom. Constr. 98, 275–288, 2019.
76. Liu, H., He, Y., Hu, Q., Guo, J., Luo, L. Risk management system and intelligent decision-making for prefabricated building project under deep learning modified teaching-learning-based optimization. PLoS ONE 15 (7), e0235980, 2020.
77. Subedi, S., Pradhananga, N., Ergun, H., Monitoring Physiological Reactions of Construction Workers in Virtual Environment: Feasibility Study Using Noninvasive Affective Sensors. J. Legal Affairs Dispute Resolut. Eng. Construct. 13 (3), 04521016, 2021.
78. Wang, Q., Automatic checks from 3D point cloud data for safety regulation compliance for scaffold work platforms. Autom. Constr. 104, 38–51, 2019.
79. Park, J., Cho, Y.K., Khodabandelu, A., Sensor-based safety performance assessment of individual construction workers. Sensors 18 (11), 3897, 2018.
80. Hayat, A., Shan, M. Fall Detection System for Labour Safety. In: 2018 International Conference on Engineering, Applied Sciences, and Technology (ICEAST) IEEE, pp. 1–4, 2018.
81. Tekbas, G., Guven, G. BIM-based automated safety review for fall prevention. In: Ofluoglu, S., Isikdag, U., Ozener, O.O. (Eds.) 1st Eurasian BIM Forum, EBF 2019. Springer, pp. 80–90, 2020.
82. Lu, Y., Gong, P., Tang, Y., Sun, S., Li, Q., BIM-integrated construction safety risk assessment at the design stage of building projects. Autom. Constr. 124, 103553, 2021.
83. Park, H., Liu, R., 2020. Improving for construction safety design: Ontology model of a knowledge system for the prevention of falls. In: Construction Research Congress 2020: Safety, Workforce, and Education American Society of Civil Engineers (ASCE), pp. 463–471, 2020.
84. Shi, Y., Du, J., Ahn, C.R., Ragan, E., Impact assessment of reinforced learning methods on construction workers’ fall risk behavior using virtual reality. Autom. Constr. 104, 197–214, 2019.
85. Xu, Z., Zheng, N., Incorporating virtual reality technology in safety training solution for construction site of urban cities. Sustainability (Switzerland) 13 (1), 1–19, 2021.
86. Ahn, S., Kim, T., Park, Y.J., Kim, J.M., Improving Effectiveness of Safety Training at Construction Worksite Using 3D BIM Simulation. Adv. Civil Eng. 2020, 2020.
87. Chihming, W., Zexin, J., Yuxin, L., Songqing, H. & Zhongwei, Y. Investigation on the eye-tracking technology in hazard identification of building construction engineering. In: 2nd IEEE International Conference on Architecture, Construction, Environment and Hydraulics, ICACEH 2020 Institute of Electrical and Electronics Engineers Inc., pp. 32–35, 2020.
88. Fang, W., Ding, L., Luo, H., Love, P.E., Falls from heights: A computer vision-based approach for safety harness detection. Autom. Constr. 91, 53–61, 2018.
89. Łabęd´z, P., Skabek, K., Ozimek, P., Nytko, M., 2021. Histogram Adjustment of Images for Improving Photogrammetric Reconstruction. Sensors 21 (14), 4654.
90. Uzun, I. M., & Cebi, S. (2020). A novel approach for classification of occupational health and safety measures based on their effectiveness by using fuzzy kano model. Journal of Intelligent & Fuzzy Systems, 38(1), 589-600.
91. Wolf, C., Joye, D., Smith, T. W., & Fu, Y.-c. The SAGE handbook of survey methodology: Sage, 2016.
92. Bowling, A. Mode of questionnaire administration can have serious effects on data quality. Journal of public health, 27(3), 281-291, 2005.
93. Stern, M. J., Bilgen, I., &Dillman, D. A. The state of survey methodology: Challenges, dilemmas, and new frontiers in the era of the tailored design. Field Methods, 26(3), 284-301, 2014.
94. Zhang, W., Zhu, S., Zhang, X., Zhao, T., 2020. Identification of critical causes of construction accidents in China using a model based on system thinking and case analysis. Saf. Sci. 121, 606–618.
95. Zuluaga, C.M., Albert, A., Winkel, M.A., Improving safety, efficiency, andproductivity: evaluation of fall protection systems for bridge work using wearabletechnology and utility analysis. J. Construct. Eng. Manage. 146 (2), 04019107, 2020.
96. Begic, H.; Galic, M. A Systematic Review of Construction 4.0 in the Context of the BIM 4.0 Premise. Buildings, 11, 337, 2021.
97. Kitagawa, K., Taguchi, Y., Wada, C., & Toya, N. Step Length Estimation Based on Arm Accelerations for Wearable Fall Prevention Systems. International Journal of Applied, 13(2), 2020.
98. Liu, P., Xie, M., Bian, J., Li, H., Song, L., A hybrid PSO–SVM model based onsafety risk prediction for the design process in metro station construction. Int. J.Environ. Res. Public Health 17 (5), 1714, 2020.
99. Zhou, Y., Yang, Y., Yang, J.-B., Barriers to BIM implementation strategies in China.Eng., Construct. Arch. Manage. 26 (3), 554–574, 2019.