Development of Organizational Cybersecurity Awareness Scale (OCAS)

Year 2026, Volume: 41 Issue: 1, 136 - 155, 23.01.2026

Nimet Özgül Ünsal , Mehmet Akif Ocak

https://doi.org/10.16986/hunefd.1739282

Abstract

This study reports the development and validation of the Organizational Cybersecurity Awareness Scale (OCAS) for employees in public institutions. The scale comprises four dimensions: (1) Personal Awareness and Proactive Approach, (2) Legal and Regulatory Awareness, (3) Technical Protection and Implementation, and (4) Organizational Policy Awareness, with a total of 26 items. Data were collected in 2021 from 110 employees working in two public organizations to evaluate the scale’s EFA analysis. Exploratory Factor Analysis (EFA) was conducted to establish the dimensional structure of the scale. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.942, indicating excellent factorability. The four-factor solution explained 60.02% of the total variance: Factor 1 (Personal Awareness and Proactive Approach) accounted for 39.54%, Factor 2 (Legal and Regulatory Awareness) for 11.19%, Factor 3 (Technical Protection and Implementation) for 4.76%, and Factor 4 (Organizational Policy Awareness) for 4.53%. All 26 items demonstrated satisfactory factor loadings, and no items required deletion. Confirmatory Factor Analysis (CFA) was performed to test the hypothesized factor structure. The overall scale demonstrated excellent internal consistency (Cronbach’s α = 0.936). Although the chi-square test was significant (χ² = 7952.667, df = 325, N = 544, p < .05), which is common in large samples, other fit indices indicated acceptable model fit: Standardized RMR = 0.085 and RMSEA = 0.083. To test the hypothesized four-factor structure derived from EFA, CFA was performed by using another 544 employees. These results support the four-dimensional structure and demonstrate that the OCAS is a reliable and valid instrument for measuring cybersecurity awareness among public sector employees.

Keywords

Cybersecurity awareness , public employees , scale development , organizational security , psychometric validation

References

• Abawajy, J. (2014). User preference of cyber security awareness delivery methods. Behaviour & Information Technology, 33(3), 237–248.
• Al-Shanfari, I., Yassin, W., Tabook, N., et al. (2022). Determinants of Information Security Awareness and Behaviour Strategies in Public Sector Organizations among Employees. International Journal of Advanced Computer Science and Applications, 13(8), 855-864. https://doi.org/10.14569/ijacsa.2022.0130855
• AlMindeel, R., & Martins, J. T. (2020). Information security awareness in a developing country context: Insights from the government sector in Saudi Arabia. Information Technology & People, 34(3), 770-792. https://doi.org/10.1108/ITP-06-2019-0269
• Almuqrin, A., Mutambik, I., Zhang, Z., et al. (2024). Cracking the Code: Unraveling the Determinants of Cybersecurity Awareness Among University Students. Journal of Organizational and End User Computing, 36(1), 1-25. https://doi.org/10.4018/joeuc.345933
• Arpaci, I., & Aslan, Ö. (2022). Development of a scale to measure cybercrime-awareness on social media. Journal of Computer Information Systems, 63(3), 695–705. https://doi.org/10.1080/08874417.2022.2101160
• Arpaci, I., & Sevinc, K. (2022). Development of the cybersecurity scale (CS-S): Evidence of validity and reliability. Information Development, 38(2), 218-226.
• Aşan, C. (2024). Developing a measurement scale to assess the perception of cybersecurity among employees in the maritime industry. Journal of Naval Science and Engineering, 20(2), 135-162. https://doi.org/10.56850/jnse.1485985
• Barrett, P. (2007). Structural equation modelling: Adjudging model fit. Personality and Individual Differences, 42(5), 815-824. https://doi.org/10.1016/j.paid.2006.09.018
• Bognár, F., & Bottyán, L. (2024). Development and Validation of a Personal Cybersecurity Awareness Scale. Sensors, 24(11), 3543. https://doi.org/10.3390/s24113543
• Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Publications.
• Chapman, T. A. (2019). Factors Affecting Perceptions of Cybersecurity Readiness Among Workgroup IT Managers [Doctoral dissertation]. https://www.proquest.com/docview/2303848162
• Creasey, J. (2013). Cybersecurity incident response guide. https://www.crestapproved.org/wpcontent/uploads/2014/11/CSIRProcurementGuide.pdf
• DeVellis, R. F. (2017). Scale Development: Theory and Applications (4th ed.). Sage Publications.
• Di Nocera, F., Ricciardi, O., Longo, Y., Ciani, G., & Ferlazzo, F. (2024). The CAIN Inventory: Measurement Invariance and Construct Validity of a Measure of Cybersecurity Awareness. Cyberpsychology, Behavior, and Social Networking, 27(3), 183-190. https://doi.org/10.1089/cyber.2023.0350
• Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299. https://doi.org/10.1037/1082-989X.4.3.272
• Fadhil, A., & Yazid, S. (2023). Measurement of Employee Information Security Awareness: A Case Study of National Civil Service Agency. The Indonesian Journal of Computer Science, 12(6).
• Faklaris, C., Dabbish, L., & Hong, J. I. (2022). SA-13: A Validated Security Attitudes Inventory. ACM Transactions on Computer-Human Interaction, 29(6), Article 51. https://doi.org/10.1145/3532867
• Fox, R. J. (2010). Confirmatory factor analysis. In Wiley Encyclopedia of Management. https://doi.org/10.1002/9781444316568.WIEM02060
• Georgiadou, A., Mouzakitis, S., Bounas, K., et al. (2020). A Cyber-Security Culture Framework for Assessing Organization Readiness. Journal of Computer Information Systems, 62(3), 452-462. https://doi.org/10.1080/08874417.2020.1845583
• Goode, S., Hoehle, H., Venkatesh, V., & Brown, S. A. (2018). User compensation as a data breach recovery action: An investigation of the Sony PlayStation Network breach. MIS Quarterly, 41(3), 703-727. https://doi.org/10.25300/MISQ/2017/41.3.03
• Güldüren, C., Çetinkaya, L., & Keser, H. (2016). Ortaöğretim Öğrencilerine Yönelik Bilgi Güvenliği Farkındalık Ölçeği (BGFÖ) Geliştirme Çalışması. İlköğretim Online, 15(2). https://doi.org/10.17051/io.2016.27218.
• Hadlington, L. (2017). Human factors in cybersecurity; examining the link between Internet addiction, impulsivity, attitudes towards cybersecurity, and risky cybersecurity behaviours. Heliyon, 3(7), e00346. https://doi.org/10.1016/j.heliyon.2017.e00346
• He, W., & Zhang, Z. J. (2019). Enterprise cybersecurity training and awareness programs: Recommendations for success. Journal of Organizational Computing and Electronic Commerce, 29(4), 249–257. https://doi.org/10.1080/10919392.2019.1611528
• Hijji, M., & Alam, G. (2022). Cybersecurity Awareness and Training (CAT) Framework for Remote Working Employees. Sensors, 22(22), 8663. https://doi.org/10.3390/s22228663
• Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104-121. https://doi.org/10.1177/109442819800100106
• Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.
• Jacobs, C., Horsch, A., Adda, M., & Shiraz, M. (2023). Measuring the Effectiveness of Corporate Security Awareness Programs: A Systematic Review. Computers & Security, 126, 103055. https://doi.org/10.1016/j.cose.2022.103055
• Kannelønning, K., & Katsikas, S. K. (2023). A systematic literature review of how cybersecurity-related behavior has been assessed. Information & Computer Security, 31(5), 631-660. https://doi.org/10.1108/ics-08-2022-0139
• Kävrestad, J., Burvall, F., & Nohlberg, M. (2024). A taxonomy of factors that contribute to organizational cybersecurity awareness (CSA). Information & Computer Security. https://doi.org/10.1108/ics-11-2023-0209
• Kävrestad, J., Nohlberg, M., & Ramezanian, S. (2024). A Taxonomy of Cybersecurity Awareness Activities in Organizations. Computers & Security, 138, 103654. https://doi.org/10.1016/j.cose.2023.103654
• Keser, H., & Güldüren, C. (2015). Development of information security awareness scale. Kastamonu Education Journal, 23(3), 1167–1184. https://kefdergi.kastamonu.edu.tr/index.php/Kefdergi/article/view/531/233
• Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating Training Programs: The Four Levels (3rd ed.). Berrett-Koehler Publishers.
• Korovessis, P., Furnell, S., Papadaki, M., & Haskell-Dowland, P. (2017). A toolkit approach to information security awareness and education. Journal of Cybersecurity Education, Research and Practice, 2017(2), Article 5.
• Kovačević, M. (2022). Online Deliberation and Personal Identity. EMERGE 2022 Book of Abstracts, 32-33.
• Kruger, H. A., & Kearney, W. D. (2006). A prototype for assessing information security awareness. Computers & Security, 25(4), 289-296. https://doi.org/10.1016/j.cose.2006.02.008
• Kumar, S., Biswas, B., Bhatia, M. S., et al. (2021). Antecedents for enhanced level of cyber-security in organisations. Journal of Enterprise Information Management, 35(4/5), 1094-1119. https://doi.org/10.1108/JEIM-06-2020-0240
• Langner, R. (2011). Stuxnet: Dissecting a cyberwarfare weapon. IEEE Security & Privacy, 9(3), 49-51. https://doi.org/10.1109/MSP.2011.67
• Li, L., He, W., Xu, L., Ash, I., Anwar, M., & Yuan, X. (2019). Investigating the impact of cybersecurity policy awareness on employees’ cybersecurity behavior. International Journal of Information Management, 45, 13-24. https://doi.org/10.1016/j.ijinfomgt.2018.10.017
• MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84-99. https://doi.org/10.1037/1082-989X.4.1.84
• Marnewick, A., & von Solms, S. H. (2022). Measuring Cybersecurity Awareness: A Comprehensive Approach. Information & Computer Security, 30(3), 437-458. https://doi.org/10.1108/ICS-09-2021-0138
• Mihai, I. C. (2024). AR-in-a-Box: A Structured 8-Step Framework for Cybersecurity Awareness. Proceedings of CyberCon 2024. https://doi.org/10.19107/cybercon.2024.05
• Nikel, M., & Amaechi, C. V. (2022). An Empirical Study of Employee Awareness on Cybersecurity in Cameroon. Journal of Cybersecurity and Privacy, 2(2), 374-397. https://doi.org/10.3390/jcp2020020
• Nwachukwu, U., Vidgren, J., Niemimaa, M., et al. (2023). Do SETA Interventions Change Security Behavior? – A Literature Review. Proceedings of the 56th Hawaii International Conference on System Sciences, 6344-6353. https://doi.org/10.24251/hicss.2023.762
• Parsons, K., McCormac, A., Butavicius, M., Pattinson, M., & Jerram, C. (2017). Determining employee awareness using the Human Aspects of Information Security Questionnaire (HAIS-Q). Computers & Security, 42, 165-176. https://doi.org/10.1016/j.cose.2013.12.003
• Pasipamire, E. (2025). Building a Culture of Cybersecurity Awareness in Libraries: A Systematic Review of Best Practices and Frameworks. Proceedings of the European Conference on Information Warfare and Security, 24(1), 357-365. https://doi.org/10.34190/eccws.24.1.3608
• Puhakainen, P., & Siponen, M. (2010). Improving employees’ compliance through information systems security training: An action research study. MIS Quarterly, 34(4), 757-778. https://doi.org/10.2307/25750704
• Salahdine, F., & Kaabouch, N. (2019). Social Engineering Attacks: A Survey. Future Internet, 11(4), 89. https://doi.org/10.3390/fi11040089
• Schütz, A. E., & Fertig, T. (2023). The Forgotten Model – Validating the Integrated Behavioral Model in Context of Information Security Awareness. Proceedings of the 56th Hawaii International Conference on System Sciences, 6809-6818. https://doi.org/10.24251/hicss.2023.828
• Shaw, R. S., Chen, C. C., Harris, A. L., & Huang, H. J. (2009). The impact of information richness on information security awareness training effectiveness. Computers & Education, 52(1), 92-100. https://doi.org/10.1016/j.compedu.2008.06.011
• Sulaiman, N. S., Fauzi, M. A., Hussain, S., et al. (2022). Cybersecurity Behavior among Government Employees: The Role of Protection Motivation Theory and Responsibility in Mitigating Cyberattacks. Information, 13(9), 413. https://doi.org/10.3390/info13090413
• Steinbart, P. J., Raschke, R. L., Gal, G., & Dilla, W. N. (2016). SECURQUAL: An instrument for evaluating the effectiveness of enterprise information security programs. Journal of Information Systems, 30(1), 71–92. https://doi.org/10.2308/isys-51257
• Tabachnick, B. G., & Fidell, L. S. (2013). Using Multivariate Statistics (6th ed.). Pearson.
• Vortia, W. (2025). Modelling cybersecurity awareness, perceived threats and secure online behavioral intentions among Ghanaian university students: A PLS-SEM Approach. Magna Scientia Advanced Research and Reviews, 14(2), 114-131. https://doi.org/10.30574/msarr.2025.14.2.0094
• Vrhovec, S., & Markelj, B. (2024). We need to aim at the top: Factors associated with cybersecurity awareness of cyber and information security decision-makers. arXiv preprint. https://doi.org/10.48550/arxiv.2404.04725
• Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806-838. https://doi.org/10.1177/0011000006288127
• Zwilling, M., Klien, G., Lesjak, D., Wiechetek, Ł., Çetin, F., & Başım, H. N. (2022). Cyber security awareness, knowledge and behavior: A comparative study. Journal of Computer Information Systems, 62(1), 82–97.