Modeling digital competence in education: A bifactor-ESEM approach to general and domain-specific skills

Abstract

Across European and Western Balkan countries, a persistent misalignment remains between graduates’ digital skill levels and the rapidly evolving demands of the contemporary workforce. Addressing this gap requires evidence-based interventions grounded in multidimensional and psychometrically rigorous assessments of digital competence. This study applies a bifactor exploratory structural equation modeling (B-ESEM) framework to evaluate university students’ functional digital skills using items from the Youth Digital Skills Indicator. The model integrates a general digital competence factor with four domain-specific dimensions, capturing both shared and unique variance across skill areas. Using data from 603 students, results indicate a strong general digital competence factor alongside meaningful domain-specific contributions. The model demonstrates full measurement invariance across gender and shows predictive validity for technology affinity and digital self-efficacy. These findings underscore the relevance of conceptualizing digital competence as both transversal and domain-sensitive when informing curriculum design and educational policy. The study provides a validated and flexible measurement framework that can guide future research and targeted digital skill enhancement initiatives.

Keywords

• Digital competence
• Bifactor-ESEM
• Educational policy
• Measurement invariance
• Domain-specific skills
• Structural equation modeling

References

1. Alamer, A. (2022). Exploratory structural equation modeling (ESEM) and bifactor ESEM for construct validation purposes: Guidelines and applied example. Research Methods in Applied Linguistics, 1(3), Article 100027. https://doi.org/10.1016/j.rmal.2022.100027
2. Alamer, A., & Marsh, H.W. (2022). Exploratory structural equation modeling in second language research: An applied example using the dualistic model of passion. Studies in Second Language Acquisition, 44(1), 1-22. https://doi.org/10.1017/S0272263121000863
3. Andrade, H.L. (2019). A critical review of research on student self-assessment. Frontiers in Education, 4, Article 87. https://doi.org/10.3389/feduc.2019.00087
4. Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397-438. https://doi.org/10.1080/1070551090300820
5. Beierlein, C., Kemper, C.J., Kovaleva, A., & Rammstedt, B. (2013). Short Scale for Measuring General Self efficacy Beliefs (ASKU). Methods, Data, Analyses, 7(2), 251 278. https://doi.org/10.12758/mda.2013.014
6. Burnham, K.P., & Anderson, D.R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods & Research, 33(2), 261 304. https://doi.org/10.1177/0049124104268644
7. Carretero, S., Vuorikari, R., & Punie, Y. (2017). DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use. Publications Office of the European Union. https://doi.org/10.2760/38842
8. Chen, F.F. (2007). Sensitivity of goodness-of-fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464 504. https://doi.org/10.1080/10705510701301834

9. Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233 255. https://doi.org/10.1207/S15328007SEM0902_5
10. Clifford, I., Kluzer, S., Troia, S., Jakobsone, M., Zandbergs, U., Vuorikari, R., Punie, Y., Castaño Muñoz, J., Centeno Mediavilla, I.C., O’Keeffe, W., & Cabrera Giraldez, M. (2020). DigCompSat: A self-reflection tool for the European Digital Competence Framework for Citizens. Publications Office of the European Union. https://publications.jrc.ec.europa.eu/repository/handle/JRC123226
11. Codagnone, C., Liva, G., Gunderson, L., Misuraca, G., & Rebesco, E. (2021). Europe’s digital decade and autonomy. Publication for the Committee on Industry, Research and Energy (ITRE), Policy Department for Economic, Scientific and Quality of Life Policies, European Parliament.
12. Collier, J.E. (2020). Applied structural equation modeling using AMOS: Basic to advanced techniques. Routledge. https://doi.org/10.4324/9781003018414
13. Cromhout, A., Schutte, L., & Wissing, M.P. (2021). Factor structure and measurement invariance of the Basic Psychological Needs Scale in three South African samples: A bifactor exploratory structural equation modelling approach. Psychological Reports, 124(6), 2491 2514. https://doi.org/10.1177/00332941211025275
14. Dierendonck, C. (2023). Measuring the classroom level of the Dynamic Model of Educational Effectiveness through teacher self-report: Development and validation of a new instrument. Frontiers in Education, 8, Article 1281431. https://doi.org/10.3389/feduc.2023.1281431
15. European Commission. (2021). Digital education action plan (2021 2027). https://education.ec.europa.eu/focus-topics/digital-education/action-plan
16. European Commission. (2024). Report on the state of the digital decade 2024. https://digital strategy.ec.europa.eu/en/library/report-state-digital-decade-2024
17. European Commission. (2025). Europe’s Digital Decade: Digital targets for 2030. https://commission.europa.eu/strategy and policy/priorities 2019 2024/europe fit digital age/europes digital decade digital-targets-2030_en
18. Fabrigar, L.R., Wegener, D.T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272 299. https://doi.org/10.1037/1082-989X.4.3.272
19. Falma, F.O., & Putra, A.H. (2025). Correlation between academic self-efficacy and digital competence: An empirical study on first-year students. Journal of Counseling, Education and Society, 6(1), 1-9. https://jurnal.iicet.org/index.php/jces/article/download/5403/pdf
20. Fang, C.-S., Li, C.-H., Ma, S.-C., Hwang, F.-M., Martins, M. M., & Chou, F.-H. (2025). Psychometric properties of the Chinese version of the Quality of Nursing Care Scale among hospital nurses: A bifactor exploratory structural equation modeling analysis. BMC Nursing, 24(1), 1 13. https://doi.org/10.1186/s12912-025-03012-0
21. Ferrari, A. (2013). DIGCOMP: A framework for developing and understanding digital competence in Europe. Publications Office of the European Union. https://doi.org/10.2788/52966
22. Finney, S.J., & DiStefano, C. (2013). Nonnormal and categorical data in structural equation models. In G.R. Hancock & R.O. Mueller (Eds.), A second course in structural equation modeling (pp. 439-492). Information Age Publishing.
23. Gegenfurtner, A. (2022). Bifactor exploratory structural equation modeling: A meta-analytic review of model fit. Frontiers in Psychology, 13, Article 1037111. https://doi.org/10.3389/fpsyg.2022.1037111
24. Gignac, G.E. (2016). The higher-order model imposes a proportionality constraint: That is why the bifactor model tends to fit better. Intelligence, 55, 57-68. https://doi.org/10.1016/j.intell.2016.01.009
25. Gillet, N., Morin, A.J.S., Huart, I., Colombat, P., & Fouquereau, E. (2020). The forest and the trees: Investigating the globality and specificity of employees’ basic need satisfaction at work. Journal of Personality Assessment, 102(5), 702-713. https://doi.org/10.1080/00223891.2019.1591426
26. González-Mujico, F.L. (2024). Measuring student and educator digital competence beyond self-assessment: Developing and validating two rubric-based frameworks. Education and Information Technologies, 29, 13299-13324. https://doi.org/10.1007/s10639-023-12363-7
27. Gorsuch, R.L. (1983). Factor analysis (2nd ed.). Lawrence Erlbaum Associates.
28. Hancock, G.R., & Mueller, R.O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future (pp. 195–216). Scientific Software International.
29. Helsper, E.J., van Deursen, A.J.A.M., & Eynon, R. (2020a). Digital skills, digital inequalities and the digital divide: Theories and evidence. In J.A.G.M. van Dijk & A.J.A.M. van Deursen (Eds.), Digital skills: Unlocking the information society (pp. 21-42). Springer. https://doi.org/10.1007/978-3-031-46929-9_3
30. Helsper, E.J., Schneider, L.S., van Deursen, A.J.A.M., & van Laar, E. (2020b). The Youth Digital Skills Indicator: Report on the conceptualisation and development of the ySKILLS digital skills measure. ySKILLS / KU Leuven.
31. Hoffman, L. (2020). Lecture 7: Measurement invariance. PSQF 6249: Structural Equation Modeling. University of Iowa. https://www.lesahoffman.com/PSQF6249_Fall2020/PSQF6249_Lecture7_Invariance.pdf
32. Howard, J.L., Gagné, M., Morin, A.J.S., & Forest, J. (2018). Using bifactor exploratory structural equation modeling to test for a continuum structure of motivation. Journal of Management, 44(7), 2638-2664. https://doi.org/10.1177/0149206316645653
33. Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
34. Jin, Y. (2020). A note on the cutoff values of alternative fit indices to evaluate measurement invariance for ESEM models. International Journal of Behavioral Development, 44(2), 166 174. https://doi.org/10.1177/0165025419866911
35. Jokisch, C., Schlimbach, R., Strzebkowski, M., Hobert, S., Kurpejovic, H., & Schumann, M. (2025). Empirical insights on the impact of technology affinity on programming skill development through an adaptive game-based learning system. In Proceedings of the 58th Hawaii International Conference on System Sciences (pp. 4885 4894). HICSS Conference Office. https://doi.org/10.24251/HICSS.2025.588
36. Jöreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202. https://doi.org/10.1007/BF02289343
37. Kaplan, D. (2009). Structural equation modeling: Foundations and extensions (2nd ed.). SAGE Publications. https://doi.org/10.4135/9781452226576
38. Kim, H.-Y. (2013). Statistical notes for clinical researchers: Assessing normal distribution (2) using skewness and kurtosis. Restorative Dentistry & Endodontics, 38(1), 52 54. https://doi.org/10.5395/rde.2013.38.1.52
39. Kyriazos, T., & Poga Kyriazou, M. (2023). Applied psychometrics: Estimator considerations in commonly encountered conditions in CFA, SEM, and EFA practice. Psychology, 14(5), 799–828. https://doi.org/10.4236/psych.2023.145043
40. Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936 949. https://doi.org/10.3758/s13428-015-0619-7
41. Marsh, H.W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit in structural equation models. In A. Maydeu-Olivares & J.J. McArdle (Eds.), Contemporary psychometrics: A festschrift for Roderick P. McDonald (pp. 275-340). Lawrence Erlbaum Associates.
42. Marsh, H.W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A.J.S., Trautwein, U., & Nagengast, B. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16(3), 439 476. https://doi.org/10.1080/10705510903008220
43. Marsh, H.W., Morin, A.J.S., Parker, P.D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85-110. https://doi.org/10.1146/annurev-clinpsy-032813-153700
44. Mattar, J., Ramos, D.K., & Lucas, M.R. (2022). DigComp based digital competence assessment tools: Literature review and instrument analysis. Education and Information Technologies, 27, 10843-10867. https://doi.org/10.1007/s10639-022-11034-3
45. Mishra, P., & Koehler, M.J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x
46. Morin, A.J.S., Arens, A.K., & Marsh, H.W. (2016). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Structural Equation Modeling, 23(1), 116 139. https://doi.org/10.1080/10705511.2014.961800
47. Morin, A.J.S., Myers, N.D., & Lee, S. (2020a). Modern factor analytic techniques: Bifactor models, exploratory structural equation modeling (ESEM), and bifactor-ESEM. In G. Tenenbaum & R.C. Eklund (Eds.), Handbook of sport psychology (4th ed., pp. 48-82). Wiley.
48. Morin, A.J.S., Boudrias, J.S., Marsh, H.W., McInerney, D.M., & Madore, I. (2020b). Integrating bifactor-ESEM within CFA: A framework for modeling complex constructs in applied settings. Psychological Methods, 25(4), 537-560. https://doi.org/10.1037/met0000250
49. Morin, A.J.S., Marsh, H.W., & Nagengast, B. (2020c). Modern factor analytic techniques: Bifactor models, exploratory structural equation modeling (ESEM), and bifactor–ESEM. In G. Tenenbaum, R. Eklund, & N. Boiangin (Eds.), Handbook of sport psychology: Exercise, methodologies, & special topics (4th ed., pp. 1044-1073). John Wiley & Sons.
50. Morin, A.J.S. (2023). Exploratory structural equation modeling. In R.H. Hoyle (Ed.), Handbook of structural equation modeling (2nd ed., pp. 503-524). Guilford Press.
51. Muthén, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52(3), 431-462.
52. Muthén, L.K. (2012, February 3). Re: Measurement invariance. Mplus Discussion Forum. https://www.statmodel.com/discussion/messages/11/8794.html
53. OECD. (2024). Western Balkans competitiveness outlook 2024: Regional profile, competitiveness and private sector development. OECD Publishing. https://doi.org/10.1787/170b0e53-en
54. Raykov, T. (2012). Scale construction and development using structural equation modeling. In R.H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 472-492). Guilford Press.
55. Redecker, C., & Punie, Y. (2017). European framework for the digital competence of educators: DigCompEdu. Publications Office of the European Union. https://doi.org/10.2760/159770
56. Reise, S.P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667-696. https://doi.org/10.1080/00273171.2012.715555
57. Reise, S.P., Bonifay, W.E., & Haviland, M.G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129-140.
58. Rodriguez, A., Reise, S.P., & Haviland, M.G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137 150. https://doi.org/10.1037/met0000045
59. Rubach, C., Quast, J., Porsch, R., & Arndt, M. (2024). Exploring the link between basic ICT competence beliefs and technology use for student teachers and teachers: A use of bifactor exploratory structural equation models. Unterrichtswissenschaft, 51, 533 557. https://doi.org/10.1007/s42010-023-00188-9
60. Rubin, D.B. (1976). Inference and missing data. Biometrika, 63(3), 581 592. https://doi.org/10.1093/biomet/63.3.581
61. Runge, I., Lazarides, R., Rubach, C., Richter, D., & Scheiter, K. (2023). Teacher-reported instructional quality in the context of technology-enhanced teaching: The role of teachers’ digital competence-related beliefs in empowering learners. Computers & Education, 198, Article 104761. https://doi.org/10.1016/j.compedu.2023.104761
62. Saltos-Rivas, R., Novoa-Hernández, P., & Serrano Rodríguez, R. (2022). How reliable and valid are the evaluations of digital competence in higher education? A systematic mapping study. SAGE Open, 12(1). https://doi.org/10.1177/21582440211068492
63. Sánchez-Oliva, D., Morin, A.J.S., Teixeira, P.J., Carraça, E.V., Palmeira, A.L., & Silva, M.N. (2017). A bifactor exploratory structural equation modeling representation of the structure of the Basic Psychological Needs at Work Scale. Journal of Vocational Behavior, 98, 178 187. https://doi.org/10.1016/j.jvb.2016.11.002
64. Schauffel, N., Schmidt, I., Peiffer, H., & Ellwart, T. (2021). Self-concept related to information and communication technology: Scale development and validation. Computers in Human Behavior Reports, 4, Article 100149. https://doi.org/10.1016/j.chbr.2021.100149
65. Shi, D., Maydeu-Olivares, A., & DiStefano, C. (2018). The relationship between the standardized root mean square residual and model misspecification in factor analysis models. Multivariate Behavioral Research, 53(5), 676-694. https://doi.org/10.1080/00273171.2018.1476221
66. Siddiq, F., Hatlevik, O.E., Olsen, R.V., Throndsen, I., & Scherer, R. (2016). Taking a future perspective by learning from the past: A systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educational Research Review, 19, 58 84. https://doi.org/10.1016/j.edurev.2016.05.002
67. Sillat, L.H., Tammets, K., & Laanpere, M. (2021). Digital competence assessment methods in higher education: A systematic literature review. Education Sciences, 11(8), 402. https://doi.org/10.3390/educsci11080402
68. Stucky, B.D., & Edelen, M.O. (2015). Using hierarchical IRT models to create unidimensional measures from multidimensional data. In S.P. Reise & D.A. Revicki (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (pp. 183-206). Routledge.
69. Swami, V., Maiano, C., & Morin, A.J.S. (2024). A guide to exploratory structural equation modeling (ESEM) and bifactor ESEM in body image research. [Preprint]. https://smslabstats.weebly.com/uploads/1/0/0/6/100647486/swami_et_al_2023_esem_guide_bi_prepub.pdf
70. Tóth-Király, I., Morin, A.J.S., Bőthe, B., Orosz, G., & Rigó, A. (2018). Investigating the multidimensionality of need fulfillment: A bifactor exploratory structural equation modeling representation. Structural Equation Modeling: A Multidisciplinary Journal, 25(2), 267 286. https://doi.org/10.1080/10705511.2017.1374867
71. UNESCO. (2017). Albania: Educational policy review: Issues and recommendations. https://unesdoc.unesco.org/ark:/48223/pf0000247993
72. van Zyl, L.E., & Veldsman, D. (2024). The psychometric properties of the Psychological Work Immersion Scale: An ESEM vs. ICM-CFA approach. International Journal of Applied Positive Psychology, 9(1), 251-286. https://doi.org/10.1007/s41042-023-00121-6
73. Vuorikari, R., Jerzak, N., Karpinski, Z., Pokropek, A., & Tudek, J. (2022). Measuring digital skills across the EU: Digital Skills Indicator 2.0. Publications Office of the European Union. https://doi.org/10.2760/897803
74. Vuorikari, R., Pokropek, A., & Castaño Muñoz, J. (2025). Enhancing digital skills assessment: Introducing compact tools for measuring digital competence. Technology, Knowledge and Learning. https://doi.org/10.1007/s10758-025-09825-x
75. Vuorikari, R., Punie, Y., Carretero, S., & Van den Brande, G. (2016). DigComp 2.0: The digital competence framework for citizens - Update phase 1: The conceptual reference model. Publications Office of the European Union. https://doi.org/10.2791/11517
76. Wessel, D., Attig, C., & Franke, T. (2019). ATI S: An ultra short scale for assessing affinity for technology interaction in user studies. In F. Alt, A. Bulling, & T. Döring (Eds.), Proceedings of the Mensch und Computer 2019 (MuC ’19) (pp. 147-154). Association for Computing Machinery. https://doi.org/10.1145/3340764.3340766
77. Ximénez, C., Maydeu-Olivares, A., Shi, D., & Revuelta, J. (2022b). Assessing cutoff values of SEM fit indices: Advantages of the unbiased SRMR index and its cutoff criterion based on communality. Structural Equation Modeling: A Multidisciplinary Journal, 29(3), 368 380. https://doi.org/10.1080/10705511.2021.1992596
78. Ximénez, C., Revuelta, J., & Castañeda, R. (2022a). What are the consequences of ignoring cross-loadings in bifactor models? A simulation study assessing parameter recovery and sensitivity of goodness of fit indices. Frontiers in Psychology, 13, Article 923877. https://doi.org/10.3389/fpsyg.2022.923877