Development and psychometric evaluation of the E-sports Attitude Scale

Abstract

This study aimed to develop and conduct a comprehensive psychometric evaluation of a scale designed to measure university students’ attitudes toward e-sports. The research was carried out using a correlational research design. Two independent samples selected using maximum variation sampling were utilized: 797 students participated in the exploratory factor analysis (EFA), while 794 students were included in the confirmatory factor analysis (CFA). The EFA results revealed a four-factor structure -socialization, individualization, career, and risks- explaining 64% of the total variance. The CFA supported this structure with excellent model fit indices (CFI = .99, TLI = .98, RMSEA = .05, SRMR = .06). Reliability analyses demonstrated high internal consistency, with Cronbach’s α ranging from .89 to .93, McDonald’s ω between .90 and .94, and stratified α (.94) as well as Revelle’s ω (.97) for the total scale. Furthermore, measurement invariance across gender was examined and supported, indicating that the scale functions equivalently for male and female respondents. Additionally, Differential Item Functioning (DIF) analysis based on ordinal logistic regression revealed no significant gender-related item bias, reinforcing item-level fairness. These findings provide strong evidence for the validity, reliability, and fairness of the E-sports Attitude Scale, confirming its suitability for measuring attitudes toward e-sports in higher education contexts.

Keywords

• E-sports
• Attitude
• Scale development
• Validation
• University students

References

1. Adachi, P.J.C., & Willoughby, T. (2013a). More than just fun and games: The longitudinal relationships between strategic video games, self-reported problem solving skills, and academic grades. Journal of Youth and Adolescence, 42(7), 1041-1052. https://doi.org/10.1007/s10964-013-9913-9
2. Adachi, P.J.C., & Willoughby, T. (2013b). Do video games promote positive youth development? Journal of Adolescent Research, 28(2), 155-165. https://doi.org/10.1177/0743558412464522
3. Agresti, A. (2010). Analysis of ordinal categorical data (2nd ed.). Wiley. https://doi.org/10.1002/9780470594001
4. Akkaya, S., Şen, B.G., & Kapıdere, M. (2021). A multi-directional assessment related to E-sports as a new game experience field and socialising tool. Atatürk Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 25(3), 968-988. https://doi.org/10.53487/ataunisosbil.944458
5. Bediou, B., Adams, D.M., Mayer, R.E., Tipton, E., Green, C.S., & Bavelier, D. (2018). Meta-analysis of action video game impact on perceptual, attentional, and cognitive skills. Psychological Bulletin, 144(1), 77–110. https://doi.org/10.1037 /bul0000130
6. Brown, T.A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.
7. Büyüköztürk, Ş. (2010). Handbook of data analysis for the social sciences: Statistics, research design, SPSS applications, and interpretation (11th ed.). Pegem Akademi.
8. Byrne, B. (2016). Structural equation modeling with Amos (3rd ed.). Routledge.

9. Byrne, B.M., Shavelson, R.J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456-466. https://doi.org/10.1037/0033-2909.105.3.456
10. Chen, F.F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464 504. https://doi.org/10.1080/10705510701301834
11. Choi, S.W., Gibbons, L.E., & Crane, P.K. (2011). lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. Journal of Statistical Software, 39(8), 1 30. https://doi.org/10.18637/jss.v039.i08
12. Cole, J.D. (2023). Understanding school counselors' perceptions of esports and igaming as a career choice [Unpublished doctoral dissertation]. https://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/j9602803m
13. Costello, A.B., & Osborne, J.W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(7), 1-9. https://doi.org/10.7275/jyj1-4868
14. Creswell, J. (2015). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (5th ed.). Pearson.
15. Crites Jr, S.L., Fabrigar, L.R., & Petty, R.E. (1994). Measuring the affective and cognitive properties of attitudes: Conceptual and methodological issues. Personality and Social Psychology Bulletin, 20(6), 619-634. https://doi.org/10.1177/0146167294206001
16. Çokluk, Ö., Şekercioğlu, G. & Büyüköztürk, Ş. (2014). Multivariate statistics for the social sciences: Applications of SPSS and LISREL (4th ed.). Pegem Akademi.
17. Delello, J.A., McWhorter, R.R., & Camp, K.M. (2021). Using emerging technologies to develop professional competencies in higher education. Education and Information Technologies, 26(2), 1825-1846. https://doi.org/10.1007/s10639-020-10337-y
18. Delello, J.A., McWhorter, R.R., Roberts, P., Dockery, H.S., De Giuseppe, T., & Corona, F. (2021). The rise of esports: Insights into the perceived benefits and risks for college students. International Journal of esports Research (IJER), 1(1), 1-19. http://doi.org/10.4018/IJER.20210101.oa5
19. DeVellis, R.F. (2017). Scale development: Theory and applications (4th ed.). Sage Publications.
20. Eagly, A. & Chaiken, S. (1993). The psychology of attitudes. Wadsworth.
21. Fabrigar, L.R., Wegener, D.T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299. https://doi.org/10.1037/1082-989X.4.3.272
22. Fornell, C., & Larcker, D.F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39 50. https://doi.org/10.1177/002224378101800104
23. Fraenkel, J.R., Wallen, N.E., & Hyun, H.H. (2018). How to design and evaluate research in education (10th ed.). McGraw-Hill Education.
24. Funk, D.C., Pizzo, A.D., & Baker, B.J. (2018). Esport management: Embracing esports education and research opportunities. Sport Management Review, 21(1), 7 13. https://doi.org/10.1016/j.smr.2017.07.008
25. Granic, I., Lobel, A., & Engels, R.C.M.E. (2014). The benefits of playing video games. American Psychologist, 69(1), 66-78. https://doi.org/10.1037/a0034857
26. Gül, M., Gül, O., & Uzun, R. (2019). Participation motivation scale for e-sports: The study of validity and reliability (PMSES). Turkish Journal of Sport and Exercise, 21(2), 281 294. https://doi.org/10.15314/tsed.563111
27. Gwet, K.L. (2019). irrCAC: Computing Chance-Corrected Agreement Coefficients (CAC) (Version 1.0) [Computer software]. https://CRAN.R-project.org/package=irrCAC
28. Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R.E. (2019). Multivariate data analysis (9th ed.). Prentice-Hall.
29. Hamari, J., & Sjöblom, M. (2017). What is esports and why do people watch it? Internet Research, 27(2), 211-232. https://doi.org/10.1108/IntR-04-2016-0085
30. Hayton, J.C., Allen, D.G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7(2), 191 205. https://doi.org/10.1177/10944281042636
31. Hilvoorde, I.V., & Pot, N. (2016). Embodiment and fundamental motor skills in esports. Sport, Ethics and Philosophy, 10(1), 14-27. https://doi.org/10.1080/17511321.2016.1159246
32. Hoffelner, C., Nägele, C., & Düggeli, A. (2025). Play for the future: A cross-sectional study on the role of video game skills in student career planning. Simulation & Gaming, 56(4), 350-373. https://doi.org/10.1177/10468781251328894
33. Horn, J.L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447
34. Hosky, A.M.W., & Wibowo, T.O. (2023). Group communication process between mobile legend game players in pendekar esports community. Sibatik Journal: Jurnal Ilmiah Bidang Sosial, Ekonomi, Budaya, Teknologi, Dan Pendidikan, 2(9), 2791-2802. https://doi.org/10.54443/sibatik.v2i9.1300
35. Huettermann, M., Trail, G.T., Pizzo, A.D., & Stallone, V. (2023). Esports sponsorship: An empirical examination of esports consumers’ perceptions of non-endemic sponsors. Journal of Global Sport Management, 8(2), 524-549. https://doi.org/10.1080/24704067.2020.1846906
36. Jodoin, M.G., & Gierl, M.J. (2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329-349. https://doi.org/10.1207/S15324818AME1404_2
37. Kaye, L.K., & Bryce, J. (2012). Putting the “fun factor” into gaming: The influence of social contexts on experiences of playing video games. International Journal of Internet Science, 7(1), 23-37. http://www.ijis.net/ijis7_1/ijis7_1_kaye_and_bryce.pdf
38. Ketelhut, S., Martin-Niedecken, A.L., Zimmermann, P., & Nigg, C.R. (2021). Physical activity and health promotion in esports and gaming–discussing unique opportunities for an unprecedented cultural phenomenon. Frontiers in Sports and Active Living, 3, Article 693700. https://doi.org/10.3389/fspor.2021.693700
39. King, D.L., Delfabbro, P.H., Billieux, J., & Potenza, M.N. (2018). Problematic online gaming and gaming disorder: A systematic review of clinical research. World Journal of Psychiatry, 8(1), 10-21. https://doi.org/10.5498/wjp.v8.i1.10
40. Kline, R.B. (2016). Principles and practice of structural equation modeling (4th ed.). Guilford.
41. Kowert, R., Domahidi, E., & Quandt, T. (2014). The relationship between online video game involvement and gaming related friendships among emotionally sensitive individuals. Cyberpsychology, Behavior, and Social Networking, 17(7), 447 453. https://doi.org/10.1089/cyber.2013.0656
42. Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310
43. Lawshe, C.H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563-575. https://parsmodir.com/wp-content/uploads/2015/03/lawshe.pdf
44. Leung, J., Kier, C., Fung, T., Fung, L., & Sproule, R. (2021). Development and validation of a video game attitude scale. Frontiers in Psychology, 12, Article 653833. https://doi.org/10.3389/fpsyg.2021.653833
45. Leung, K.M., Wong, M.Y.C., Ou, K.L., Chung, P.K., & Lau, K.L. (2021). Assessing esports participation intention: The development and psychometric properties of the theory of planned behavior-based esports intention questionnaire (TPB-Esport-Q). International Journal of Environmental Research and Public Health, 18(23), Article 12653. https://doi.org/10.3390/ijerph182312653
46. Lobel, A., Engels, R.C.M.E., Stone, L.L., Burk, W.J., & Granic, I. (2017). Video gaming and children’s psychosocial well-being: A longitudinal study. Journal of Youth and Adolescence, 46(4), 884-897. https://doi.org/10.1007/s10964-017-0646-z
47. Maio, G., & Haddock, G. (2014). The psychology of attitudes and attitude change. (2nd ed.) Sage Publications. https://doi.org/10.4135/9781446214299
48. McDonald, R.P. (2013). Test theory: A unified treatment. Psychology Press.
49. Meyers, L.S., Gamst, G., and Guarino, A.J. (2016). Applied multivariate research: Design and interpretation. Sage Publications.
50. Milfont, T.L., & Fischer, R. (2010). Testing measurement invariance across groups: Applications in cross cultural research. International Journal of Psychological Research, 3(1), 111 121. https://doi.org/10.21500/20112084.857
51. Nájera Catalán, H.E. (2019). Reliability, population classification and weighting in multidimensional poverty measurement: a monte carlo study. Social Indicators Research, 142(3), 887 910. https://doi.org/10.1007/s11205-018-1950-z
52. Petty, R.E., Briñol, P., Fabrigar, L.R., & Wegener, D.T. (2019). Attitude structure and change. In R.F. Baumeister & E.J. Finkel (Eds.), Advanced social psychology (2nd ed., pp. 117-156). Oxford University Press.
53. Pituch, K.A., & Stevens, J.P. (2016). Applied multivariate statistics for the social sciences: Analyses with SAS and IBM's SPSS (6th ed.). Routledge.
54. Price, L.R. (2017). Psychometric methods: Theory into practice. Guilford.
55. Putnick, D.L., & Bornstein, M.H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71 90. https://doi.org/10.1016/j.dr.2016.06.004
56. Qian, T.Y., Wang, J.J., Zhang, J.J., & Lu, L.Z. (2019). It is in the game: Dimensions of esports online spectator motivation and development of a scale. European Sport Management Quarterly, 20(4), 458-479. https://doi.org/10.1080/16184742.2019.1630464
57. Qian, T.Y., Zhang, J.J., Wang, J.J., & Hulland, J. (2019). Beyond the game: Dimensions of esports online spectator demand. Communication & Sport, 8(6), 825 851. https://doi.org/10.1177/2167479519839436
58. Revelle, W. (2023). Psych: Procedures for psychological, psychometric, and personality research (Version 2.3.9) [Computer software]. https://cran.r-project.org/package=psych
59. Robitzsch, A. (2023). sirt: Supplementary Item Response Theory models (Version 3.13-228) [Computer software]. https://CRAN.R-project.org/package=sirt
60. Rochat, S., & Armengol, J. (2020). Career counseling interventions for video game players. Journal of Career Development, 47(2), 207-219. https://doi.org/10.1177/0894845318793537
61. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. https://doi.org/10.18637/jss.v048.i02
62. Rothwell, G., & Shaffer, M. (2019). Esports in K–12 and post-secondary schools. Journal of Applied Sport Management, 11(2), 16-25. https://doi.org/10.18666/JASM-2019-V11-I2-9500
63. Salo, M. (2021). Career transitions of esports athletes: A proposal for a research framework. In I. Management Association (Ed.), Research anthology on business strategies, health factors, and ethical implications in sports and eSports (pp. 478-490). IGI Global Scientific Publishing. https://doi.org/10.4018/978-1-7998-7707-3.ch027
64. Savaş, B.Ç., & Turan, M. (2023). E-Sport attitude scale: A study of validity and reliability. International Journal of Education Technology and Scientific Researches, 8(23), 1524 1561. http://dx.doi.org/10.35826/ijetsar.629
65. Schober, P., Boer, C., & Schwarte, L. A. (2018). Correlation coefficients: Appropriate use and interpretation. Anesthesia & Analgesia, 126(5), 1763 1768. https://doi.org/10.1213/ANE.0000000000002864
66. Scholz, T.M. (2019). Esports is business: Management in the world of competitive gaming. Springer. https://doi.org/10.1007/978-3-030-11199-1
67. Schuman, H., & Johnson, M.P. (1976). Attitudes and behavior. Annual review of sociology, 2, 161-207. https://www.jstor.org/stable/2946091
68. Seo, Y. (2016). Professionalized consumption and identity transformations in the field of esports. Journal of Business Research, 69(1), 264-272. https://doi.org/10.1016/j.jbusres.2015.07.039
69. Shan, D., Xu, J., Liu, T., Zhang, Y., Dai, Z., Zheng, Y., Liu, C., Wei, Y., & Dai, Z. (2023). Subjective attitudes moderate the social connectedness in esports gaming during COVID-19 pandemic: A cross-sectional study. Frontiers in Public Health, 10, Article 1020114. https://doi.org/10.3389/fpubh.2022.1020114
70. Shi, M., & Ren, R. (2023). Do Chinese viewers watch e sports games for a different reason? Motivations, attitude, and team identification in predicting e sports online spectatorship. Frontiers in Psychology, 14, Article 1234305. https://doi.org/10.3389/fpsyg.2023.1234305
71. Soares, A.K.S., Goedert, M.C.F., & Vargas, A.F. (2022). Mental health and social connectedness during the COVID-19 pandemic: An analysis of sports and E-sports players. Frontiers in psychology, 13, Article 802653. https://doi.org/10.3389/fpsyg.2022.802653
72. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361 370. https://doi.org/10.1111/j.1745-3984.1990.tb00754.x
73. Tabachnick, B.G., & Fidell, L.S. (2013). Using multivariate statistics (6th ed.). Pearson.
74. Tekkurşun-Demir, G.T., & Bozkurt, T.M. (2019). Dijital Oyun Oynama Tutumu Ölçeği (DOOTÖ): Geçerlik ve güvenirlik çalışması [The Digital Gaming Attitude Scale (DGAS): Reliability and validity study]. Sportif Bakış: Spor ve Eğitim Bilimleri Dergisi, 6(1), 1 18. https://doi.org/10.33468/sbsebd.79
75. Tekkurşun-Demir, G.T., & Hazar, Z. (2018). Dijital Oyun Oynama Motivasyonu Ölçeği: Geçerlik ve güvenirlik çalışması [Digital Game Playing Motivation Scale: Validity and reliability study]. Niğde Üniversitesi Beden Eğitimi ve Spor Bilimleri Dergisi, 12(2), 128 139. https://dergipark.org.tr/tr/download/article-file/1028703
76. Turan, M., Akman, O., & Savaş, A. C. (2023). An investigation of university students’ e-sports participation motivations and socialization. International Journal of Sport Culture and Science, 11(1), 12-25. https://doi.org/10.14486/IntJSCS.2023.100
77. Vandenberg, R.J., & Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. https://doi.org/10.1177/109442810031002
78. Wattanapisit, A., Tuangratananon, T., Saman, W., & Saengow, U. (2020). Gaming behavior, internet gaming disorder, and associated factors among medical students. BMC Research Notes, 13, 409. https://doi.org/10.1186/s13104-020-05250-9
79. Weinstein, A., & Lejoyeux, M. (2015). New developments on the neurobiological and pharmaco-genetic mechanisms underlying internet and videogame addiction. The American Journal on Addictions, 24(2), 117-125. https://doi.org/10.1111/ajad.12110
80. World Health Organization. (2019). Gaming disorder (6C51). In International classification of diseases for mortality and morbidity statistics (11th revision). https://icd.who.int/browse11/l m/en#/http://id.who.int/icd/entity/1448597234
81. Worthington, R.L., & Whittaker, T.A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806 838. https://doi.org/10.1177/0011000006288127
82. Yang-Wallentin, F., Jöreskog, K.G., & Luo, H. (2010). Confirmatory factor analysis of ordinal variables with misspecified models. Structural Equation Modeling: A Multidisciplinary Journal, 17(3), 392-423. http://dx.doi.org/10.1080/10705511.2010.489003
83. Ye, J.N., Ye, J.H., Wang, C.M., & Hong, J.C. (2021). Development of 5 Cs educational value scale for eSport games. International Journal of Technology in Education and Science (IJTES), 5(3), 362-374. https://doi.org/10.46328/ijtes.215
84. Yin, K., Zi,Y., Zhuang, W., Gao, Y., Tong, Y., Song, L., & Liu,Y. (2020). Linking e-sports to health risks and benefits: Current knowledge and future research needs. Journal of Sport and Health Science, 9(6), 485-488. https://doi.org/10.1016/j.jshs.2020.04.006
85. Zumbo, B.D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4(2), 223 233. https://doi.org/10.1080/15434300701375832