A Study of Reliability and Validity for Citizenship Knowledge and Skill Scale

Abstract

The study aims to develop a current scale that has a higher validity and reliability and reveals high school students’ perceptions of measuring their citizenship knowledge and skills. The study was conducted with two different groups. The first group is the group where data is collected to conduct Exploratory Factor Analysis (EFA) and it consists of 258 students. The second group is the group where the data is collected to carry out Confirmatory Factor Analysis (CFA) and it consists of 180 students. A total of 438 students participated in the study providing different students were in both groups. As a result of the analyzes, it was determined that the Citizenship Knowledge and Skill Scale, which includes a total of 24 items, consists of a 5-factor structure. These factors are termed as “Participation in Social Life”, “Right to Education”, “Individual Duties”, “Duties of the State” and “Common Rights”. The total variance explained by the scale is 61.79%. Additionally, there is a significant relationship between these 5 factors and there is no autocorrelation problem. Item-factor and item-test correlation coefficients were calculated for all items of the scale and it was determined that each item was consistent not only with the factor it contained but also with the whole test. Cronbach Alpha reliability of the general of Citizenship Knowledge and Skill Scale is 0.91 and Omega reliability is 0.92. It can be said that the reliability and validity of the scale are applicable and high.

Keywords

• Citizenship
• Social studies
• Citizenship education
• Reliability
• Validity

References

1. Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16 (Spring), 74-94.
2. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588 606. https://doi.org/10.1037/0033-2909.88.3.588
3. Brubaker, R. (2009). Fransa ve Almanya’da vatandaşlık ve ulus ruhu (V. Pekel, Çev.). Ankara: Dost Kitabevi.
4. Briggs, N.E., & MacCallum. R.C. (2003). Recovery of weak common factors by Maximum Likelihood and Ordinary Least Squares Estimation. Multivariate Behavioral Research. 38(1). 25-56.
5. Bryne, B. M. (2001), Structural equation modeling with AMOS Mahwah, NJ: Lawrence Erlbaum Associates.
6. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105.
7. Cattell, R. B. (1978). The scientific use of factor analysis in behavioral and life sciences. New York: Plenum.
8. Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2014). Sosyal bilimler için çok değişkenli istatistik: SPSS ve LISREL Ankara: Pegem Akademi.

9. Colton, D., & Covert, R. (2007). Designing and constructing ınstruments for social research and evaluation. San Francisco, CA: Jossey-Bass.
10. Comrey, A.L. & Lee, H. B. (1992). A first course in factor analysis. Hillsdale, NJ: Erlbaum.
11. Çayır, K., & Bağlı, M. T. (2011). ‘No‐one respects them anyway’: secondary school students’ perceptions of human rights education in Turkey. Intercultural Education, 22(1), 1-14.
12. Ding, L., Velicer, W. F. & Harlow, L. L. (1995). Effects of estimation methods, number indicators per factor, and improper solutions on structural equation modeling fit indices. Structural Equation Modeling, 2, 119 144. https://doi.org/10.1080/10705519509540000
13. Doğanay, A. (2008). What does democracy mean to 14-year-old turkish children? A Comparison with Results of the 1999 IEA Civic Education Study. Research Papers in Education, 25(1), 51-71.
14. Doğanay, A., & Sarı, M. (2009). Lise öğrencilerinin vatandaşlık algılarına etki eden faktörlerin analizi [Analysis of the factors affecting high school students' perception of citizenship]. In A. Şişman, İ. Acun, C. Balkır, C. Yücel, H. Busher, T. Lawson, C. Wilkins, …, H. Ermiş (Eds.), 1st International European Union, Democracy, Citizenship and Citizenship Education (p. 36-51). Uşak: Uşak University European Union Research Center.
15. Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A Practical solution to the pervasive problem of ınternal consistency estimation. British Journal of Psychology, 105, 399-412. https://doi.org/10.1111/bjop.12046
16. Durualp, E. & Doğan, İ. (2017). Vatandaşlık algısı ölçeği’nin faktör yapısının incelenmesi. International Journal of Social Science, Number: 62, p. 65 83. http://dx.doi.org/10.9761/JASSS7173
17. Ersoy, A. F. (2014). Active and democratic citizenship education and its challenges in social studies classroom. Eurasian Journal of Educational Research, 55, 1-20.
18. Eurofound (2014). Quality of the life trends in Turkey: 2003–2012. Luxemburg: Publications Office of the European Union.
19. Fabrigar, L.R., Wegener, D.T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299
20. Ford, J. K., McCallum, R. S. & Tait, M. (1986). The application of exploratory factor analysis in applied psychology: A critical review and analysis. Personnel Psychology, 39, 291-314.
21. Fornell, C. & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50.
22. Gomez, R., & Fisher, J. W. (2003). Domains of spiritual well-being and development and validation of the Spiritual Well-Being Questionnaire. Personality and Individual Differences, 35(8), 1975–1991. https://doi.org/10.1016/S0191-8869(03)00045-X
23. Gouveia, V. V., & Soares, A. K. S. (2015). Calculadoras de validade de construto (CVC). João Pessoa, PB: BNCS/ Universidade Federal da Paraíba, [Construct Validity Calculators (CVC)] Retrieved from http://akssoares.com/psicometria/calculadora-vme-e-cc
24. Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis with readings, (4 th ed.), Englewood Cliffs, NJ: Prentice-Hall.
25. Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (1998). Multivariate data analysis. Upper Saddle River, NJ: Prentice hall.
26. Hair, J.F., Black, W., Babin, B., & Anderson, R. (2010). Multivariate data analysis (7th ed.). Upper Saddle River, NJ: Prentice-Hall.
27. Harrington, D. (2009). Confirmatory factor analysis. Oxford university press.
28. Ho, R. (2006). Handbook of univariate and multivariate data analysis and interpretation with SPSS. NY: Chapman and Hall/CRC.
29. Hoyle, R. H., (1995). The structural equation modeling approach: Basic concepts and fundamental issues. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts issues and applications (pp. 1-15). Thousand Oaks, CA: Sage.
30. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179-185.
31. Hu, L. T., & Bentler, P. M. (1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. https://doi.org/10.1080/10705519909540118
32. İçen, M., Öztürk, C., & Yılmaz, A. (2017). Vatandaşlık duygusu ölçeği güvenirlik ve geçerlik çalışması. Uluslararası Alan Eğitimi Dergisi, 3(2), 26-36.
33. Johnson, L., & Morris, P. (2010). Towards a framework for critical citizenship education. The Curriculum Journal, 21(1), 77-96. https://doi.org/10.1080/09585170903560444
34. Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183-202.
35. Kandemir, M. A., Tezci, E., Shelley, M., & Demirli, C. (2019). Measurement of creative teaching in mathematics class. Creativity Research Journal, 31(3), 1 12. https://doi.org/10.1080/10400419.2019.1641677
36. Karaman-Kepenekçi, Y. (2005). A study of effectiveness of human rights education in Turkey. Journal of Peace Education, 2(1), 39-55.
37. Keyman, F. (2008). Kimlik, vatandaşlık ve demokratikleşme: Türkiye örneği. İstanbul: Osmanlı Bankası Arşiv ve Araştırma Merkezi Yayınları.
38. Kline, P. (1994). An easy guide to factor analysis. New York: Routledge.
39. Küçükkaragöz, H. (2009). Bilişsel gelişim ve dil gelişimi (Cognitive development and language development). In B. Yeşilyaprak (Ed.), Gelişim ve Öğrenme Psikolojisi (Development and Learning Psychology) (s.75-107). Ankara: Pegem Akademi Yayınları.
40. MacCallum, R. C., Browne, M. W., & Cai, L. (2007). Factor analysis models as approximations. In R. Cudeck & R. C. MacCallum (Eds.). Factor analysis at 100: Historical developments and future directions (pp.153-175). Mahwah. NJ: Lawrence Erlbaum Associates. Publishers.
41. MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84–99. https://doi.org/10.1037/1082-989x.4.1.84
42. Malhotra, N. K. (2011). Pesquisa de Marketing: uma orientação aplicada, (6th ed.) São Paulo: Bookman.
43. Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika. 57, 519-530.
44. Ministry of National Education of Turkey. (2018). Sosyal bilgiler dersi öğretim programı (İlkokul ve ortaokul 4, 5, 6 ve 7. sınıflar) [Social studies curriculum (primary and middle school 4, 5, 6, and 7 grades)]. Ankara: MEB Devlet Kitapları. Retrieved from http://mufredat.meb.gov.tr/Programlar.aspx
45. Morais, D. B., & Ogden, A. C. (2011). Initial development and validation of the global citizenship scale. Journal of studies in international education, 15(5), 445-466.
46. O'Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer's MAP test. Behavior Research Methods, Instrumentation, and Computers, 32, 396-402.
47. Peterson, R. A. (2000). A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis. Marketing letters, 11(3), 261-275.
48. Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173-184.
49. Sağlam, H. İ. (2000). Sosyal bilgiler dersinin demokratik tutum geliştirmedeki rolü. Milli Eğitim Dergisi, 146, 67-71.
50. Sağlam, H. İ. (2011). Öğretmen adaylarının etkili vatandaşlık yeterlik düzeyleri. Kastamonu Eğitim Dergisi, 19(1), 39-50.
51. Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323-38.
52. Stevens, J.P. (1984). Outliers and influential data points in regression analysis. Psychological Bulletin, 95, 334-344.
53. Şen, A. (2018). Militarisation of citizenship education curriculum in Turkey. Journal of Peace Education, 16(1), 1-26. https://doi.org/10.1080/17400201.2018.1481019
54. Şen, A. (2019). Vatandaşlık eğitiminde değişiklik ve süreklilikler: 2018 sosyal bilgiler öğretim programı nasıl bir vatandaşlık eğitimi öngörüyor? (Changes and continuities in citizenship education: what kind of citizenship education does the 2018 social studies programme of study envisage?) Eğitimde Nitel Araştırmalar Dergisi, 7(1), 1-28. https://doi.org/10.14689/issn.2148-2624.1.7c1s.1m
55. Tabachnick, B. G., & Fidel, L. S. (2001). Using Multivariate Statistics. Boston, MA: Allyn and Bacon.
56. Tatlıdil, H. (1992). Uygulamalı çok değişkenli istatistik (Applied multivariate statistics). Ankara: Akademi Publishing.
57. Tezci, E. (2016). Eğitimde ölçme ve değerlendirme (Measurement and evaluation in education). Ankara: Detay Publishing.
58. Thompson, B. (2004). Explaratory and Confirmatory Factor Analysis: Understanding Concepts and Applications. Washington: American Psychological Association.
59. Uğurlu, C. T. (2011). Citizeinship education in European Union countries and Turkey. Electronic Journal of Social Sciences, 10(37), 153-169.
60. Üstel, F. (2016). Makbul vatandaşın peşinde II. Meşrutiyet'ten bugüne vatandaşlık eğitimi (In pursuit of the acceptable citizen citizenship education since the II. Constitutional Monarchy) (7th ed.). İstanbul: İletişim Publishing.
61. Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41, 321-327.
62. Velicer, W. F., & Jackson, D. N. (1990). Component analysis versus common factor analysis: Some issues in selecting an appropriate procedure. Multivariate Behavioral Research, 25, 1-28.
63. Williams, B., Onsman, A., & Brown, T. (2020). Exploratory factor analysis: A five step guide for novices. Journal of Emergency Primary Health Care, 8(3), 1-13.
64. Yavuz, G., & Doğan, N. (2015). Boyut sayısı belirlemede Velicer’in map testi ve Horn’un paralel analizinin kullanılması. Hacettepe University Journal of Education, 30(3), 176-188.
65. Zinbarg, R. E., Yovel, I., Revelle, W., & McDonald, R. P. (2006). Estimating generalizability to a latent variable common to all of a scale's indicators: A comparison of estimators for ωh. Applied Psychological Measurement, 30(2), 121-144.
66. Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99, 432-442.