Adaptation of the Self-efficacy Beliefs in STEM Education Scale and Testing Measurement Invariance across Groups

Abstract

Academic performance on science, technology, engineering, and mathematics (STEM) education is important for the economic development of countries. From the perspectives of social cognitive theory, one of the predictors of academic performance is self-efficacy. In order to measure middle school students’ self-efficacy beliefs in STEM education, STEM Competency Beliefs scale was developed in English originally by Chen, Cannady, Schunn, and Dorph (2017). In this study, it is aimed to adapt the English scale into Turkish and to provide evidence regarding reliability and validity. Throughout the adaptation process, forward and backward translation was completed. In the pilot study (n = 77), the reliability of the data and the clarity of the statements in the Turkish version of the scale was examined. In the main study, the Turkish version was administered to 330 middle school students to investigate the psychometric properties of the scale. The results pointed out that the scores obtained by the Turkish version of the scale had good internal consistency. Regarding the dimensionality of the scale, in contrast to the original version, the adapted scale showed a two-dimensional structure. Measurement invariance findings for gender groups supported configural and metric invariance, whereas scalar invariance was partially achieved. Measurement invariance findings for career choice groups supported configural, metric, and scalar invariance. Scale scores of students were estimated using multidimensional Item Response Theory. The findings suggested that the scale can be utilized for STEM-related research to assess the competency beliefs of students.

Keywords

• Self-Efficacy Beliefs,scale adaptation,confirmatory factor analysis,measurement invariance,multidimensional item response theory

References

1. Akgündüz, D., Aydeniz, M., Çakmakçı, G., Çavaş, B., Çorlu, M. S., Öner, T., & Özdemir, S. (2015). STEM eğitimi Türkiye raporu. Istanbul: Scala Publication.
2. Atkinson, R., & Mayo, M. (2010, December). Refueling the U.S. innovation economy fresh approaches to science, technology, engineering and mathematics (STEM) education. The Information Technology & Innovation Foundation, Forthcoming. Retrieved from https://ssrn.com/abstract=1722822
3. Bandura, A. (1999). Social cognitive theory: An agentic perspective. Asian Journal of Social Psychology, 2(1), 21-41.
4. Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman and Company.
5. Baran, E., Canbazoglu-Bilici, S., Mesutoglu, C. & Ocak, C. (2016). Moving STEM beyond schools: Students’ perceptions about an out-of-school STEM education program. International Journal of Education in Mathematics, Science and Technology, 4(1), 9-19. doi: 10.18404/ijemst.71338
6. Bentler, P. M., & Chou, C. P. (1987). Practical issues in structural equation modeling. Sociological Methods and Research, 16(1), 78-117. doi: 10.1177/0049124187016001004
7. Bouffard-Bouchard, T., Parent, S., & Larivee, S. (1991). Influence of self-efficacy on self-regulation and performance among junior and senior high-school-age students. International Journal of Behavioral Development, 14(2), 153-164. doi: 10.1177/016502549101400203
8. Breiner, J. M., Harkness, S. S., Johnson, C. C., & Koehler, C. M. (2012). What is STEM? A discussion about conceptions of STEM in education and partnerships. School Science and Mathematics, 112(1), 3-11. doi: 10.1111/j.1949-8594.2011.00109.x

9. Cai, L., Thissen, D., & du Toit, S. H. C. (2017). IRTPRO 4.2 for Windows [Computer software]. Skokie, IL: Scientific Software International.
10. Chen, F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modelling, 14(3), 464-504. doi: 10.1080/10705510701301834
11. Chen, Y. F., Cannady, M. A., Schunn, C., & Dorph, R. (2017) Measures technical brief: Competency beliefs in STEM. Retrieved from: http://activationlab.org/wp-content/uploads/2018/03/CompetencyBeliefs_STEM-Report_20170403.pdf
12. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255. doi: 10.1207/S15328007SEM0902_5
13. Chittum, J. R., Jones, B. D., Akalin, S., & Schram, Á. B. (2017). The effects of an afterschool STEM program on students’ motivation and engagement. International Journal of STEM education, 4(11), 1-16. doi: 10.1186/s40594-017-0065-4
14. Chung, J., Cannady, M. A., Schunn, C., Dorph, R., & Vincent-Ruz, P. (2016). Measures technical brief: Competency beliefs in science. Retrieved from: http://activationlab.org/wp-content/uploads/2018/03/Competency-Beliefs-Report-3.2-20160331.pdf
15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New York, NY: Routledge.
16. Colakoglu, M. H. (2016). STEM applications in Turkish science high schools. Journal of Education in Science, Environment and Health (JESEH), 2(2), 176-187.
17. Colakoğlu, M. H., & Gökben, A. G. (2017). Türkiye’de eğitim fakültelerinde fetemm (stem) çalışmaları. İnformal Ortamlarda Araştırmalar Dergisi, 2(2), 46-69.
18. Corlu, M. S., Capraro, R. M., & Capraro, M. M. (2014). Introducing STEM education: Implications for educating our teachers in the age of innovation. Eğitim ve Bilim, 39(171), 74-85.
19. Dawes, M. E., Horan, J. J., & Hackett, G. (2000). Experimental evaluation of self-efficacy treatment on technical/scientific career outcomes. British Journal of Guidance & Counselling, 28(1), 87-99. doi: 10.1080/030698800109637
20. Durik, A. M., Vida, M., & Eccles, J. S. (2006). Task values and ability beliefs as predictors of high school literacy choices: A developmental analysis. Journal of Educational Psychology, 98(2), 382-393. doi: 10.1037/0022-0663.98.2.382
21. English, L. D. (2016). STEM education K-12: Perspectives on integration. International Journal of STEM Education, 3(3), 1-8. doi: 10.1186/s40594-016-0036-1
22. Ercan, S., Altan, E. B., Taştan, B., & Dağ, İ. (2016). Integrating GIS into science classes to handle STEM education. Journal of Turkish Science Education, 13, 30-43. doi: 10.12973/tused.10169a
23. Field, A. (2013). Discovering statistics using IBM SPSS statistics. London: Sage.
24. Gainor, K. A. (2006). Twenty-five years of self-efficacy in career assessment and practice. Journal of Career Assessment, 14(1), 161-178. doi: 10.1177/1069072705282435
25. George, D., & Mallery, P. (2001). SPSS for Windows Step by Step: A Simple Guide and Reference. Boston: Allyn & Bacon.
26. Green, A., & Sanderson, D. (2018). The roots of STEM achievement: An analysis of persistence and attainment in STEM majors. The American Economist, 63(1), 79-93. doi: 10.1177/0569434517721770
27. Hacıoğlu, Y., Yamak, H., & Kavak, N. (2016). Mühendislik tasarim temelli fen egitimi ile ilgili ögretmen görüsleri. Bartin Üniversitesi Egitim Fakültesi Dergisi, 5(3), 807-830. doi: 10.14686/buefad.v5i3.5000195411
28. Hackett, G., & Betz, N. (1982, March). Mathematics self-efficacy expectations, math performance, and the consideration of math-related majors. Paper presented at the annual meeting of the American Educational Research Association. New York.
29. Hambleton, R. K., & Jones, R. W. (1993). An NCME instructional module on: Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38-47. doi: 10.1111/j.1745-3992.1993.tb00543.x
30. Hambleton, R. K., Merenda, P. F., & Spielberger, C. D. (2012). Adapting educational and psychological tests for cross-cultural assessment. New York, NY: Psychology Press.
31. Han, S., Capraro, R. M., & Capraro, M. M. (2016). How science, technology, engineering, and mathematics project-based learning affects high-need students in the US. Learning and Individual Differences, 51, 157-166. doi: 10.1016/j.lindif.2016.08.045
32. Han, S., Yalvac, B., Capraro, R. M., & Capraro, M. M. (2015). In-service teachers’ implementation and understanding of STEM project-based learning. Eurasia Journal of Mathematics, Science & Technology Education, 11(1), 63-76. doi: 10.12973/eurasia.2015.1306a
33. Hidi, S., & Ainley, M. (2008). Interest and self-regulation: Relationships between two variables that influence learning. In D. H. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 77-109). Lawrence Erlbaum Associates Publishers.
34. International Test Commission. (2017). The ITC guidelines for translating and adapting tests (2nd ed.). Retrieved from https://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf
35. Jenson, R. J., Petri, A. N., Day, A. D., Truman, K. Z., & Duffy, K. (2011). Perceptions of self-efficacy among STEM students with disabilities. Journal of Postsecondary Education and Disability, 24(4), 269-283.
36. Jinks, J., & Lorsbach, A. (2003). Introduction: Motivation and self-efficacy belief. Reading and Writing Quarterly, 19(2), 113-118. doi: 10.1080/10573560308218
37. John, M., Bettye, S., Ezra, T., & Robert, W. (2016). A formative evaluation of a southeast high school integrative science, technology, engineering, and mathematics (STEM) academy. Technology in Society, 45, 34-39. doi: 10.1016/j.techsoc.2016.02.001
38. Johnson, C. C., Peters-Burton, E. E., & Moore, T. J. (2016). STEM road map: A framework for integrated STEM education. New York, NY: Routledge, Taylor & Francis Group.
39. Kanny, M. A., Sax, L. J., & Riggers-Piehl, T. A. (2014). Investigating forty years of STEM research: How explanations for the gender gap have evolved over time. Journal of Women and Minorities in Science and Engineering, 20(2), 127-148. doi: 10.1615/JWomenMinorScienEng.2014007246
40. Lent, R. W., Brown, S. D., & Larkin, K. C. (1986). Self-efficacy in the prediction of academic performance and perceived career options. Journal of Counseling Psychology, 33(3), 265-269. doi: 10.1037/0022-0167.33.3.265
41. Milfont, T. L., & Fischer, R. (2010). Testing measurement invariance across groups: Applications in crosscultural research. International Journal of Psychological Research, 3(1), 111-121.
42. Milner, D. I., Horan, J. J., & Tracey, T. J. (2014). Development and evaluation of STEM interest and self-efficacy tests. Journal of Career Assessment, 22(4), 642-653. doi: 10.1177/1069072713515427
43. Ministry of National Education. (2009). Ortaöğretim okulları öğrenci kulüp faliyetlerine yönelik eğitim materyali ve donanım ihtiyacının değerlendirilmesi. Ankara: EARGED.
44. Ministry of National Education. (2016). STEM Education Report. Ankara: SESAM.
45. Ministry of National Education. (2018a). Fen bilimleri dersi öğretim programı. Ankara: MEB.
46. Ministry of National Education. (2018b). Matematik dersi öğretim programı. Ankara: MEB.
47. Monterastelli, T., Bayles, T., & Ross, J. (2008, June). High school outreach program: Attracting young ladies with “engineering in health care.” Paper presented at the Annual Conference, & Exposition, Pittsburgh, Pennsylvania. Retrieved from https://peer.asee.org/3621
48. Muthén, L. K., & Muthén, B. O. (2015). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén and Muthén. National Research Council. (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research. Washington, DC: National Academic Press.
49. Nelson, B. C., & Ketelhut, D. J. (2008). Exploring embedded guidance and self-efficacy in educational multi-user virtual environments. Computer-Supported Collaborative Learning, 3(4), 413-427. doi: 10.1007/s11412-008-9049-1
50. Pajares, F., & Miller, M. D. (1997). Mathematics self-efficacy and mathematical problem solving: Implications of using different forms of assessment. The Journal of Experimental Education, 65(3), 213-228. doi: 10.1080/00220973.1997.9943455
51. Reckase, M. D. (2009). Multidimensional item response theory. New York, NY: Springer. doi: 10.1007/978-0-387-89976-3
52. Sireci, S. G., & Sukin, T. (2013). Test validity. In K. F. Geisinger, B. A. Bracken, J. F. Carlson, J.-I. C. Hansen, N. R. Kuncel, S. P. Reise, & M. C. Rodriguez (Eds.), APA handbooks in psychology. APA handbook of testing and assessment in psychology, Vol. 1. Test theory and testing and assessment in industrial and organizational psychology (pp. 61-84). Washington, DC: American Psychological Association. doi: 10.1037/14047-004
53. Smith, S. M. (2019). A comparison of computer-based and robotic programming instruction: Impact of Scratch versus Cozmo on middle school students’ computational thinking, spatial skills, competency beliefs, and engagement (Doctoral dissertation, Kent State University).
54. Tellhed, U., Backström, M., & Björklund, F. (2017). Will I fit in and do well? The importance of social belongingness and self-efficacy for explaining gender differences in interest in STEM and HEED majors. Sex Roles, 77(1-2), 86-96. doi: 10.1007/s11199-016-0694-y
55. Tsupros, N., R. Kohler, and J. Hallinen (2009). STEM education: A project to identify the missing components. Intermediate Unit 1 and Carnegie Mellon, Pennsylvania.
56. TÜSİAD (2019). 2023‘e doğru Türkiye‘de STEM gereksinimi. https://tusiad.org/tr/yayinlar/raporlar/item/9735-2023-e-dog-ru-tu-rkiye-de-stem-gereksinimi adresinden edinilmiştir.
57. Ullman, J. B. (2001). Structural equation modeling. In B. Tabachnick & L. S. Fidell (Eds.), Using multivariate statistics (4th ed.) (pp. 653-771). Boston, MA: Allyn & Bacon.
58. Urdan, T. C. (2010). Statistics in plain English (3rd ed.). New York, NY: Taylor & Francis Group.
59. Van de Vijver, F., & Hambleton, R. K. (1996). Translating tests: Some practical guidelines. European Psychologist, 1(2), 89-99.
60. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. doi: 10.1177/109442810031002
61. Vincent‐Ruz, P., & Schunn, C. D. (2017). The increasingly important role of science competency beliefs for science learning in girls. Journal of Research in Science Teaching, 54(6), 790-822. doi: 10.1002/tea.21387
62. Yerdelen, S., Kahraman, N., & Taş, Y. (2016). Low socioeconomic status students’ STEM career interest in relation to gender, grade level, and STEM attitude [Special issue]. Journal of Turkish Science Education (TUSED), 13, 59-74.
63. Zeldin, A. L., Britner, S. L., & Pajares, F. (2008). A comparative study of the self‐efficacy beliefs of successful men and women in mathematics, science, and technology careers. Journal of Research in Science Teaching, 45(9), 1036-1058. doi: 10.1002/tea.20195
64. Zimmerman, B. J. (1995). Self-efficacy and educational development. In A. Bandura, (Ed.), Self-efficacy in changing societies (pp. 202-231). New York, NY: Cambridge University Press.
65. Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary Educational Psychology, 25(1), 82-91. doi: 10.1006/ceps.1999.1016