Analyzing TALIS Indicators and PISA Results with Data Envelopment: Comparison of EMS, DEAP and R Software

Abstract

The Teaching and Learning International Survey (TALIS) and the Programme for International Student Assessment (PISA) are large-scale measurements about teaching and learning. There is a link between TALIS indicators and PISA results. We investigated which countries are effective according to TALIS indicators as inputs and PISA 2015 mathematics, scientific, and reading literacy scores as outputs in this research. Common 24 countries' data from TALIS 2013 and PISA 2015 were analyzed. Data envelopment analysis was used in this quantitative research. Belgium, Denmark, Finland, Italy, Korea, Mexico, Netherlands, Norway, and Portugal were found to be effective countries in EMS 1.3, DEAP-XP 2.1, and R-4.0.3 software according to the input-oriented CCR model. Belgium, Canada, Denmark, Estonia, Finland, Italy, Japan, Korea, Mexico, Netherlands, Norway, and Portugal were found to be effective countries in EMS 1.3, DEAP-XP 2.1, and R-4.0.3 software according to the input-oriented BCC model. The results obtained from the BCC and CCR model differ partially. Italy and Norway should be taken as reference the mostly by ineffective countries for getting better PISA score according to both models analyzing with EMS 1.3, DEAP-XP 2.1, and R-4.0.3.

Keywords

• TALIS
• PISA
• Efficiency
• Data envelopment analysis

References

1. Ainley, J., & Carstens, R. (2018). Teaching and learning international survey (TALIS) 2018 conceptual framework. OECD Education Working Papers, No. 187, OECD Publishing, Paris. https://doi.org/10.1787/799337c2-en
2. Iqbal, A., Fakhra, A. Z. I. Z., Farooqi, T. K., & Shabbir, A. L. I. (2016). Relationship between teachers’ job satisfaction and students’ academic performance. Eurasian Journal of Educational Research, 16(65), 335-344. Retrieved from https://dergipark.org.tr/en/pub/ejer/issue/42412/510720
3. Bandura, A. (1997). Self-Efficacy: The Exercise of Control. Freeman, New York, NY.
4. Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychological Review, 84(2), 191-215. https://doi.org/10.1037/0033-295X.84.2.191
5. Banker, R. D., Charnes, A., & Cooper, W. W. (1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science, 30(9), 1078-1092. https://doi.org/10.1287/mnsc.30.9.1078
6. Bezanilla, M. J., Fernández-Nogueira, D., Poblete, M., & Galindo-Domínguez, H. (2019). Methodologies for teaching-learning critical thinking in higher education: The teacher’s view. Thinking Skills and Creativity, 33, 1-10, 100584. https://doi.org/10.1016/j.tsc.2019.100584
7. Boonen, T., Van Damme, J., & Onghena, P. (2014). Teacher effects on student achievement in first grade: which aspects matter most?. School Effectiveness and School Improvement, 25(1), 126-152. https://doi.org/10.1080/09243453.2013.778297
8. Buddin, R., & Zamarro, G. (2009). Teacher qualifications and student achievement in urban elementary schools. Journal of Urban Economics, 66(2), 103-115. https://doi.org/10.1016/j.jue.2009.05.001

9. Caprara, G. V., Barbaranelli, C., Steca, P., & Malone, P. S. (2006). Teachers' self-efficacy beliefs as determinants of job satisfaction and students' academic achievement: A study at the school level. Journal of School Psychology, 44(6), 473-490. https://doi.org/10.1016/j.jsp.2006.09.001
10. Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2(6), 429-444. https://doi.org/10.1016/0377-2217(78)90138-8
11. Cooper, W. W., Seiford, L. M. & Zhu, J. (2004). Data Envelopment Analysis: History, Models and İnterpretations. In Cooper, W. W., Seiford, L. M. ve Zhu, J. Handbook on Data Envelopment Analysis (p.1-39). Kluwer Academic Publisher, New York.
12. Evans, L. (1997). Addressing problems of conceptualization and construct validity in researching teachers’ job satisfaction. Educational Research, 39(3), 319-331. https://doi.org/10.1080/0013188970390307
13. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). Mc Graw Hill Higher Education. New York, NY.
14. Franco, A. R., Costa, P. S., Butler, H. A., & Almeida, L. S. (2017). Assessment of undergraduates’ real-world outcomes of critical thinking in everyday situations. Psychological Reports, 120(4), 707-720. https://doi.org/10.1177/0033294117701906,
15. Gurria, A. (2016). PISA 2015 results in focus. PISA in Focus, (67), 1. http://dx.doi.org/10.1787/22260919
16. Ingersoll, R. M., & Collins, G. J. (2017). Accountability and control in American schools. Journal of Curriculum Studies, 49(1), 75-95. https://doi.org/10.1080/00220272.2016.1205142
17. Judge, T. A., Thoresen, C. J., Bono, J. E., & Patton, G. K. (2001). The job satisfaction–job performance relationship: A qualitative and quantitative review. Psychological Bulletin, 127(3), 376-407. https://doi.org/10.1037/0033-2909.127.3.376
18. Khezrimotlagh, D., Cook, W. D., & Zhu, J. (2021). Number of performance measures versus number of decision making units in DEA. Annals of Operations Research, 303(1), 529-562. https://doi.org/10.1007/s10479-019-03411-y
19. Klassen, R. M., Bong, M., Usher, E. L., Chong, W. H., Huan, V. S., Wong, I. Y., & Georgiou, T. (2009). Exploring the validity of a teachers’ self-efficacy scale in five countries. Contemporary Educational Psychology, 34(1), 67-76. https://doi.org/10.1016/j.cedpsych.2008.08.001
20. Locke, E. A. (1970). Job satisfaction and job performance: A theoretical analysis. Organizational Behavior and Human Performance, 5(5), 484-500. https://doi.org/10.1016/0030-5073(70)90036-X
21. Locke, E. A. (1969). What is job satisfaction?. Organizational Behavior and Human Performance, 4(4), 309-336. https://doi.org/10.1016/0030-5073(69)90013-0
22. Luthans, F. (2000). Organizational Behavior (12th Edition). McGraw-Hill/Irwin, Boston.
23. OECD (2016). PISA 2015 Results (Volume I): Excellence and Equity in Education, PISA, OECD Publishing, Paris. https://doi.org/10.1787/9789264266490-en
24. Osagie, C. I., & Akinlosotu, N. T. (2017). Causal Relationship between Teachers’ Job Performance and Students’ Academic Achievement in Secondary Schools in Nigeria. African Research Review, 11(4), 45-54. https://doi.org/10.4314/afrrev.v11i4.5
25. Perera, H. N., & John, J. E. (2020). Teachers’ self-efficacy beliefs for teaching math: Relations with teacher and student outcomes. Contemporary Educational Psychology, 61, 101842. https://doi.org/10.1016/j.cedpsych.2020.101842
26. Pushpakumari, M. D. (2008, January). The impact of job satisfaction on job performance: An empirical analysis. In City Forum, 9 (1), 89-105.
27. R Core Team. (2021). R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. http://www.R-project.org/
28. Rutkowski, D., Rutkowski, L., Bélanger, J., Knoll, S., Weatherby, K., & Prusinski, E. (2013). Teaching and Learning International Survey TALIS 2013: Conceptual Framework. OECD Publishing. https://eric.ed.gov/?id=ED566883
29. Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17(7), 783-805. https://doi.org/10.1016/S0742-051X(01)00036-1
30. Tschannen-Moran, M., Hoy, A. W., & Hoy, W. K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research, 68(2), 202-248. https://doi.org/10.3102/00346543068002202
31. Yerdelen, S., Sungur, S. and Klassen, R.M. (2016) Relationship between Turkish Elementary Science Teachers’ Occupational Well-Being and Some Contextual and Demographic Characteristics: A Multivariate Analysis. Education and Science, 41, 147-161. https://doi.org/10.15390/EB.2016.4257
32. Zee, M., & Koomen, H. M. (2016). Teacher self-efficacy and its effects on classroom processes, student academic adjustment, and teacher well-being: A synthesis of 40 years of research. Review of Educational Research, 86(4), 981-1015. https://doi.org/10.3102/0034654315626801