A Bibliometric Analysis of Articles on High-Stakes Testing Between 2000 and 2024

Abstract

This study aims to examine articles on high-stakes testing using a bibliometric analysis method. High-stakes testing represents a critical tool through which schools and educators are evaluated based on student performance, thereby shaping educational policies. Within the scope of this research, 1,947 articles published between 2000 and 2024 were retrieved from the Web of Science Core Collection (WoS) database using the "Topic" field with the keywords "education AND high stakes testing OR accountability" and subsequently analyzed. The analysis employed the VOSviewer software. Findings indicate a notable increase in publications in this field, particularly after 2010. Among collaborating organizations, Stanford University emerged as prominent, underscoring the United States' leadership in the domain. Co-citation analysis identified the American Educational Research Journal as the most influential journal. Furthermore, "accountability" was the most frequently used keyword. The trends revealed in the study highlight the role of high-stakes testing within educational systems, the increase in the number of studies over the years, and its multidimensional nature. The results demonstrate how high-stakes testing shape education and indicate that more research could be conducted in this area in the future.

Keywords

• High-stakes testing
• assessment systems
• education policy
• standardized testing
• bibliometric analysis

2000-2024 Arası Yüksek Riskli Testlerle İlgili Makalelerin Bibliyometrik Analizi

Abstract

Bu çalışmada, yüksek riskli testlerle ilgili yapılan makalelerin bibliyometrik analiz yöntemiyle incelenmesi amaçlanmıştır. Yüksek riskli testler, öğrenci performansı üzerinden okul ve eğitimcilerin değerlendirildiği ve eğitim politikalarının şekillendiği önemli bir araçtır. Çalışma kapsamında, Web of Science Core Collection (WoS) veri tabanından “Topic” etiketi ile “education AND high stakes testing OR accountability” anahtar kelimeleri kullanılarak 2000-2024 yılları arasında konuyla ilgili yapılan 1947 makale analiz edilmiştir. Analizde VOSviewer programı kullanılmıştır. Araştırma sonucunda özellikle 2010'dan sonra bu alandaki yayınlarda artış gözlenmiştir. En çok iş birliği yapılan kurumlar arasında Stanford Üniversitesi öne çıkarken, ABD’nin bu alandaki liderliği vurgulanmıştır. Ortak atıf analizinde, American Educational Research Journal en etkili dergi olarak belirlenmiştir. Ayrıca, anahtar kelimeler arasında “accountability” en çok kullanılan kavram olmuştur. Araştırmada ortaya çıkan eğilimler, yüksek riskli testlerin eğitim sistemleri içindeki yerini, yıllar içinde çalışma sayısında yaşanan artışı ve çok boyutlu yapısını ortaya koymaktadır. Sonuçlar, yüksek riskli testlerin eğitimi nasıl şekillendirdiğini ve gelecekte bu alanda daha fazla araştırma yapılabileceğini göstermektedir.

Keywords

• Yüksek riskli testler
• değerlendirme sistemleri
• eğitim politikaları
• standart testler
• bibliyometrik analiz

Supporting Institution

Bu makalede herhangi bir kurum veya kuruluştan destek alınmamıştır.

Ethical Statement

Çalışma, ULAKBİM TR Dizin tarafından açıklanan kriterlere göre Etik Kurul Onayı gerektirmemektedir.

References

1. Arviv Elyashiv, R., & Avidov-Ungar, O. (2024). Teachers’ perceptions of national large-scale assessment: the pedagogical dimension. Educational Review, 76(6), 1691–1707. https://doi.org/10.1080/00131911.2023.2256996
2. Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36(5), 258-267. https://doi.org/10.3102/0013189X07306523
3. Au, W. (2011). Teaching under the new Taylorism: High‐stakes testing and the standardization of the 21st century curriculum. Journal of Curriculum Studies, 43(1), 25–45. https://doi.org/10.1080/00220272.2010.521261
4. Au, W. (2016). Meritocracy 2.0: High-stakes, standardized testing as a racial project of neoliberal multiculturalism. Educational Policy, 30(1), 39-62. https://doi.org/10.1177/0895904815614916
5. Ball, S. J. (2003). The teacher's soul and the terrors of performativity. Journal of education policy, 18(2), 215-228. https://doi.org/10.1080/0268093022000043065
6. Bae, S. (2018). Redesigning systems of school accountability: A multiple measures approach to accountability and support. Education Policy Analysis Archives, 26(8). http://dx.doi.org/10.14507/epaa.26.2920
7. Booher-Jennings, J. (2005). Below the bubble: “Educational triage” and the Texas accountability system. American Educational Research Journal, 42(2), 231-268. https://doi.org/10.3102/00028312042002231
8. Burdett, N. (2016). The good, the bad, and the ugly–testing as a part of the education ecosystem (RISE-WP-16/010). Working paper series, research on improving systems of education (RISE).

9. Burgess, S., Metcalfe, R., & Sadoff, S. (2021). Understanding the response to financial and non-financial incentives in education: Field experimental evidence using high-stakes assessments. Economics of Education Review, 85, 102195. https://doi.org/10.1016/j.econedurev.2021.102195
10. Camphuijsen, M. K., Møller, J., & Skedsmo, G. (2021). Test-based accountability in the Norwegian context: Exploring drivers, expectations and strategies. Journal of Education Policy, 36(5), 624–642. https://doi.org/10.1080/02680939.2020.1739337
11. Carnoy, M., & Loeb, S. (2002). Does external accountability affect student outcomes? A cross-state analysis. Educational evaluation and policy analysis, 24(4), 305-331. https://doi.org/10.3102/01623737024004305
12. Dee, T. S., & Jacob, B. (2011). The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and Management, 30(3), 418-446. https://doi.org/10.1002%2Fpam.20586
13. Diamond, J. B., & Spillane, J. P. (2004). High-stakes accountability in urban elementary schools: Challenging or reproducing inequality?. Teachers College Record, 106(6), 1145-1176. https://doi.org/10.1111/j.1467-9620.2004.00375.x
14. Donthu, N., Kumar, S., & Pattnaik, D. (2020). Forty-five years of Journal of Business Research: A bibliometric analysis. Journal of Business Research, 109. https://doi.org/10.1016/j.jbusres.2019.10.039
15. Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research, 133, 285-296. https://doi.org/10.1016/j.jbusres.2021.04.070
16. Ellegaard, O., & Wallin, J. A. (2015). The bibliometric analysis of scholarly production: How great is the impact?. Scientometrics, 105, 1809-1831. https://doi.org/10.1007/s11192-015-1645-z
17. Elo, J., & Nygren-Landgärds, C. (2021). Teachers’ perceptions of autonomy in the tensions between a subject focus and a cross-curricular school profile: A case study of a Finnish upper secondary school. Journal of Educational Change, 22(3), 423-445. https://doi.org/10.1007/s10833-020-09412-0
18. Erdağ, C. (2019). Bir hesap verebilirlik teknolojisi olarak merkezi sınavlar: Finlandiya, Estonya ve Yeni Zelanda örnekleri. OPUS International Journal of Society Researches, 14(20), 1528-1563. https://doi.org/10.26466/opus.592335
19. Erduran, S., El Masri, Y., Cullinane, A., & Ng, Y. (2020). Assessment of practical science in high stakes examinations: A qualitative analysis of high performing English-speaking countries. International Journal of Science Education, 42(9), 1544–1567. https://doi.org/10.1080/09500693.2020.1769876
20. Escamilla, K., Chávez, L., & Vigil, P. (2005). Rethinking the “gap” high-stakes testing and Spanish-speaking students in Colorado. Journal of Teacher Education, 56(2), 132-144. https://doi.org/10.1177/0022487104273791
21. Falabella, A. (2021). The seduction of hyper-surveillance: Standards, testing, and accountability. Educational Administration Quarterly, 57(1), 113-142. https://doi.org/10.1177/0013161X20912299
22. Fidjeland, A. (2023). Using high-stakes grades to incentivize learning. Economics of Education Review, 94. https://doi.org/10.1016/j.econedurev.2023.102377
23. Finn, C. E. (2022). School accountability: Yesterday, today, and tomorrow. Phi Delta Kappan, 104(3), 12-17. https://doi.org/10.1177/00317217221136588
24. Fischer, C., Fishman, B., Levy, A. J., Eisenkraft, A., Dede, C., Lawrenz, F., Jia, Y., Kook, J. F., Frumin, K., & McCoy, A. (2020). When do students in Low-SES schools perform better-than-expected on a high-stakes test? Analyzing school, teacher, teaching, and professional development characteristics. Urban Education, 55(8-9), 1280-1314. https://doi.org/10.1177/0042085916668953
25. Fitchett, P. G., & Heafner, T. L. (2010). A national perspective on the effects of high-stakes testing and standardization on elementary social studies marginalization. Theory & Research in Social Education, 38(1), 114-130. https://doi.org/10.1080/00933104.2010.10473418
26. Goertz, M., & Duffy, M. (2003). Mapping the landscape of high-stakes testing and accountability programs. Theory Into Practice, 42(1), 4–11. https://doi.org/10.1207/s15430421tip4201_2
27. Gorgodze, S., & Chakhaia, L. (2021). The uses and misuses of centralised high stakes examinations-Assessment Policy and Practice in Georgia. Assessment in Education: Principles, Policy & Practice, 28(3), 322–342. https://doi.org/10.1080/0969594X.2021.1900775
28. Gulek, C. (2003). Preparing for high-stakes testing. Theory into practice, 42(1), 42-50. https://doi.org/10.1207/s15430421tip4201_6
29. Hammack, R., & Wilson, E. (2019). The dangers of high stakes testing in social studies. Journal of Social Studies and History Education, 2-26. Retrieved from https://www.uhd.edu/documents/academics/public-service/urban-education/jsshe/archives/2019/the-dangers-high-stakes-rhammack2.pdf
30. Hanushek, E. A., & Raymond, M. E. (2004). The effect of school accountability systems on the level and distribution of student achievement. Journal of the European Economic Association, 2(2-3), 406-415. Retrieved from https://academic.oup.com/jeea/article-abstract/2/2-3/406/2194942
31. Hanushek, E. A., & Raymond, M. E. (2005). Does school accountability lead to improved student performance?. Journal of Policy Analysis and Management, 24(2), 297-327. https://doi.org/10.1002/pam.20091
32. Hargreaves, A. (2020). Large-scale assessments and their effects: The case of mid-stakes tests in Ontario. Journal of Educational Change, 21(3), 393-420. https://doi.org/10.1007/s10833-020-09380-5
33. Hatfield, J. L., & Soløst, T. E. T. (2025). Assessing the assessment: Exploring Norwegian primary education teachers’ perceptions of national accountability testing. Scandinavian Journal of Educational Research, 69(2), 391-408. https://doi.org/10.1080/00313831.2024.2308883
34. Heilig, J. V., & Darling-Hammond, L. (2008). Accountability Texas-Style: The Progress and learning of urban minority students in a high-stakes testing context. Educational Evaluation and Policy Analysis, 30(2), 75-110. https://doi.org/10.3102/0162373708317689
35. Hilton, A. L., & Saunders, R. (2024). LANTITE’s impact on teacher diversity: Unintended consequences of testing pre-service teachers. The Australian Educational Researcher, 51(4), 1063-1082. https://doi.org/10.1007/s13384-023-00628-6
36. Hofflinger, A., & von Hippel, P. T. (2020). Missing children: How Chilean schools evaded accountability by having low-performing students miss high-stakes tests. Educational Assessment, Evaluation and Accountability, 32(2), 127-152. https://doi.org/10.1007/s11092-020-09318-8
37. Hursh, D. (2005). The growth of high‐stakes testing in the USA: Accountability, markets and the decline in educational equality. British Educational Research Journal, 31(5), 605–622. https://doi.org/10.1080/01411920500240767
38. Jacob, B. A. (2005). Accountability, incentives and behavior: The impact of high-stakes testing in the Chicago public schools. Journal of Public Economics, 89(5-6), 761-796. https://doi.org/10.1016/j.jpubeco.2004.08.004
39. Jacobs, M., van der Velden, R., & van Vugt, L. (2024). High-stakes testing and educational careers: Exploiting the differences in cutoffs between test recommendations in the Netherlands. Journal of Research on Educational Effectiveness, 17(4), 711–745. https://doi.org/10.1080/19345747.2023.2242342
40. Jerrim, J. (2023). Test anxiety: Is it associated with performance in high-stakes examinations?. Oxford Review of Education, 49(3), 321–341. https://doi.org/10.1080/03054985.2022.2079616
41. Karakus, M., Ersozlu, A., & Clark, A. C. (2019). Augmented reality research in education: A bibliometric study. Eurasia Journal of Mathematics, Science and Technology Education, 15(10). https://doi.org/10.29333/ejmste/103904
42. Kavanagh, K. M., & Fisher-Ari, T. R. (2020). Curricular and pedagogical oppression: Contradictions within the juggernaut accountability trap. Educational Policy, 34(2), 283-311. https://doi.org/10.1177/0895904818755471
43. Kenney, A. W., Dulong Langley, S., Hemmler, V., Callahan, C. M., Gubbins, E. J., & Siegle, D. (2024). Different or differentiated? Recoupling Policy and practice in an era of accountability. Educational Policy, 38(1), 134-160. https://doi.org/10.1177/08959048231153612
44. Kirby, S. N., & Stecher, B. M. (2004). Accountability in education. In B.M. Stecher, & S. N. Kirby (Eds.), Organizational improvement and accountability: Lessons for education from other sectors (pp. 1-10). Rand Corporation.
45. Klinger, D. A., & Rogers, W. T. (2011). Teachers’ perceptions of large-scale assessment programs within low-stakes accountability frameworks. International Journal of Testing, 11(2), 122–143. https://doi.org/10.1080/15305058.2011.552748
46. Knoester, M., & Meshulam, A. (2022). Beyond deficit assessment in bilingual primary schools. International Journal of Bilingual Education and Bilingualism, 25(3), 1151–1164. https://doi.org/10.1080/13670050.2020.1742652
47. Levatino, A., Parcerisa, L., & Verger, A. (2024). Understanding the stakes: The influence of accountability policy options on teachers’ responses. Educational Policy, 38(1), 31-60. https://doi.org/10.1177/08959048221142048
48. McCoy, S., & Byrne, D. (2024). Shadow education uptake in Ireland: Inequalities and wellbeing in a high-stakes context. British Journal of Educational Studies, 72(6), 693–719 https://doi.org/10.1080/00071005.2024.2331476
49. McElroy, K. (2023). Does test-based accountability improve more than just test scores?. Economics of Education Review, 94, 102381. https://doi.org/10.1016/j.econedurev.2023.102381
50. McNeil, L. M. (2000). Contradictions of school reform: Educational costs of standardized testing. New York: Routledge.
51. Merigó, J. M., & Yang, J. B. (2017). A bibliometric analysis of operations research and management science. Omega, 73, 37-48. https://doi.org/10.1016/j.omega.2016.12.004
52. Merigó, J. M., Pedrycz, W., Weber, R., & de la Sotta, C. (2018). Fifty years of Information Sciences: A bibliometric overview. Information Sciences, 432, 245-268. https://doi.org/10.1016/j.ins.2017.11.054
53. Mintz, J. A., & Kelly, A. M. (2021). Science teacher motivation and evaluation policy in a high-stakes testing state. Educational Policy, 35(1), 3-40. https://doi.org/10.1177/0895904818810520
54. Moore, D. P. (2025). Culturally sustaining catch-up? How English language arts teachers “compensate” with culturally sustaining pedagogy after the state exam. Urban Education, 60(7), 2016-2050. https://doi.org/10.1177/00420859241266605
55. Munoz-Chereau, B., González, Á., & Meyers, C. V. (2022). How are the ‘losers’ of the school accountability system constructed in Chile, the USA and England? Compare: A Journal of Comparative and International Education, 52(7), 1125–1144. https://doi.org/10.1080/03057925.2020.1851593
56. Neal, D., & Schanzenbach, D. W. (2010). Left behind by design: Proficiency counts and test-based accountability. The Review of Economics and Statistics, 92(2), 263-283. https://doi.org/10.1162/rest.2010.12318
57. Ober, T. M., Hong, M. R., Carter, M. F., Brodersen, A. S., Rebouças-Ju, D., Liu, C., & Cheng, Y. (2022). Are high school students accurate in predicting their AP exam scores?: Examining inaccuracy and overconfidence of students’ predictions. Assessment in Education: Principles, Policy & Practice, 29(1), 27–50. https://doi.org/10.1080/0969594X.2022.2037508
58. Orfield, G., & Wald, J. (2000). Testing, testing: The high-stakes testing mania hurts poor and minority students the most. The Nation, 270(22), 38–40. Retrieved from https://thenation.s3.amazonaws.com/pdf/testingtesting2000.pdf
59. Ozga, J. (2009). Governing education through data in England: from regulation to self‐evaluation. Journal of Education Policy, 24(2), 149–162. https://doi.org/10.1080/02680930902733121
60. Özaslan, A. & Beyhan, Ö. (2023). Students’ understandings related to their upcoming high-stakes tests: A phenomenographic study. Journal of Teacher Education and Lifelong Learning, 5(2), 789-803. https://doi.org/10.51535/tell.1288890
61. Pace, J. L. (2011). The complex and unequal impact of high stakes accountability on untested social studies. Theory & Research in Social Education, 39(1), 32-60. https://doi.org/10.1080/00933104.2011.10473446
62. Parkison, P. (2009). Political economy and the NCLB regime: Accountability, standards, and high-stakes testing. The Educational Forum, 73(1), 44–57. https://doi.org/10.1080/00131720802539606
63. Passas, I. (2024). Bibliometric analysis: the main steps. Encyclopedia, 4(2), 1014-1025. https://doi.org/10.3390/encyclopedia4020065
64. Rhoten, D., Carnoy, M., Elmore, R., & Chabran, M. (2003). The conditions and characteristics of as- sessment and accountability: The case of four states. In M. Carnoy, R. Elmore, & L. S. Siskin (Eds.), The new accountability: High schools and high-stakes testing (pp. 13-53). Routledge.
65. Sloane, F. C., & Kelly, A. E. (2003). Issues in high-stakes testing programs. Theory Into Practice, 42(1), 12-17. https://doi.org/10.1207/s15430421tip4201_3
66. Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265-269. https://doi.org/10.1002/asi.4630240406
67. Supovitz, J. (2009). Can high stakes testing leverage educational improvement? Prospects from the last decade of testing and accountability reform. Journal of Educational Change, 10, 211–227. https://doi.org/10.1007/s10833-009-9105-2
68. Tan, S. H. (2010). Singapore’s educational reforms: The case for un-standardizing curriculum and reducing testing. AASA Journal of Scholarship and Practice, 6(4), 50-58.
69. Thomas, R. M. (2005). High-stakes testing: Coping with collateral damage. Routledge.
70. Thompson, G. (2013). NAPLAN, MySchool and accountability: Teacher perceptions of the effects of testing. International Education Journal: Comparative Perspectives, 12(2), 62–84. Retrieved from https://openjournals.library.sydney.edu.au/IEJ/article/view/7456
71. Thompson, G., & Lašič, T. (2011). Doing something about it: Representations of NAPLAN in the public domain [Paper presentation]. AARE Annual Conference, Hobart, TAS, Australia. Retrieved from https://www.aare.edu.au/data/publications/2011/aarefinal00577.pdf
72. Trujillo, T., Møller, J., Jensen, R., Kissell, R. E., & Larsen, E. (2021). Images of educational leadership: How principals make sense of democracy and social justice in two distinct policy contexts. Educational Administration Quarterly, 57(4), 536-569. https://doi.org/10.1177/0013161X20981148
73. Van Eck, N. J., & Waltman, L. (2023, Ekim 31). VOSviewer manual (Sürüm 1.6.20). Centre for Science and Technology Studies (CWTS), Leiden University. https://www.vosviewer.com
74. Verger, A., Parcerisa, L., & Fontdevila, C. (2019). The growth and spread of large-scale assessments and test-based accountabilities: A political sociology of global education reforms. Educational Review, 71(1), 5–30. https://doi.org/10.1080/00131911.2019.1522045
75. Von der Embse, N. P., & Witmer, S. E. (2014). High-stakes accountability: Student anxiety and large-scale testing. Journal of Applied School Psychology, 30(2), 132–156. https://doi.org/10.1080/15377903.2014.888529
76. Wallin, J. A. (2005). Bibliometric methods: Pitfalls and possibilities. Basic & Clinical Pharmacology & Toxicology, 97(5), 261-275. https://doi.org/10.1111/j.1742-7843.2005.pto_139.x
77. Wang, J., Li, Q., & Luo, Y. (2022). Physics identity of Chinese students before and after Gaokao: The effect of high-stake testing. Research in Science Education, 52(2), 675-689. https://doi.org/10.1007/s11165-020-09978-y
78. Xie, Q. (2022). The role of parents in the school accountability system: Insights from a Hong Kong case. Studies in Educational Evaluation, 72. https://doi.org/10.1016/j.stueduc.2021.101114
79. Yin, Y. M., & Mu, G. M. (2022). Examination-oriented or quality-oriented? A question for fellows of an alternative teacher preparation program in China. The Australian Educational Researcher, 49(4), 727-742. https://doi.org/10.1007/s13384-021-00462-8
80. Yung, K. W. H. (2021). Shadow education as a form of oppression: conceptualizing experiences and reflections of secondary students in Hong Kong. Asia Pacific Journal of Education, 41(1), 115–129. https://doi.org/10.1080/02188791.2020.1727855
81. Zakharov, A., & Carnoy, M. (2021). Does teaching to the test improve student learning?. International Journal of Educational Development, 84, 102422. https://doi.org/10.1016/j.ijedudev.2021.102422