Development of Test-Taking Strategies Scale: High School and Undergraduate Form

Abstract

Test-taking strategies are discussed in the literature as an important factor affecting test scores, and are recommended to be taken into consideration regarding the validity of tests. Although studies have been conducted for more than a quarter of a century, no agreement has yet been reached as to the dimensions of test-taking strategies. The aim of this study is to develop a valid and reliable test-taking strategies scale for undergraduate and high school students who experience intensive exam periods. In the scale prepared for this purpose, we consider tests containing different item types and focus on pre-test, during-test and post-test strategies, excluding test-preparation. Two separate forms of test-taking strategies at high school (27 items) and undergraduate levels (18 items) were developed using promising measurement theories and models. Results indicated that the Person Separation Index, as a reliability index, was calculated as .88 for the high-school form and .93 for the undergraduate form. This study is significant in presenting valid and reliable tools for measuring test-taking strategies, and is considered an exemplary research that uses Partial Credit Model for Likert-type scale development.

Keywords

• Scale development
• Likert type
• Partial Credit Model
• Test-taking strategies
• test skilss

References

1. Abrams, L. M. (2004). Teachers’ views on high-stakes testing: Implications for the classroom policy brief. Arizona State University, Education Policy Studies Laboratory. http://edpolicylab.orghttp//edpolicylab.org
2. American Psychological Association, American Educational Research Education, & National Council on Measurement in Standards. (1966). Standards for educational and psychological tests and manuals. APA.
3. Anderson, N. J. (1991). Individual differences in strategy use in second language reading and testing. The Modern Language Journal, 75(4), 460-472. https://doi.org/10.2307/329495
4. Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford university.
5. Baykul, Y. (2010). Eğitim ve psikolojide ölçme ve değerlendirme [Measurement and evaluation in education and psychology]. Pegem Akademi.
6. Beidel, D. C., Turner, S. M., & Taylor-Ferreira, J. C. (1999). Teaching study skills and test-taking strategies to elementary school students, 23(4), 630-646. https://doi.org/10.1177%2F0145445599234007
7. Bıçak, B. (2013). Teste hazırlık ve test yanıtlama stratejileri ölçeği [Scale for test preparation and test taking strategies]. Kuram ve Uygulamada Eğitim Bilimleri, 13(1), 273-289.
8. Bland, J. M., & Altman, D. G. (1995). Multiple significance tests: the Bonferroni method. Bmj, 10(6973), 170. Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge.

9. Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4
10. Boud, D., & Falchikov, N. (2007). Rethinking assessment in higher education: Learning for the longer term. Routledge.
11. Boyd, R. T. C. (1989). Improving your test-taking skills. Practical Assessment, Research, and Evaluation, 1(1), Article 2. https://doi.org/https://doi.org/10.7275/ypnv-6b10
12. Bruch, M. A. (1981). Relationship of test-taking strategies to test anxiety and performance: Toward a task analysis of examination behavior. Cognitive Therapy and Research, 5(1), 41–56.
13. Brunfaut, T., & McCray, G. (2015). Looking into test-takers’ cognitive processes while completing reading tasks: A mixed-method eye-tracking and stimulated recall study. British Council
14. Chittooran, M. M., & Miles, D. D. (2001). Test-taking skills for multiple-choice formats: Implications for school psychologists. Saint Louis University.
15. Cohen, A. D. (2007). The coming of age for research on test-taking strategies. In Language Testing Reconsidered (pp. 89–112). University of Ottawa Press.
16. Crocker, L., & Algina, J. (2006). Introduction to classical and modern test theory. Cengage Learning.
17. Demirci, İ., & Erden, S. (2016). Bilişsel davranışçı yaklaşıma dayalı grupla psikolojik danışma uygulamasının 8. sınıf öğrencilerinin sınav kaygısına etkisi [The effect of cognitive-behavioral group counseling on test anxiety of eight-grade elementary school students]. Marmara Üniversitesi Atatürk Eğitim Fakültesi Eğitim Bilimleri Dergisi, 43(43), 67. https://doi.org/10.15285/ebd.51646
18. Demirtaşli, N., Yalçin, S., & Ayan, C. (2016). Ölçme ve değerlendirme dersine yönelik tutum ölçeğinin madde tepki kuramına dayalı olarak geliştirilmesi [The Development of IRT Based Attitude Scale towards Educational Measurement Course]. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 7(1), 133–144. https://doi.org/10.21031/epod.43804
19. Dodeen, H. (2008). Assessing test-taking strategies of university students: Developing a scale and estimating its psychometric indices. Assessment and Evaluation in Higher Education, 33(4), 409–419. https://doi.org/10.1080/02602930701562874
20. Dodeen, H. (2014). Test-related characteristics of 0UAEU students: Test-anxiety, test- taking skills, guessing, attitudes toward tests, and cheating. Journal of Faculty of Education, 26(J2009), 31–66.
21. Dodeen, H. (2015). Teaching test-taking strategies: importance and techniques. Psychology Research, 5(2), 108–113. https://doi.org/10.17265/2159-5542/2015.02.003
22. Dodeen, H. M., Abdelfattah, F., & Alshumrani, S. (2014). Test-taking skills of secondary students: The relationship with motivation, attitudes, anxiety and attitudes towards tests. South African Journal of Education, 34(2), Article 866. https://doi.org/10.15700/201412071153
23. Educational Testing Service. (2001). Graduate Record Examinations psychology practice book. Educational Testing Service.
24. Embretson, S., & Reise, S. P. (2000). Item Response Theory for Psychologists (1st ed.). Erlbaum.
25. Engelhard, G., & Wind, S. A. (2017). Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments. Routledge. https://doi.org/10.4324/9781315766829
26. Erkuş, A. (2012). Psikolojide ölçme ve ölçek geliştirme [Measurement and scale development in psychology]. Pegem Akademi.
27. Fallows, S. J., Fallows, S., & Steven, C. (2000). Integrating key skills in higher education: Employability, transferable skills, and learning for life. Psychology Press.
28. Flitcroft, D., Woods, K., & Putwain, D. W. (2017). Developing school practice in preparing students for high-stake examinations in English and Mathematics. Educational and Child Psychology, 34(3), 7–19.
29. Güler, N. (2014). Analysis of open-ended statistics questions with many facet rasch model. Eurasian Journal of Educational Research, 55, 73–90. https://doi.org/10.14689/ejer.2014.55.5
30. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Sage.
31. Hong, E., Sas, M., & Sas, J. C. (2006). Test-taking strategies of high and low mathematics achievers. Journal of Educational Research, 99(3), 144–155. https://doi.org/10.3200/JOER.99.3.144-155
32. İlhan, M., & Güler, N. (2018). Likert tipi ölçeklerde Rasch modelinin kullanımı: olumsuz değerlendirilme korkusu ölçeği-öğrenci formu (ODKÖ-ÖF) üzerinde bir uygulama [The use of Rasch model in Likert types scales: An application on the fear of negative evaluation scale-student form. Trakya Üniversitesi Eğitim Fakültesi Dergisi, 8(4), 756-775.
33. Kartal, K., & Dirlik, M. (2016). Historical development of the concept of validity and the most preferred technique of reliability: Cronbach alpha coefficient. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 16(4), 1865-1879.
34. Kaya Uyanık, G., Güler, N., Taşdelen Teker, G., & Demir, S. (2019). Fen bilimleri dersi etkinliklerinin Çok Yüzeyli Rasch Modeliyle [The analysıs of elementary science education course actıvıtıes through Many-Facet Rasch Model]. Kastamonu Eğitim Dergisi, 27(1), 139–150. https://doi.org/10.24106/kefdergi.2417
35. Kesselman-Turkel, J., & Peterson, F. (2004). Test-taking strategies. University of Wisconsin.
36. Kizilkaya, G., & Aşkar, P. (2009). Problem çözmeye yönelik yansitici düşünme becerisi ölçeǧinin geliştirilmesi [The development of a reflective thinking skill scale towards problem solving]. Egitim ve Bilim, 34(154), 82-92.
37. Koch, W. R. and Dodd, B. G. (1989). An investigation of procedures for computerized adaptive testing using partial credit scoring. Applied Measurement in Education, 2(4), 335-357.
38. Kline, R. B. (2011). Principles and practice of structural equation modeling. New York: The Guilford Press.
39. Kutlu, Ö., Yıldırım, Ö., & Bilican, S. (2009). Öğretmenlerin dereceli puanlama anahtarlarına ilişkin tutum ölçeği geliştirme çalışması [Study of attıtudes scale development aımed at scorıng rubrıcs for prımary school teachers]. Yüzüncü Yıl Üniversitesi Eğitim Fakültesi Dergisi, 6(2), 76-88.
40. Lewandowski, L., Gathje, R. A., Lovett, B. J., & Gordon, M. (2013). Test-taking skills in college students with and without ADHD. Journal of Psychoeducational Assessment, 31(1), 41–52. https://doi.org/10.1177/0734282912446304 Linacre, J. M. (1994). Many-facet Rasch measurement (2nd ed.). Mesa.
41. Pehlivan, E. B., & Kutlu, Ö. (2014). Türkçe test maddelerinde yanıtlama davranışlarının incelenmesi [Investigation of answering behaviour in Turkish test]. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 5(1), 61–71. https://doi.org/10.21031/epod.20130
42. Peng, Y., Hong, E., & Mason, E. (2014). Motivational and cognitive test-taking strategies and their influence on test performance in mathematics. Educational Research and Evaluation, 20(5), 366–385. https://doi.org/10.1080/13803611.2014.966115
43. Primi, R., Silvia, P. J., Jauk, E., & Mathias, B. (2019). Applying Many-Facet Rasch Modeling in the Assessment of Creativity. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 176–186. https://doi.org/10.1037/aca0000230
44. Roderer, T., & Roebers, C. M. (2014). Can you see me thinking (about my answers)? Using eye-tracking to illuminate developmental differences in monitoring and control skills and their relation to performance. Metacognition Learning, 9, 1–23. https://doi.org/10.1007/s11409-013-9109-4
45. Rozakis, L. (2003). Test-taking strategies and study skills for the utterly confused. McGraw-Hill. Şenel, S., & Kutlu, Ö. (2018). Comparison of two test methods for VIS: paper-pencil test and CAT. European Journal of Special Needs Education, 33(5), 631–645. https://doi.org/10.1080/08856257.2017.1391014
46. Şenel, S., & Şenel, H. C. (2018). Bilgisayar tabanlı testlerde evrensel tasarım: Özel gereksinimli öğrenciler için düzenlemeler [Universal design for computer-based testing: Accommodations for students with special needs]. In S. Dinçer (Ed.), Değişen dünyada eğitim [Education in versatile world] (1st ed., pp. 113-124). Pegem Akademi. https://doi.org/10.14527/9786052412480.08
47. Smith, M. D. (2017). Cognitive Validity: Can Multiple-Choice Items Tap Historical Thinking Processes? American Educational Research Journal, 54(6), 1256-1287. https://doi.org/10.3102/0002831217717949
48. Tabachnick B G & Fidell L S (2007) Using multivariate statistics. Boston, Pearson Education, Inc.
49. Tekin, H. (1996). Eğitimde ölçme ve değerlendirme [Measurement and evaluation in education] (9th ed.). Yargı.
50. Tennant, A., & Conaghan, P. G. (2007). The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Care & Research, 57(8), 1358-1362.
51. Therrien, W. J., Hughes, C., Kapelski, C., & Mokhtari, K. (2009). Effectiveness of a test-taking strategy on achievement in essay tests for students with learning disabilities. Journal of Learning Disabilities, 42(1), 14–23. https://doi.org/10.1177/0022219408326218
52. Ün Açıkgöz, K. (2003). Etkili öğrenme ve öğretme [Effective learning and teaching] (6th ed.). Biliş.
53. Van Zile-Tamsen, C. (2017). Using Rasch Analysis to Inform Rating Scale Development. Research in Higher Education, 58(8), 922-933. https://doi.org/10.1007/s11162-017-9448-0
54. Wongpakaran, N., Wongpakaran, T., Pinyopornpanish, M., Simcharoen, S., Suradom, C., Varnado, P., & Kuntawong, P. (2020). Development and validation of a 6‐item Revised UCLA Loneliness Scale (RULS‐6) using Rasch analysis. British Journal of Health Psychology, 25(2), 233-256. https://doi.org/10.1111/bjhp.12404
55. Wright, B. D. (1996). Local dependency, correlations and principal components. Rasch Measurement Transactions, 10(3), 509-511.
56. Yaşar, M., & Aybek, E. C. (2019). A resilience scale development for university students: Validity and reliability study based on item response theory. Elementary Education Online, 18(4), 1687-1699. https://doi.org/10.17051/ilkonline.2019.635031
57. Yıldırım, A., Doğanay, A., & Türkoğlu, A. (2000). Okulda başarı için ders çalışma ve öğrenme yöntemleri [Study and learning methods for success in school]. Seçkin Academy.