Ability Level Estimation of Students on Probability Unit via Computerized Adaptive Testing

Abstract

Problem Statement: Learning-teaching activities bring along the need to determine whether they achieve their goals. Thus, multiple choice tests addressing the same set of questions to all are frequently used. However, this traditional assessment and evaluation form contrasts with modern education, where individual learning characteristics are featured. Hence, the use of Computerized Adaptive Testing (CAT) systems, which set the difficulty level in accordance with the ability levels of individuals, is spreading. However, these systems are not prevalent in Turkey. Therefore, it is important to develop and assess a CAT system to be integrated into Turkish curriculum.

Purpose of Study: The purpose of this study is to develop a CAT system that can be used in the sub-levels of Turkish curricula (high school) and assess it in terms of reliability.

Methods: In this study, a CAT system aimed at the sub-topics (permutation, combination, binomial expansion, and probability) of the unit of probability covered in 11^th grade mathematics was developed. Estimation sensitivity of the developed CAT system, from lower to higher ability levels, was assessed. To this end, an exam was implemented in a high school located in Trabzon in the 2011-2012 fall semester. Eighty-four 11^th graders participated in the implementation. Using the data derived from the implementation, reliability coefficient values of each learner were calculated. System records were used for data collection. 

Keywords

• -

References

1. Baki, A. (2008). Teoriden pratiğe matematik eğitimi [Mathematics education from theory to practice]. Ankara: Harf Eğitim Yayıncılık.
2. Bejar, I. I., Weiss, D. J., & Gialluca, K .A. (1977). An information comparison of conventional and adaptive tests in the measurement of classroom achievement (RR 77-7). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.
3. Bulut, O., & Kan, A. (2012) Application of computerized adaptive testing to entrance examination for graduate studies in Turkey. Eurasian Journal of Educational Research, 49, 61-80.
4. Choi, S. W., Reise, S. P., Pilkonis P. A., Hays, R. D., & Cella, D. (2010). Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Quality of Life Research, 19(1), 125-136.
5. Fliege, H., Becker, J., Walter, O. B., Bjorner J. B., Klapp, B. F., & Rose M. (2005). Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research, 14(1), 2277–2291.
6. Frey, A., & Seitz, N .N. (2011). Hypothetical use of multidimensional adaptive testing for the assessment of student achievement in the programme for international student assessment. Educational and Psychological Measurement, 71(3), 503-522.
7. Georgiadou, E., Triantafillou, E., & Economides, A.A. (2006). Evaluation parameters for computer-adaptive testing, British Journal of Educational Technology, 37(2), 261-278.
8. Gouli, E., Kornilakis, H., Papinakolaou, K., & Grigoriadou, M. (2001). Adaptive Assessment Improving Interaction in an Educational Hypermedia System. Proceedings of the PanHellenic Conference with International Participation in Human-Computer Interaction, 217- 222.

9. Hambleton, R.K. (1990). Item response theory: Introduction and bibliography. Psicothema, 2(1), 97-107.
10. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. California: SAGE Publications.
11. Huang, Y-M., Lin, Y-T., & Cheng, S-C. (2009). An adaptive testing system for supporting versatile educational assessment. Computers & Education, 52(1), 53-67.
12. Koong C-S., & Wu, C-Y. (2010). An interactive item sharing website for creating and conducting on-line testing. Computers & Education, 55(2010), 131–144.
13. Lilley, M. (2007). The development and application of computer-adaptive testing in a higher education environment. Unpublished doctoral dissertation, Hertfordshire University, England.
14. Liu, Y-C., & Chen, P-J. (2012). Discovering discriminative test items for achievement tests. Expert Systems with Applications, 39(1), 1426-1434.
15. López-Cuadrado, J., Perez, T. A., Vadillo, J. A., & Arruabarrena, R. (2002). Integrating adaptive testing in an educational system. In Proc, First International Conference on Educational Technology in Cultural Context: ETCC2002, Joensuu, Finland, 133-149
16. López-Cuadrado, J., Armendariz, A., & Perez, T.A. (2006). Adaptive evaluation in an e-learning system architecture. Current Developments in Technology-Assisted Education, 1507-1511.
17. Lunz, M. E., Bergstrom, B. A., & Gershon, R. C. (1994). Computer adaptive testing. International Journal of Educational Research, 21(6), 623-634.
18. Marinagi, C. C., Kaburlasos, V. G., & Tsoukalas, V. T. (2007). An architecture for an adaptive assessment tool. 37th ASEE/IEEE Frontiers in Education Conference, Milwaukee, WI.
19. Önder, İ. (2007). An investigation of goodness of model data fit. Hacettepe University Journal of Education, 32, 210-220.
20. Özdamar, K. (2004). Paket programlarla veri analizi [Statistical data analysis through packages]. Eskişehir: Kaan Yayıncılık.
21. Özyurt, Ö., Özyurt, H., & Baki, A. (2013). Design and development of an innovative individualized adaptive and intelligent e-learning system for teaching-learning of probability unit: Details of UZWEBMAT. Expert Systems with Applications, 40(8), 2914-2940.
22. Öztuna, D. (2008). Kas iskelet sistem sorunlarının özürlülük değerlendiriminde bilgisayar uyarlamalı test yönteminin uygulanması [Implementing computer adaptive testing method to estimate disability levels in musculoskeletal system disorders]. Unpublished doctoral dissertation, Ankara Üniversitesi Sağlık Bilimleri Enstitüsü. Ankara.
23. Ponsoda, V. (2000). Overview of computerized adaptive testing special section. Psicologica, 21(1), 115-120.
24. Reckase, M. D. (2010). Designing item pools to optimize the functioning of a computerized adaptive test. Psychological Test and Assessment Modeling, 52(2), 127-141.
25. Rios, A., Millán, E., Trella, M., Pérez-de-la-Cruz, J. L., & Conejo, R. (1999). Internet based evaluation system. In Proceedings of the 9th World Conference of Artificial Intelligence and Education AIED'99, 387-394.
26. Rudner, L.M. (2002). An examination of decision-theory adaptive testing procedures. Conference of American Educational Research Association, New Orleans, April 1-5.
27. Semerci, Ç., & Bektaş, C., 2005. New approaches in ensuring the validity of internet-based measurements. The Turkish Online Journal of Educational Technology, 4(1), 130-134.
28. Thissen, D. (2000). Reliability and measurement precision. In: H. Wainer (Ed). Computerized adaptive testing: A primer (159-183), Mahwah, NJ: Lawrence Erlbaum Associates.
29. Vispoel, W. P., Rocklin, T. R., & Wang, T. (1994). Individual differences and test administration procedures: a comparison of fixed-item, computerized-adaptive, and self-adapted testing. Applied Measurement in Education, 7(1), 53-79.
30. Walter O. B., Becker J., Bjorner J. B., Fliege, H, Klapp, B. F., & Rose M. (2007). Development and evaluation of a computer adaptive test for 'anxiety' (Anxiety-CAT), Quality of Life Research, 16(1), 143–155.
31. Weiss, D. J. (1982) Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473-492.
32. Weiss, D. J. (1985). Adaptive testing by computer. Journal of Consulting and Clinical Psychology, 53(6), 774-789.
33. Weiss, D. J. (2004). Computerized adaptive testing for effective and efficient measurement in counseling and education. Measurement and Evaluation in Counseling and Development, 37(2), 70-84.
34. Wise, S. L., & Kingsbury, G .G. (2000). Practical issues in developing and maintaining a computerized adaptive testing program. Psicológica, 21, 135-155.