-

Abstract

Many studies examining on the comparison of computer-based tests and paper-and-pencil tests are available in the literature. These studies are applied to various fields, variables and workgroups. So it is important to compile and review these studies by using meta-analysis for reaerchers and authorities. In this paper, a survey of the studies on comparison of student performances in paper-based and computer-based tests is presented. By considering these studies, a meta-analysis on the comparison of students’ achievements for paperbased and computer-based tests is provided. To determine if the studies included into meta-analysis have bias, Funnel Graph and Begg-Mazumdar correlation test are utilized for sensitivity analysis. Influence quantities pertaining the studies are analyzed by using Comprehensive Meta-Analysis software. Results of heterogeneity test are analyzed to determine the best effect model and then it is seen that effect quatities shows normal distribution. Based on that, Fixed Effect Model is selected. Hedge’s g coefficient is used to calculate the common effect. There are two basic restrictions of this study. First, due to the lack of time, it could not be possible to reach all studies available for the comparison of paper-based and computer-based tests. Second, results of the meta-analysis are not compared with respect to the topic fields. It could be better to compare the results for topic fields as future prospect. According to 35 findings related to the 9 different studies included to the meta-analysis, no reasonable difference is observed for paper-based and computer-based tests for 29 comparisons, and some reasonable differences are observed 6 ones. Common influence quantity obtained from all the studies is found as -0.09. This necessarily means that the differences between scores obtained from paper-based and computer-based tests corresponds to 0.09 standard deviation, which is not a considerable influence quantity. In other words, according to the meta-analysis results, there is no substantial difference for the students’ performance in paper-based and computer-based tests. As a result, even though it is not inconvenient to use paper-based tests, it should be determined while preparing a computer-based test whether student achievement is affected by applied test form via some trial applications

Keywords

• Computer-based test, paper-based test, meta-analysis

KAĞIT-KALEM VE BİLGİSAYAR FORMUNDA UYGULANAN TESTLERDE ÖĞRENCİ BAŞARISININ KARŞILAŞTIRILDIĞI ÇALIŞMALARIN META-ANALİZİ

Öz

Araştırmada, kağıt-kalem formu ve bilgisayar ortamında uygulanan testlerde gösterilen öğrenci performanslarını karşılaştıran çalışmalar taranmıştır. Bu çalışmalar üzerinde kağıt-kalem formu ve bilgisayar ortamında uygulanan testlerin öğrenci performansı bakımından karşılaştırılması ile ilgili meta analiz çalışması yapılmıştır. İncelenen çalışmalara ait etki büyüklükleri Comprehensive Meta Analysis yazılımı ile incelenmiştir. Meta-analize dahil edilen 9 farklı çalışmaya ait toplam 35 bulguya göre, 29 karşılaştırmada bilgisayar ve kağıt-kalem formları arasında manidar bir fark bulunmazken, 6 çalışmada bilgisayar ve kağıt-kalem formları arasında manidar bir fark bulunmuştur. Tüm çalışmalardan elde edilen ortak etki büyüklüğünün -.09 olduğu görülmüştür. Bu durum, kağıt-kalem ve bilgisayar ortamında uygulanan testlerden elde edilen puanlar arasındaki farkın .09 standart sapmaya denk olduğunu belirtmektedir –ki bu oldukça küçük bir etki büyüklüğünü gösterir. Dolayısıyla meta-analiz sonuçlarına göre kağıt-kalem ve bilgisayar ortamında uygulanan testlerde gösterilen öğrenci performansları arasında manidar bir fark yoktur. Bu durumda, bilgisayarlı testlere geçilmesinde bir sakınca görülmemekle birlikte, bilgisayarlı test uygulamaları geliştirilirken yapılacak deneme uygulamaları ile öğrenci performansının formlara göre farklılık gösterip göstermediğinin araştırılması gerekmektedir.

Anahtar Kelimeler

• Bilgisayar ortamında uygulanan test, kağıt-kalem formu test, meta-analiz

References

1. Aybek, E. C. (2012). Kağıt-Kalem Formu ve Bilgisayar Ortamında Uygulanan Genel Yetenek Testinin Psikometrik Özelliklerinin Karşılaştırılması, Yayınlanmamış Yüksek Lisans Tezi, Ankara Üniversitesi Eğitim Bilimleri Enstitüsü.
2. Bayazıt, A. (2007). Çevrimiçi Sınavlar Ve Kâğıt-Kalem Sınavları Arasındaki Sınav Süresi ve Öğrenci Başarım Farklılıkları, Yayınlanmamış Yüksek Lisans Tezi, Hacettepe Üniversitesi Fen Bilimleri Enstitüsü.
3. Boo, J., & Vıspoel, W. (2012). Computer Versus Paper-And-Pencıl Assessment Of Educatıonal Development: A Comparıson Of Psychometrıc Features And Examınee Preferences 1. Psychological Reports, 111(2), 443–460. doi:10.2466/10.03.11.PR0.111.5.443-460
4. Bugbee Jr., A. C., Bernt, F. M. (1990). Testing by computer: Findings in six years of use 1982-1988. Journal of Research on Computing in Education, Vol. 23 Issue 1, 87-101.
5. Chan-Pensley, E. (1999). Alcohol-Use Disorders Identification Test: a comparison between paper and pencil and computerized versions. Alcohol and alcoholism (Oxford, Oxfordshire), 34(6), 882–885. 04.04.2013 tarihinde http://www.ncbi.nlm.nih.gov/pubmed/10659724 adresinden alınmıştır.
6. Drasgow, F (2002). The work ahead: A psychometric infrastructure for computerized adaptive tests. In C.N. Mills,
7. M.T. Potenza, J.J. Fremer, & W.C. Ward (Eds.), Computer-based testing: Building the foundation for future assessments (pp. 67–88). Hillsdale, NJ: Lawrence Erlbaum.
8. Durlak, J.A. (1995). Understanding meta-analysis. In L.G. Grimm, & P.R. Yarnold (Eds.), Reading and understanding multivariate statistics (pp. 319-352). Washington, DC: American Psychological Association.

9. Helgeson, S. L., Kumar, D. D. (1993). A Review of Educational Technology in Science Assessment. National Center for Science Teaching and Learning, Columbus, Monograph Series Number 7, OH., (ERIC Document Reproduction Service No. ED366507).
10. Kim, D., & Huynh, H. (2007). Comparability of computer and paper-and-pencil versions of algebra and biology assessments. The Journal of Technology, Learning and Assessment, 6(4), 1–29. 08.05.2013 tarihinde http://napoleon.bc.edu/ojs/index.php/jtla/article/view/1634 adresinden alınmıştır.
11. Kumar, D. (1996). Computers and Assessment in Science Education. ERIC Clearinghouse for Science, Mathematics and Environmental Education, Columbus, OH., (ERIC Document Reproduction Service No. ED395770).
12. Luecht, R. M. (2001). Challenges of Web-Based Assessment. (ERIC Document Reproduction Service No. ED458294).
13. Mark Pomplun, Sharon Frey and Douglas F. Becker (2002). The Score Equıvalence Of Paper-And-Pencıl And Computerızed Versıons Of A Speeded Test Of Readıng Comprehensıon. Educational and Psychological Measurement 2002 62: 337.
14. Natal, D. (1998). On-Line Assessment: What, Why, How. Imagen Multimedia Corp., Lompoc, CA., (ERIC Document Reproduction Service No. ED419552).
15. Özalp-Yaman, Ş., & Çağıltay, N. E. (2010). Paper-based versus computer-based testing in engineering education. IEEE EDUCON Education Engineering (pp. 1631–1637). 05.05.2013 tarihinde http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5492397 adresinden alınmıştır.
16. Poggio, J., & Glasnapp, D. (2005). A comparative evaluation of score results from computerized and paper & pencil mathematics testing in a large scale state assessment program. The Journal of Technology, Learning and Assessment, 3(6), 1–30. 08.05.2013 tarihinde http://escholarship.bc.edu/ojs/index.php/jtla/article/view/1659 adresinden alınmıştır.
17. Pomplun, M., Frey, S., & Becker, D. F. (2002). The Score Equivalence of Paper-and-Pencil and Computerized Versions of a Speeded Test of Reading Comprehension. Educational and Psychological Measurement, 62(2), 337–354. doi:10.1177/0013164402062002009
18. Sampson, J.P. (2000). Using the İnternet to enhance testing in counseling. Journal of Counseling and Development
19. Vispoel, W. P., Boo, J., & Bleiler, T. (2001). Computerized and Paper-and-Pencil Versions of the Rosenberg Self-Esteem Scale: A Comparison of Psychometric Features and Respondent Preferences. Educational and Psychological Measurement, 61(3), 461–474. doi:10.1177/00131640121971329
20. Vrabel M. Computerized versus paper-and-pencil testing methods for a nursing certification examination: a review of the literature. Comput Inform Nurs 2004;22:94-8.