This study aimed to reveal the attitudes of the students who were subjected to measurement and evaluation in an online testing environment towards the computer based testing (CBT) platform and the factors that affect their attitudes. It also examined the students’ cognitive loads in the exam designed with multimedia elements from message design principles by using the facilities of this platform. The study was conducted with a case study design. The participants consisted of 32 students. Participants took a previously used and redesigned English achievement test based on the multimedia design principle. Then, cognitive load scale and attitude towards computer scale were implemented. In addition, seven volunteer participants were interviewed. The results of the study showed that factors such as students’ equipment advantage, time, success, flexibility, individuality and less mental effort affected the participants’ attitudes towards the CBT environment. A significant difference was found between the scores of the participants in the cognitive load scale and the duration of the exam. There was a low and positive correlation between the cognitive load score of the participants for the listening test and the time to complete these questions, and a low and negative correlation between the correct answer scores of the participants in the listening test.
This research is based on a M.A. thesis submitted to the Institute of Educational Sciences, Mersin University (Odacı, 2019) which was supported by Mersin University, Scientific Research Project Office, under the project number 2019-2-TP2-3524.
Supporting Institution
Mersin University, Scientific Research Project Office
Project Number
2019-2-TP2-3524
References
Akdemir, Ö. & Oğuz, A. (2008). Computer-based testing: an alternative for the assessment of Turkish undergraduate students, Computers and Education, 1, 1198-1204. https://doi.org/10.1016/j.compedu.2007.11.007
Aybek, E. C. (2012). Kâğıt-kalem formu ve bilgisayar ortamında uygulanan genel yetenek testinin psikometrik özelliklerinin karşılaştırılması [Master’s thesis, Ankara University]. YÖK Açıkbilim. https://acikbilim.yok.gov.tr/handle/20.500.12812/85211
Brünken, R., Seufert, T., & Paas, F. (2010). Measuring Cognitive Load. In J. L. Plass, R. Moreno, & R. Brünken (Eds.), Cognitive Load Theory (pp. 181–202). Cambridge University Press.
Bugbee, A.C. (1996). The Equivalence of Paper-and-Pencil and Computer-Based Testing. Journal of Research on Computing in Education, 28(3), 282. Retrieved September 19, 2023 from https://www.learntechlib.org/p/80961/.
Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö. E., Karadeniz Oran, Ş., & Demirel, F., (2019). Bilimsel Araştırma Yöntemleri. Pegem Akademi.
Çaka, C. & Dursun, Ö.Ö. (2022). Evaluation of the effectiveness of different infographic designs. Journal of Educational Technology & Online Learning, 5(3), 519-534. https://doi.org/10.31681/jetol.1075352
Çetin, İ. (2016). Nitel içerik analizi. In Özden, M. Y. & Durdu, L. (Eds.), Eğitimde Üretim Tabanlı Çalışmalar İçin Nitel Araştırma Yöntemleri (pp. 125-148). Anı Yayıncılık.
Dinar, M., Kabakçı Yurdakul, I. & Dönmez, F. (2015). Measuring Cognitive Load in Test Items: Static Graphics versus Animated Graphics. Journal of Computer Assisted Learning, 31, 148-161. https://doi.org/10.1111/jcal.12086
Fraenkel, J. R., Wallen, N. E., ve Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill.
Hamad, W. (2023). Students’ experiences of learning in virtual classrooms facilitated by Google Classroom. Journal of Educational Technology and Online Learning, 6 (2), 362-383. http://doi.org/10.31681/jetol.1250095
Kirschner, P.A. (2003). Cognitive Load Theory: Implications of Cognitive Load Theory on the Design of Learning, Learning and Instruction, 12, 1-10. https://doi.org/10.1016/S0959-4752(01)00014-7
Kılıç, E. (2009). The effects of cognitive load in learning from goal based scenario designed multimedia learning environment for learners having different working memory capacities (Publication No. 255259) [Doctoral Dissertation, METU]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=NtBAevXNhYaNqJFoAcdBdoczOCd_zbRfD8kVAKPsDubMuN-FCZVksYHFE-wH0v9e
Kılıç, E. ve Karadeniz, Ş. (2004). Hiper ortamlarda öğrencilerin bilişsel yüklenme ve kaybolma düzeylerinin belirlenmesi. Kuram ve Uygulamada Eğitim Yönetimi, 40, 56—579. Retrieved from https://dergipark.org.tr/tr/pub/kuey/issue/10357/126808
Lemmo, A., Mariotti, M-A. (2017, 1-5 February). From paper and pencil to computer-based assessment: some issues raised in the comparison, In. Dooley, T. & Gueudet, G. (Eds.), Proceedings of the Tenth Congress of the European Society for Research in Mathematics Education, DCU Institute of Education & ERME, 3540 – 3547.
Lindner, M. A., Eitel, A., Barenthien, J., & Köller, O. (2018). An integrative study on learning and testing with multimedia: Effects on students’ performance and metacognition. Learning and Instruction, 1(2), 1-9. https://doi.org/10.1016/j.learninstruc.2018.01.002
Mayer, R.E. (2003). The Promise of Multimedia Learning: Using The Same İnstructional Design Methods Across Different Media, Learning and Instruction, 13, 125-139. https://doi.org/10.1016/S0959-4752(02)00016-6
Özturan, T. (2016). The impact of computer assisted assessment on the exam success and attitudes of prospective English teachers. (Publication No. 435226) [ Master’s thesis, Hacettepe University]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=cbOXH84ZayrLjc0tI-QXKtTWLLFqln0MiUoLfO-MxUyK83GNRlTkrzR5Bs7ix3I1
Paas, F. G., & Van Merriënboer, J. J. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 351-371. https://doi.org/10.1007/BF02213420
Paas, F., Renkl, A. & Sweller, J. (2003). Cognitive load theory and instructional design: recent developments, Educational Psychologist, 38(1), 1-4. https://doi.org/10.1207/S15326985EP3801_1
Prisacari, A.A. & Danielson, J. (2017). Computer-based versus paper-based testing: Investigating testing mode with cognitive load and scratch paper use, Computers in Human Behavior, 77, 1-10. https://doi.org/10.1016/j.chb.2017.07.044
Şenel, S. (2018). Programlama Konusu “Kazanımları” ile “Sınıf İçi Ölçme Süreçlerinin” Bilişsel Düzeylerinin Karşılaştırılması. International Journal of Computers in Education, 1 (2), 1-20. Retrieved from https://dergipark.org.tr/en/pub/ijce/issue/42323/486158
Şenel, S., Pekdağ B., & Günaydın, S. (2018). Kimya öğretmenlerinin ölçme ve değerlendirme sürecinde yaşadıkları problemler ve yetersizlikler. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi (EFMED). 12(1). 419-441. https://doi.org/10.17522/balikesirnef.437824
Sweller, J., van Merrienboer, J.J.G. & Paas FGWC. (1998). Cognitive Architecture and Instructional Design, Educational Psychology Review, 10(3), 251-296. https://doi.org/10.1023/A:1022193728205
Takır, A. (2011). The effect of an instruction designed by cognitive load theory principles on 7th grade students' achievement in algebra topics and cognitive load. (Publication No. 255259) [ Doctoral Dissertation, METU]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=EEdeQgIdFRxX5NbvVau-AhjV-QzHr495E11DeqZt2L6gI88s6pwJxJAMh4_IhC9C
Woodside, A. G. (2010). Case Study Research: Theory, Methods, Practice. Emerald Group Pub. Ltd., ebrary Inc.
Yıldırım, A. & Şimşek, H. (2018). Sosyal Bilimlerde Nitel Araştırma Yöntemleri. Seçkin Yayıncılık.
Yıldırım, Z. (2016). Öğretim teknolojileri ve ileti tasarımı. In Çağıltay, K. & Göktaş, Y. (Eds), Öğretim Teknolojilerinin Temelleri: Teoriler, Araştırmalar, Eğilimler (279-296). Pegem Akademi.
Akdemir, Ö. & Oğuz, A. (2008). Computer-based testing: an alternative for the assessment of Turkish undergraduate students, Computers and Education, 1, 1198-1204. https://doi.org/10.1016/j.compedu.2007.11.007
Aybek, E. C. (2012). Kâğıt-kalem formu ve bilgisayar ortamında uygulanan genel yetenek testinin psikometrik özelliklerinin karşılaştırılması [Master’s thesis, Ankara University]. YÖK Açıkbilim. https://acikbilim.yok.gov.tr/handle/20.500.12812/85211
Brünken, R., Seufert, T., & Paas, F. (2010). Measuring Cognitive Load. In J. L. Plass, R. Moreno, & R. Brünken (Eds.), Cognitive Load Theory (pp. 181–202). Cambridge University Press.
Bugbee, A.C. (1996). The Equivalence of Paper-and-Pencil and Computer-Based Testing. Journal of Research on Computing in Education, 28(3), 282. Retrieved September 19, 2023 from https://www.learntechlib.org/p/80961/.
Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö. E., Karadeniz Oran, Ş., & Demirel, F., (2019). Bilimsel Araştırma Yöntemleri. Pegem Akademi.
Çaka, C. & Dursun, Ö.Ö. (2022). Evaluation of the effectiveness of different infographic designs. Journal of Educational Technology & Online Learning, 5(3), 519-534. https://doi.org/10.31681/jetol.1075352
Çetin, İ. (2016). Nitel içerik analizi. In Özden, M. Y. & Durdu, L. (Eds.), Eğitimde Üretim Tabanlı Çalışmalar İçin Nitel Araştırma Yöntemleri (pp. 125-148). Anı Yayıncılık.
Dinar, M., Kabakçı Yurdakul, I. & Dönmez, F. (2015). Measuring Cognitive Load in Test Items: Static Graphics versus Animated Graphics. Journal of Computer Assisted Learning, 31, 148-161. https://doi.org/10.1111/jcal.12086
Fraenkel, J. R., Wallen, N. E., ve Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill.
Hamad, W. (2023). Students’ experiences of learning in virtual classrooms facilitated by Google Classroom. Journal of Educational Technology and Online Learning, 6 (2), 362-383. http://doi.org/10.31681/jetol.1250095
Kirschner, P.A. (2003). Cognitive Load Theory: Implications of Cognitive Load Theory on the Design of Learning, Learning and Instruction, 12, 1-10. https://doi.org/10.1016/S0959-4752(01)00014-7
Kılıç, E. (2009). The effects of cognitive load in learning from goal based scenario designed multimedia learning environment for learners having different working memory capacities (Publication No. 255259) [Doctoral Dissertation, METU]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=NtBAevXNhYaNqJFoAcdBdoczOCd_zbRfD8kVAKPsDubMuN-FCZVksYHFE-wH0v9e
Kılıç, E. ve Karadeniz, Ş. (2004). Hiper ortamlarda öğrencilerin bilişsel yüklenme ve kaybolma düzeylerinin belirlenmesi. Kuram ve Uygulamada Eğitim Yönetimi, 40, 56—579. Retrieved from https://dergipark.org.tr/tr/pub/kuey/issue/10357/126808
Lemmo, A., Mariotti, M-A. (2017, 1-5 February). From paper and pencil to computer-based assessment: some issues raised in the comparison, In. Dooley, T. & Gueudet, G. (Eds.), Proceedings of the Tenth Congress of the European Society for Research in Mathematics Education, DCU Institute of Education & ERME, 3540 – 3547.
Lindner, M. A., Eitel, A., Barenthien, J., & Köller, O. (2018). An integrative study on learning and testing with multimedia: Effects on students’ performance and metacognition. Learning and Instruction, 1(2), 1-9. https://doi.org/10.1016/j.learninstruc.2018.01.002
Mayer, R.E. (2003). The Promise of Multimedia Learning: Using The Same İnstructional Design Methods Across Different Media, Learning and Instruction, 13, 125-139. https://doi.org/10.1016/S0959-4752(02)00016-6
Özturan, T. (2016). The impact of computer assisted assessment on the exam success and attitudes of prospective English teachers. (Publication No. 435226) [ Master’s thesis, Hacettepe University]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=cbOXH84ZayrLjc0tI-QXKtTWLLFqln0MiUoLfO-MxUyK83GNRlTkrzR5Bs7ix3I1
Paas, F. G., & Van Merriënboer, J. J. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 351-371. https://doi.org/10.1007/BF02213420
Paas, F., Renkl, A. & Sweller, J. (2003). Cognitive load theory and instructional design: recent developments, Educational Psychologist, 38(1), 1-4. https://doi.org/10.1207/S15326985EP3801_1
Prisacari, A.A. & Danielson, J. (2017). Computer-based versus paper-based testing: Investigating testing mode with cognitive load and scratch paper use, Computers in Human Behavior, 77, 1-10. https://doi.org/10.1016/j.chb.2017.07.044
Şenel, S. (2018). Programlama Konusu “Kazanımları” ile “Sınıf İçi Ölçme Süreçlerinin” Bilişsel Düzeylerinin Karşılaştırılması. International Journal of Computers in Education, 1 (2), 1-20. Retrieved from https://dergipark.org.tr/en/pub/ijce/issue/42323/486158
Şenel, S., Pekdağ B., & Günaydın, S. (2018). Kimya öğretmenlerinin ölçme ve değerlendirme sürecinde yaşadıkları problemler ve yetersizlikler. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi (EFMED). 12(1). 419-441. https://doi.org/10.17522/balikesirnef.437824
Sweller, J., van Merrienboer, J.J.G. & Paas FGWC. (1998). Cognitive Architecture and Instructional Design, Educational Psychology Review, 10(3), 251-296. https://doi.org/10.1023/A:1022193728205
Takır, A. (2011). The effect of an instruction designed by cognitive load theory principles on 7th grade students' achievement in algebra topics and cognitive load. (Publication No. 255259) [ Doctoral Dissertation, METU]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=EEdeQgIdFRxX5NbvVau-AhjV-QzHr495E11DeqZt2L6gI88s6pwJxJAMh4_IhC9C
Woodside, A. G. (2010). Case Study Research: Theory, Methods, Practice. Emerald Group Pub. Ltd., ebrary Inc.
Yıldırım, A. & Şimşek, H. (2018). Sosyal Bilimlerde Nitel Araştırma Yöntemleri. Seçkin Yayıncılık.
Yıldırım, Z. (2016). Öğretim teknolojileri ve ileti tasarımı. In Çağıltay, K. & Göktaş, Y. (Eds), Öğretim Teknolojilerinin Temelleri: Teoriler, Araştırmalar, Eğilimler (279-296). Pegem Akademi.
Odacı, M. M., & Uzun, E. (2024). Measuring cognitive loads and attitudes of pre-service teachers in computer based testing environment. Journal of Educational Technology and Online Learning, 7(1), 84-101. https://doi.org/10.31681/jetol.1366359