SÜREÇ ODAKLI DEĞERLENDİRME TASARIMININ ANALİTİK TEMELLİ PERFORMANSA GÖRE DEĞERLENDİRMESİ

Year 2022, Volume: 12 Issue: 2, 377 - 411, 30.07.2022

Denizer Yıldırım

https://doi.org/10.17943/etku.1062731

Cited By: 1

Abstract

Bu araştırmanın amacı, COVID sonrası dönemde eğitimde dijital dönüşümün niteliğini artırmaya katkı sağlamak için, değerlendirmenin sürece yayıldığı bir çevrimiçi derste analitik temelli öğrenme performansı farklı olan grupların, ara sınav ve final performansını, e-değerlendirme tasarımı algısını ve genel öğrenme deneyimini incelemektir. Araştırmada öğrenme analitiği süreci yürütülmüş olup, betimleyici analitik yöntemi kullanılmıştır. Bu süreç ara sınava kadar ve finale kadar olan dönemlerde öğrenme performansı ile ilişkilendirilebilecek metriklerinin toplanmasını ve analiz edilmesini içermektedir. Çalışma grubu uzaktan eğitim programlarına kayıtlı olup Bilgi ve İletişim Teknolojileri dersini alan 285 öğrenciden oluşmaktadır. Veriler her konu için ön test, MOODLE içerisinde öğrenci izleme araçları (canlı derse katılım, çevrimiçi çalışma süresi, etkinlik tamamlama yüzdesi, öğrenme kaynaklarına erişim), e-değerlendirme tasarımı algısı ve genel öğrenme deneyimi boyutlarının kullanıldığı e-değerlendirme ölçeği ve çevrimiçi sınav (ara ve final) aracılığıyla toplanmıştır. Analitik temelli öğrenme performansını betimleyebilmek için kümeleme analizi (k-means ve hiyerarşik) kullanılmıştır. Kümelere göre ara sınav ve final performansı, e-değerlendirme tasarımı algısı ve genel öğrenme deneyimleri arasında farklılık t-testi ile analiz edilmiştir. Sonuç olarak, analitikler bakımından yüksek performans gösteren öğrencilerin akademik başarılarının daha yüksek olduğu bulunmuştur. Fakat, kurumların uzaktan eğitime ilişkin yönetmeliklerindeki sınırlılıklar nedeni ile adil bir değerlendirme sürecinin garanti edilemeyeceği tartışılmaktadır. Bu doğrultuda başarı ölçütlerinin daha iyi nasıl belirlenebileceğine odaklanılarak öğrenme performansını daha nitelikli ortaya koyabilecek uygulama örneklerinin çoğaltılması faydalı olabilir.

Keywords

e-değerlendirme, öğrenme performansı, MOODLE analitikleri, kümeleme

EVALUATION OF PROCESS-FOCUSED ASSESSMENT DESIGN ACCORDING TO ANALYTICS-BASED PERFORMANCE

Abstract

This research aims to examine the midterm and final performance, e-assessment design perception, and general learning experiences of learners whose performance is different according to the analytics used in e-assessment. In the research, the learning analytics process was carried out and the descriptive analytics method was used. This process includes the collection and analysis of metrics that can be associated with learning performance in the periods until the midterm and the final. The study group consists of 285 students enrolled in distance education programs and taking the Information and Communication Technology course. Data were collected through pre-test for each subject, student monitoring tools in MOODLE LMS, an online assessment scale, and midterm and final exams (midterm and final). Clustering analysis (k-means and hierarchical) was used to describe learning performance. Differences between academic achievement, e-assessment, and general learning experiences by clusters were analyzed by t-test. As a result, in an e-assessment design like the one in this study, it was found that students with high performance in terms of the variables considered had higher academic achievement. However, it is argued that due to the limitations in the regulations, a fair assessment process cannot be guaranteed. In this respect, it may be beneficial to focus on how to determine the success criteria better and how to increase the implementation examples that can demonstrate the learning performance in a more qualified way.

Keywords

e-assessment, learning performance, MOODLE analytics, clustering analysis

References

• Adedoyin, O. B., ve Soykan, E. (2020). Covid-19 pandemic and online learning: the challenges and opportunities. Interactive Learning Environments, 1-13. https://doi.org/10.1080/10494820.2020.1813180
• Ahmed, F. R. A., Ahmed, T. E., Saeed, R. A., Alhumyani, H., Abdel-Khalek, S., ve Abu-Zinadah, H. (2021). Analysis and challenges of robust E-exams performance under COVID-19. Results in Physics, 23, 103987. https://doi.org/10.1016/j.rinp.2021.103987
• Ahmed, A., Zualkernan, I., ve Elghazaly, H. (2021, July). Unsupervised Clustering of Skills for an Online Learning Platform. In 2021 International Conference on Advanced Learning Technologies (ICALT) (pp. 200-202). IEEE. https://doi.org/10.1109/ICALT52272.2021.00066
• Akçapınar, G., Altun, A., & Aşkar, P. (2016). Çevrimiçi Öğrenme Ortamındaki Benzer Öğrenci gruplarının Kümeleme Yöntemi ile Belirlenmesi. Eğitim Teknolojisi Kuram ve Uygulama, 6(2), 46-64. https://doi.org/10.17943/etku.91440
• Akçapınar, G. ve Bayazıt, A. (2019). MoodleMiner: Moodle Öğrenme Yönetim Sistemi için Veri Madenciliği Analiz Aracı. İlköğretim Online, 18(1). s. 406-415. https://doi.org/10.17051/ilkonline.2019.527645
• Al-Hattami, A. A. (2020). E-Assessment of Students Performance During the E-Teaching and Learning. International Journal of Advanced Science and Technology, ISSN, 4238, 1537-1547.
• Alsadoon, H. (2017). Students' Perceptions of E-Assessment at Saudi Electronic University. Turkish Online Journal of Educational Technology-TOJET, 16(1), 147-153.
• Aranganayagi, S., ve Thangavel, K. (2007, December). Clustering categorical data using silhouette coefficient as a relocating measure. In International conference on computational intelligence and multimedia applications (ICCIMA 2007) (Vol. 2, pp. 13-17). IEEE. https://doi.org/10.1109/ICCIMA.2007.328
• Bayazıt, A., ve Akçapınar, G. (2018). Çevrimiçi dersler için video analitik aracının tasarlanması ve geliştirilmesi. Elementary Education Online, 17(1). http://dergipark.gov.tr/ilkonline/issue/36274/413719
• Bayrak, F., ve Yurdugül, H. (2016). Web-Tabanlı Öz-Değerlendirme Sisteminde Öğrenci Uyarı İndeksini Temel Alan Öğrenme Analitiği Modülünün Tasarlanması. Eğitim Teknolojisi Kuram ve Uygulama, 6(2), 85-99. https://doi.org/10.17943/etku.59549
• Bozkurt, A., ve Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), i-vi.
• Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö., E., Karadeniz, Ş., Demirel, F. (2020). Eğitimde Bilimsel Araştırma Yöntemleri. PEGEM AKADEMİ, ANKARA
• Bravo-Agapito, J., Romero, S. J., ve Pamplona, S. (2021). Early prediction of undergraduate Student's academic performance in completely online learning: A five-year study. Computers in Human Behavior, 115, 106595. https://doi.org/10.1016/j.chb.2020.106595
• Chowdhury, M., Demir, I., Jiang, J., ve Shahzad, N. (2021). Will Highschool Students After the Pandemic want a System of Education to be a Hybrid of Remote Learning and in Person Learning. Across The Spectrum of Socioeconomics, 4, 252. https://doi.org/10.5281/zenodo.4743649
• CINECA elearning Support. (2017). Moodle plugins directory: Attendance Register: Versions. https://moodle.org/plugins/mod_attendanceregister/versions ‘ den Haziran 2022’ de erişildi.
• Doğan, N., Kıbrıslıoğlu Uysal, N., Kelecioğlu, H. ve Hambleton, R. K. (2020). An overview of e-assessment. Hacettepe University Journal of Education, 35 (Special Issue), 1-5. https://doi.org/10.16986/HUJE.2020063669
• Du, X., Yang, J., Shelton, B. E., Hung, J. L., ve Zhang, M. (2021). A systematic meta-review and analysis of learning analytics research. Behaviour & information technology, 40(1), 49-62. https://doi.org/10.1080/0144929X.2019.1669712
• Elzainy, A., El Sadik, A., ve Al Abdulmonem, W. (2020). Experience of e-learning and online assessment during the COVID-19 pandemic at the College of Medicine, Qassim University. Journal of Taibah University Medical Sciences, 15(6), 456-462. https://doi.org/10.1016/j.jtumed.2020.09.005
• Figaredo, D. D., Jaurena, I. G., ve Encina, J. M. (2022). The Impact of Rapid Adoption of Online Assessment on Students’ Performance and Perceptions: Evidence from a Distance Learning University. Electronic Journal of e-Learning, 20(3), pp224-241. https://doi.org/10.34190/ejel.20.3.2399
• Ferri, F., Grifoni, P., ve Guzzo, T. (2020). Online learning and emergency remote teaching: Opportunities and challenges in emergency situations. Societies, 10(4), 86. https://doi.org/10.3390/soc10040086
• Foerster, M. (2019). Framework for the quality assurance of e-assessment. Voced.edu.au; ENQA. https://www.voced.edu.au/content/ngv:84627
• Glassey, J., ve Abegão, F. R. (2017, November). E-assessment and tailored feedback-are they contributing to the effectiveness of chemical engineering education?. In 2017 7th World Engineering Education Forum (WEEF) (pp. 508-512). IEEE. https://doi.org/10.1109/WEEF.2017.8467054
• Guerrero-Roldán, A. E., ve Noguera, I. (2018). A model for aligning assessment with competences and learning activities in online courses. The Internet and Higher Education, 38, 36-46. https://doi.org/10.1016/j.iheduc.2018.04.005
• Han, P., Wang, W., Shi, Q., ve Yue, J. (2021). A combined online-learning model with K-means clustering and GRU neural networks for trajectory prediction. Ad Hoc Networks, 117, 102476. https://doi.org/10.1016/j.adhoc.2021.102476
• Harlen, W., ve James, M. (1997). Assessment and learning: differences and relationships between formative and summative assessment. Assessment in Education: Principles, Policy & Practice, 4(3), 365-379. https://doi.org/10.1080/0969594970040304
• Hewson, C., ve Charlton, J. P. (2019). An investigation of the validity of course‐based online assessment methods: The role of computer‐related attitudes and assessment mode preferences. Journal of Computer Assisted Learning, 35(1), 51-60. https://doi.org/10.1111/jcal.12310
• Hodges, C., Moore, S., Lockee, B., Trust, T., ve Bond, A. (2020). The difference between emergency remote teaching and online learning. Educause Review, 27, 1-12.
• Holmes, W., Nguyen, Q., Zhang, J., Mavrikis, M., ve Rienties, B. (2019). Learning analytics for learning design in online distance learning. Distance Education, 40(3), 309-329. https://doi.org/10.1080/01587919.2019.1637716
• Ilgaz, H., ve Adanır, G. A. (2020). Providing online exams for online learners: Does it really matter for them?. Education and Information Technologies, 25(2), 1255-1269. https://doi.org/10.1007/s10639-019-10020-6
• Kearns, L. R. (2012). Student assessment in online learning: Challenges and effective practices. Journal of Online Learning and Teaching, 8(3), 198.
• Kişisel Verilerin Korunması Kanunu. (2016, 7 Nisan). Resmi Gazete (Sayı: 29677). Erişim adresi:https://www.mevzuat.gov.tr/mevzuat?MevzuatNo=6698&MevzuatTur=1&MevzuatTertip=5
• Knight, S., Shum, S. B., ve Littleton, K. (2014). Epistemology, assessment, pedagogy: where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2), 23-47. https://doi.org/10.18608/jla.2014.12.3
• Laurillard, D. (2013). Teaching as a design science: Building pedagogical patterns for learning and technology. Routledge. https://doi.org/10.4324/9780203125083
• Marín, V. I., ve Garcias, A. P. (2016). Collaborative e-Assessment as a Strategy for Scaffolding Self-Regulated Learning in Higher Education. In Formative Assessment, Learning Data Analytics and Gamification (pp. 3-24). Academic Press. https://doi.org/10.1016/B978-0-12-803637-2.00001-4
• Martin, F., & Ndoye, A. (2016). Using Learning Analytics to Assess Student Learning in Online Courses. Journal of University Teaching & Learning Practice, 13(3). https://doi.org/10.53761/1.13.3.7
• Mayer, R. E. (2019). Thirty years of research on online learning. Applied Cognitive Psychology, 33(2), 152-159. https://doi.org/10.1002/acp.3482
• Mellar, H., Peytcheva-Forsyth, R., Kocdar, S., Karadeniz, A., ve Yovkova, B. (2018). Addressing cheating in e-assessment using student authentication and authorship checking systems: teachers’ perspectives. International Journal for Educational Integrity, 14(1), 1-21. https://doi.org/10.1007/s40979-018-0025-x
• Moodle Community. (2019a). Course reports - MoodleDocs. https://docs.moodle.org/311/en/Course_reports ‘ dan Haziran 2022’de erişildi.
• Moodle Community. (2019b). Grading quick guide - MoodleDocs. https://docs.moodle.org/311/en/Grading_quick_guide ‘ dan Haziran 2022’de erişildi.
• Moodle Community. (2021a). Feedback activity - MoodleDocs. https://docs.moodle.org/311/en/Feedback_activity ‘ den Haziran 2022’de erişildi.
• Moodle Community. (2021b). Quiz activity - MoodleDocs. https://docs.moodle.org/311/en/Quiz_activity ‘ den Haziran 2022’de erişildi.
• Moodle Community. (2021c). Using Activity completion - MoodleDocs. https://docs.moodle.org/311/en/Using_Activity_completion ‘ den Haziran 2022’de erişildi.
• Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and higher Education, 31(1), 53-64. https://doi.org/10.1080/03098770601167922
• Ogange, B. O., Agak, J. O., Okelo, K. O., ve Kiprotich, P. (2018). Student perceptions of the effectiveness of formative assessment in an online learning environment. Open Praxis, 10(1), 29-39. https://doi.org/10.5944/openpraxis.10.1.705
• Peytcheva-Forsyth, R., ve Aleksieva, L. (2021, March). Forced introduction of e-assessment during COVID-19 pandemic: How did the students feel about that? (Sofia University case). In AIP Conference Proceedings (Vol. 2333, No. 1, p. 050013). AIP Publishing LLC. https://doi.org/10.1063/5.0041862
• Pishchukhina, O., ve Allen, A. (2021, September). Supporting learning in large classes: online formative assessment and automated feedback. In 2021 30th Annual Conference of the European Association for Education in Electrical and Information Engineering (EAEEIE) (pp. 1-4). IEEE. https://doi.org/10.1063/5.0041862
• Rajabalee, Y. B., Santally, M. I., ve Rennie, F. (2019). The use of learning analytics to improve online learning outcomes: A systematic literature review [Working paper]. Ninth Pan-Commonwealth Forum, Edinburgh, UK. http://hdl.handle.net/11599/3275
• Reeves, T. C. (2000). Alternative assessment approaches for online learning environments in higher education. Journal of Educational Computing Research, 23(1), 101-111. https://doi.org/10.2190%2FGYMQ-78FA-WMTX-J06C
• Rienties, Bart and Jones, Ann (2019). Evidence -Based Learning: Futures. Using learning design and learning analytics to empower teachers to meet students’ diverse needs. In: Ferguson, Rebecca; Jones, Ann and Scanlon, Eileen eds. Educational Visions: The lessons from 40 years of innovation. London: Ubiquity Press, pp. 109–125.
• Rolim, C., ve Isaias, P. (2019). Examining the use of e‐assessment in higher education: teachers and students’ viewpoints. British Journal of Educational Technology, 50(4), 1785-1800. https://doi.org/10.1111/bjet.12669
• Sa'di, R., Abdelraziq, A., ve Sharadgah, T. (2021). E-Assessment at Jordan’s Universities in the Time of the COVID-19 Lockdown: Challenges and Solutions. Arab World English Journal (AWEJ) Special Issue on Covid, 19. https://dx.doi.org/10.24093/awej/covid.3
• Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144. https://doi.org/10.1007/BF00117714
• Sandoval, A., Gonzalez, C., Alarcon, R., Pichara, K., ve Montenegro, M. (2018). Centralized student performance prediction in large courses based on low-cost variables in an institutional context. The Internet and Higher Education, 37, 76-89. https://doi.org/10.1016/j.iheduc.2018.02.002
• Saykili, A., Ozturk, A., Kumtepe, E. G., Kumtepe, A. T., ve Uğurhan, Y. Z. C. Examining the Effects of LMS Use on Academic Performance Using Learning Analytics. Hosted by UNED, Madrid (Spain), 148.
• Sharadgah, T., ve Sa'di, R. (2020). Preparedness of institutions of higher education for assessment in virtual learning environments during the Covid-19 lockdown: Evidence of bona fide challenges and pragmatic solutions. Journal of Information Technology Education: Research, 19(1), 755-774. Doi: 10.28945/4615
• Shavelson, R. J., Zlatkin-Troitschanskaia, O., ve Mariño, J. P. (2018). International performance assessment of learning in higher education (iPAL): Research and development. In Assessment of learning outcomes in higher education (pp. 193-214). Springer, Cham. https://doi.org/10.1007/978-3-319-74338-7_10
• Soffer, T., Kahan, T., ve Livne, E. (2017). E-assessment of online academic courses via students' activities and perceptions. Studies in Educational Evaluation, 54, 83–93. https://doi.org/10.1016/j.stueduc.2016.10.001
• Soffer, T., ve Nachmias, R. (2018). Effectiveness of learning in online academic courses compared with face‐to‐face courses in higher education. Journal of Computer Assisted Learning, 34(5), 534-543. https://doi.org/10.1111/jcal.12258
• Soffer, T., Kahan, T., & Nachmias, R. (2019). Patterns of students’ utilization of flexibility in online academic courses and their relation to course achievement. International Review of Research in Open and Distributed Learning, 20(3). https://doi.org/10.19173/irrodl.v20i4.3949
• Stödberg, U. (2012). A research review of e-assessment. Assessment & Evaluation in Higher Education, 37(5), 591-604. https://doi.org/10.1080/02602938.2011.557496
• Teaching with Blackboard. (2019). Accessing the Attendance Report from a Blackboard Collaborate Session [YouTube Video]. In YouTube. https://www.youtube.com/watch?v=VdFn-NnwZP4
• Tempelaar, D. (2020). Supporting the less-adaptive student: the role of learning analytics, formative assessment and blended learning. Assessment & Evaluation in Higher Education, 45(4), 579-593. https://doi.org/10.1080/02602938.2019.1677855
• Tracking progress - MoodleDocs. (2020). Retrieved January 1, 2022, from Moodle.org website: https://docs.moodle.org/311/en/Tracking_progress
• University of the People. (2020, May 10). Emergency Remote Teaching Vs. Online Learning: A Comparison. https://www.uopeople.edu/blog/emergency-remote-teaching-vs-online-learning/ ’ den Haziran 2022’de erişildi.
• Xiong, Y., ve Suen, H. K. (2018). Assessment approaches in massive open online courses: Possibilities, challenges and future directions. International Review of Education, 64(2), 241-263. https://doi.org/10.1007/s11159-018-9710-5
• Yildirim, D., Gülbahar, Y. (2022). Implementation of Learning Analytics Indicators for Increasing Learners' Final Performance. Technology Knowledge and Learning. 27. https://doi.org/10.1007/s10758-021-09583-6
• Yildirim, D., ve Seferoğlu, S. S. (2021). Evaluation of the effectiveness of online courses based on the community of inquiry model. Turkish Online Journal of Distance Education, 22(2), 147-163., https://doi.org/10.17718/tojde.906834
• Yildirim, D., ve Usluel, Y. (2022). Interrelated analysis of interaction, sequential patterns and academic achievement in online learning. Australasian Journal of Educational Technology, 38(2), 181–200. https://doi.org/10.14742/ajet.7360
• Yıldız, G., ve Çakmak, E. K. (2019). Zenginleştirilmiş E-Değerlendirme Sisteminin Ders Başarısına Etkisi ve Öğrenci Memnuniyetinin İncelenmesi. Gazi Eğitim Bilimleri Dergisi, 5, 106-139. https://dergipark.org.tr/en/pub/gebd/issue/49407/588868’ den Haziran 2022’ de erişildi.
• Yükseköğretim Kurulu (2020). Yükseköğretim Kurumlarında Uzaktan Öğretime İlişkin Usul Ve Esaslar, https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Uzaktan_ogretim/yuksekogretim_kurumlarinda_uzaktan_ogretime_iliskin_usul_ve_esaslar.pdf ‘den Haziran 2022’de erişildi.