SÜREÇ ODAKLI DEĞERLENDİRME TASARIMININ ANALİTİK TEMELLİ PERFORMANSA GÖRE DEĞERLENDİRMESİ

Öz

Bu araştırmanın amacı, COVID sonrası dönemde eğitimde dijital dönüşümün niteliğini artırmaya katkı sağlamak için, değerlendirmenin sürece yayıldığı bir çevrimiçi derste analitik temelli öğrenme performansı farklı olan grupların, ara sınav ve final performansını, e-değerlendirme tasarımı algısını ve genel öğrenme deneyimini incelemektir. Araştırmada öğrenme analitiği süreci yürütülmüş olup, betimleyici analitik yöntemi kullanılmıştır. Bu süreç ara sınava kadar ve finale kadar olan dönemlerde öğrenme performansı ile ilişkilendirilebilecek metriklerinin toplanmasını ve analiz edilmesini içermektedir. Çalışma grubu uzaktan eğitim programlarına kayıtlı olup Bilgi ve İletişim Teknolojileri dersini alan 285 öğrenciden oluşmaktadır. Veriler her konu için ön test, MOODLE içerisinde öğrenci izleme araçları (canlı derse katılım, çevrimiçi çalışma süresi, etkinlik tamamlama yüzdesi, öğrenme kaynaklarına erişim), e-değerlendirme tasarımı algısı ve genel öğrenme deneyimi boyutlarının kullanıldığı e-değerlendirme ölçeği ve çevrimiçi sınav (ara ve final) aracılığıyla toplanmıştır. Analitik temelli öğrenme performansını betimleyebilmek için kümeleme analizi (k-means ve hiyerarşik) kullanılmıştır. Kümelere göre ara sınav ve final performansı, e-değerlendirme tasarımı algısı ve genel öğrenme deneyimleri arasında farklılık t-testi ile analiz edilmiştir. Sonuç olarak, analitikler bakımından yüksek performans gösteren öğrencilerin akademik başarılarının daha yüksek olduğu bulunmuştur. Fakat, kurumların uzaktan eğitime ilişkin yönetmeliklerindeki sınırlılıklar nedeni ile adil bir değerlendirme sürecinin garanti edilemeyeceği tartışılmaktadır. Bu doğrultuda başarı ölçütlerinin daha iyi nasıl belirlenebileceğine odaklanılarak öğrenme performansını daha nitelikli ortaya koyabilecek uygulama örneklerinin çoğaltılması faydalı olabilir.

Anahtar Kelimeler

• e-değerlendirme
• öğrenme performansı
• MOODLE analitikleri
• kümeleme

EVALUATION OF PROCESS-FOCUSED ASSESSMENT DESIGN ACCORDING TO ANALYTICS-BASED PERFORMANCE

Öz

This research aims to examine the midterm and final performance, e-assessment design perception, and general learning experiences of learners whose performance is different according to the analytics used in e-assessment. In the research, the learning analytics process was carried out and the descriptive analytics method was used. This process includes the collection and analysis of metrics that can be associated with learning performance in the periods until the midterm and the final. The study group consists of 285 students enrolled in distance education programs and taking the Information and Communication Technology course. Data were collected through pre-test for each subject, student monitoring tools in MOODLE LMS, an online assessment scale, and midterm and final exams (midterm and final). Clustering analysis (k-means and hierarchical) was used to describe learning performance. Differences between academic achievement, e-assessment, and general learning experiences by clusters were analyzed by t-test. As a result, in an e-assessment design like the one in this study, it was found that students with high performance in terms of the variables considered had higher academic achievement. However, it is argued that due to the limitations in the regulations, a fair assessment process cannot be guaranteed. In this respect, it may be beneficial to focus on how to determine the success criteria better and how to increase the implementation examples that can demonstrate the learning performance in a more qualified way.

Anahtar Kelimeler

• e-assessment
• learning performance
• MOODLE analytics
• clustering analysis

Kaynakça

1. Adedoyin, O. B., ve Soykan, E. (2020). Covid-19 pandemic and online learning: the challenges and opportunities. Interactive Learning Environments, 1-13. https://doi.org/10.1080/10494820.2020.1813180
2. Ahmed, F. R. A., Ahmed, T. E., Saeed, R. A., Alhumyani, H., Abdel-Khalek, S., ve Abu-Zinadah, H. (2021). Analysis and challenges of robust E-exams performance under COVID-19. Results in Physics, 23, 103987. https://doi.org/10.1016/j.rinp.2021.103987
3. Ahmed, A., Zualkernan, I., ve Elghazaly, H. (2021, July). Unsupervised Clustering of Skills for an Online Learning Platform. In 2021 International Conference on Advanced Learning Technologies (ICALT) (pp. 200-202). IEEE. https://doi.org/10.1109/ICALT52272.2021.00066
4. Akçapınar, G., Altun, A., & Aşkar, P. (2016). Çevrimiçi Öğrenme Ortamındaki Benzer Öğrenci gruplarının Kümeleme Yöntemi ile Belirlenmesi. Eğitim Teknolojisi Kuram ve Uygulama, 6(2), 46-64. https://doi.org/10.17943/etku.91440
5. Akçapınar, G. ve Bayazıt, A. (2019). MoodleMiner: Moodle Öğrenme Yönetim Sistemi için Veri Madenciliği Analiz Aracı. İlköğretim Online, 18(1). s. 406-415. https://doi.org/10.17051/ilkonline.2019.527645
6. Al-Hattami, A. A. (2020). E-Assessment of Students Performance During the E-Teaching and Learning. International Journal of Advanced Science and Technology, ISSN, 4238, 1537-1547.
7. Alsadoon, H. (2017). Students' Perceptions of E-Assessment at Saudi Electronic University. Turkish Online Journal of Educational Technology-TOJET, 16(1), 147-153.
8. Aranganayagi, S., ve Thangavel, K. (2007, December). Clustering categorical data using silhouette coefficient as a relocating measure. In International conference on computational intelligence and multimedia applications (ICCIMA 2007) (Vol. 2, pp. 13-17). IEEE. https://doi.org/10.1109/ICCIMA.2007.328

9. Bayazıt, A., ve Akçapınar, G. (2018). Çevrimiçi dersler için video analitik aracının tasarlanması ve geliştirilmesi. Elementary Education Online, 17(1). http://dergipark.gov.tr/ilkonline/issue/36274/413719
10. Bayrak, F., ve Yurdugül, H. (2016). Web-Tabanlı Öz-Değerlendirme Sisteminde Öğrenci Uyarı İndeksini Temel Alan Öğrenme Analitiği Modülünün Tasarlanması. Eğitim Teknolojisi Kuram ve Uygulama, 6(2), 85-99. https://doi.org/10.17943/etku.59549
11. Bozkurt, A., ve Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), i-vi.
12. Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö., E., Karadeniz, Ş., Demirel, F. (2020). Eğitimde Bilimsel Araştırma Yöntemleri. PEGEM AKADEMİ, ANKARA
13. Bravo-Agapito, J., Romero, S. J., ve Pamplona, S. (2021). Early prediction of undergraduate Student's academic performance in completely online learning: A five-year study. Computers in Human Behavior, 115, 106595. https://doi.org/10.1016/j.chb.2020.106595
14. Chowdhury, M., Demir, I., Jiang, J., ve Shahzad, N. (2021). Will Highschool Students After the Pandemic want a System of Education to be a Hybrid of Remote Learning and in Person Learning. Across The Spectrum of Socioeconomics, 4, 252. https://doi.org/10.5281/zenodo.4743649
15. CINECA elearning Support. (2017). Moodle plugins directory: Attendance Register: Versions. https://moodle.org/plugins/mod_attendanceregister/versions ‘ den Haziran 2022’ de erişildi.
16. Doğan, N., Kıbrıslıoğlu Uysal, N., Kelecioğlu, H. ve Hambleton, R. K. (2020). An overview of e-assessment. Hacettepe University Journal of Education, 35 (Special Issue), 1-5. https://doi.org/10.16986/HUJE.2020063669
17. Du, X., Yang, J., Shelton, B. E., Hung, J. L., ve Zhang, M. (2021). A systematic meta-review and analysis of learning analytics research. Behaviour & information technology, 40(1), 49-62. https://doi.org/10.1080/0144929X.2019.1669712
18. Elzainy, A., El Sadik, A., ve Al Abdulmonem, W. (2020). Experience of e-learning and online assessment during the COVID-19 pandemic at the College of Medicine, Qassim University. Journal of Taibah University Medical Sciences, 15(6), 456-462. https://doi.org/10.1016/j.jtumed.2020.09.005
19. Figaredo, D. D., Jaurena, I. G., ve Encina, J. M. (2022). The Impact of Rapid Adoption of Online Assessment on Students’ Performance and Perceptions: Evidence from a Distance Learning University. Electronic Journal of e-Learning, 20(3), pp224-241. https://doi.org/10.34190/ejel.20.3.2399
20. Ferri, F., Grifoni, P., ve Guzzo, T. (2020). Online learning and emergency remote teaching: Opportunities and challenges in emergency situations. Societies, 10(4), 86. https://doi.org/10.3390/soc10040086
21. Foerster, M. (2019). Framework for the quality assurance of e-assessment. Voced.edu.au; ENQA. https://www.voced.edu.au/content/ngv:84627
22. Glassey, J., ve Abegão, F. R. (2017, November). E-assessment and tailored feedback-are they contributing to the effectiveness of chemical engineering education?. In 2017 7th World Engineering Education Forum (WEEF) (pp. 508-512). IEEE. https://doi.org/10.1109/WEEF.2017.8467054
23. Guerrero-Roldán, A. E., ve Noguera, I. (2018). A model for aligning assessment with competences and learning activities in online courses. The Internet and Higher Education, 38, 36-46. https://doi.org/10.1016/j.iheduc.2018.04.005
24. Han, P., Wang, W., Shi, Q., ve Yue, J. (2021). A combined online-learning model with K-means clustering and GRU neural networks for trajectory prediction. Ad Hoc Networks, 117, 102476. https://doi.org/10.1016/j.adhoc.2021.102476
25. Harlen, W., ve James, M. (1997). Assessment and learning: differences and relationships between formative and summative assessment. Assessment in Education: Principles, Policy & Practice, 4(3), 365-379. https://doi.org/10.1080/0969594970040304
26. Hewson, C., ve Charlton, J. P. (2019). An investigation of the validity of course‐based online assessment methods: The role of computer‐related attitudes and assessment mode preferences. Journal of Computer Assisted Learning, 35(1), 51-60. https://doi.org/10.1111/jcal.12310
27. Hodges, C., Moore, S., Lockee, B., Trust, T., ve Bond, A. (2020). The difference between emergency remote teaching and online learning. Educause Review, 27, 1-12.
28. Holmes, W., Nguyen, Q., Zhang, J., Mavrikis, M., ve Rienties, B. (2019). Learning analytics for learning design in online distance learning. Distance Education, 40(3), 309-329. https://doi.org/10.1080/01587919.2019.1637716
29. Ilgaz, H., ve Adanır, G. A. (2020). Providing online exams for online learners: Does it really matter for them?. Education and Information Technologies, 25(2), 1255-1269. https://doi.org/10.1007/s10639-019-10020-6
30. Kearns, L. R. (2012). Student assessment in online learning: Challenges and effective practices. Journal of Online Learning and Teaching, 8(3), 198.
31. Kişisel Verilerin Korunması Kanunu. (2016, 7 Nisan). Resmi Gazete (Sayı: 29677). Erişim adresi:https://www.mevzuat.gov.tr/mevzuat?MevzuatNo=6698&MevzuatTur=1&MevzuatTertip=5
32. Knight, S., Shum, S. B., ve Littleton, K. (2014). Epistemology, assessment, pedagogy: where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2), 23-47. https://doi.org/10.18608/jla.2014.12.3
33. Laurillard, D. (2013). Teaching as a design science: Building pedagogical patterns for learning and technology. Routledge. https://doi.org/10.4324/9780203125083
34. Marín, V. I., ve Garcias, A. P. (2016). Collaborative e-Assessment as a Strategy for Scaffolding Self-Regulated Learning in Higher Education. In Formative Assessment, Learning Data Analytics and Gamification (pp. 3-24). Academic Press. https://doi.org/10.1016/B978-0-12-803637-2.00001-4
35. Martin, F., & Ndoye, A. (2016). Using Learning Analytics to Assess Student Learning in Online Courses. Journal of University Teaching & Learning Practice, 13(3). https://doi.org/10.53761/1.13.3.7
36. Mayer, R. E. (2019). Thirty years of research on online learning. Applied Cognitive Psychology, 33(2), 152-159. https://doi.org/10.1002/acp.3482
37. Mellar, H., Peytcheva-Forsyth, R., Kocdar, S., Karadeniz, A., ve Yovkova, B. (2018). Addressing cheating in e-assessment using student authentication and authorship checking systems: teachers’ perspectives. International Journal for Educational Integrity, 14(1), 1-21. https://doi.org/10.1007/s40979-018-0025-x
38. Moodle Community. (2019a). Course reports - MoodleDocs. https://docs.moodle.org/311/en/Course_reports ‘ dan Haziran 2022’de erişildi.
39. Moodle Community. (2019b). Grading quick guide - MoodleDocs. https://docs.moodle.org/311/en/Grading_quick_guide ‘ dan Haziran 2022’de erişildi.
40. Moodle Community. (2021a). Feedback activity - MoodleDocs. https://docs.moodle.org/311/en/Feedback_activity ‘ den Haziran 2022’de erişildi.
41. Moodle Community. (2021b). Quiz activity - MoodleDocs. https://docs.moodle.org/311/en/Quiz_activity ‘ den Haziran 2022’de erişildi.
42. Moodle Community. (2021c). Using Activity completion - MoodleDocs. https://docs.moodle.org/311/en/Using_Activity_completion ‘ den Haziran 2022’de erişildi.
43. Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and higher Education, 31(1), 53-64. https://doi.org/10.1080/03098770601167922
44. Ogange, B. O., Agak, J. O., Okelo, K. O., ve Kiprotich, P. (2018). Student perceptions of the effectiveness of formative assessment in an online learning environment. Open Praxis, 10(1), 29-39. https://doi.org/10.5944/openpraxis.10.1.705
45. Peytcheva-Forsyth, R., ve Aleksieva, L. (2021, March). Forced introduction of e-assessment during COVID-19 pandemic: How did the students feel about that? (Sofia University case). In AIP Conference Proceedings (Vol. 2333, No. 1, p. 050013). AIP Publishing LLC. https://doi.org/10.1063/5.0041862
46. Pishchukhina, O., ve Allen, A. (2021, September). Supporting learning in large classes: online formative assessment and automated feedback. In 2021 30th Annual Conference of the European Association for Education in Electrical and Information Engineering (EAEEIE) (pp. 1-4). IEEE. https://doi.org/10.1063/5.0041862
47. Rajabalee, Y. B., Santally, M. I., ve Rennie, F. (2019). The use of learning analytics to improve online learning outcomes: A systematic literature review [Working paper]. Ninth Pan-Commonwealth Forum, Edinburgh, UK. http://hdl.handle.net/11599/3275
48. Reeves, T. C. (2000). Alternative assessment approaches for online learning environments in higher education. Journal of Educational Computing Research, 23(1), 101-111. https://doi.org/10.2190%2FGYMQ-78FA-WMTX-J06C
49. Rienties, Bart and Jones, Ann (2019). Evidence -Based Learning: Futures. Using learning design and learning analytics to empower teachers to meet students’ diverse needs. In: Ferguson, Rebecca; Jones, Ann and Scanlon, Eileen eds. Educational Visions: The lessons from 40 years of innovation. London: Ubiquity Press, pp. 109–125.
50. Rolim, C., ve Isaias, P. (2019). Examining the use of e‐assessment in higher education: teachers and students’ viewpoints. British Journal of Educational Technology, 50(4), 1785-1800. https://doi.org/10.1111/bjet.12669
51. Sa'di, R., Abdelraziq, A., ve Sharadgah, T. (2021). E-Assessment at Jordan’s Universities in the Time of the COVID-19 Lockdown: Challenges and Solutions. Arab World English Journal (AWEJ) Special Issue on Covid, 19. https://dx.doi.org/10.24093/awej/covid.3
52. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144. https://doi.org/10.1007/BF00117714
53. Sandoval, A., Gonzalez, C., Alarcon, R., Pichara, K., ve Montenegro, M. (2018). Centralized student performance prediction in large courses based on low-cost variables in an institutional context. The Internet and Higher Education, 37, 76-89. https://doi.org/10.1016/j.iheduc.2018.02.002
54. Saykili, A., Ozturk, A., Kumtepe, E. G., Kumtepe, A. T., ve Uğurhan, Y. Z. C. Examining the Effects of LMS Use on Academic Performance Using Learning Analytics. Hosted by UNED, Madrid (Spain), 148.
55. Sharadgah, T., ve Sa'di, R. (2020). Preparedness of institutions of higher education for assessment in virtual learning environments during the Covid-19 lockdown: Evidence of bona fide challenges and pragmatic solutions. Journal of Information Technology Education: Research, 19(1), 755-774. Doi: 10.28945/4615
56. Shavelson, R. J., Zlatkin-Troitschanskaia, O., ve Mariño, J. P. (2018). International performance assessment of learning in higher education (iPAL): Research and development. In Assessment of learning outcomes in higher education (pp. 193-214). Springer, Cham. https://doi.org/10.1007/978-3-319-74338-7_10
57. Soffer, T., Kahan, T., ve Livne, E. (2017). E-assessment of online academic courses via students' activities and perceptions. Studies in Educational Evaluation, 54, 83–93. https://doi.org/10.1016/j.stueduc.2016.10.001
58. Soffer, T., ve Nachmias, R. (2018). Effectiveness of learning in online academic courses compared with face‐to‐face courses in higher education. Journal of Computer Assisted Learning, 34(5), 534-543. https://doi.org/10.1111/jcal.12258
59. Soffer, T., Kahan, T., & Nachmias, R. (2019). Patterns of students’ utilization of flexibility in online academic courses and their relation to course achievement. International Review of Research in Open and Distributed Learning, 20(3). https://doi.org/10.19173/irrodl.v20i4.3949
60. Stödberg, U. (2012). A research review of e-assessment. Assessment & Evaluation in Higher Education, 37(5), 591-604. https://doi.org/10.1080/02602938.2011.557496
61. Teaching with Blackboard. (2019). Accessing the Attendance Report from a Blackboard Collaborate Session [YouTube Video]. In YouTube. https://www.youtube.com/watch?v=VdFn-NnwZP4
62. Tempelaar, D. (2020). Supporting the less-adaptive student: the role of learning analytics, formative assessment and blended learning. Assessment & Evaluation in Higher Education, 45(4), 579-593. https://doi.org/10.1080/02602938.2019.1677855
63. Tracking progress - MoodleDocs. (2020). Retrieved January 1, 2022, from Moodle.org website: https://docs.moodle.org/311/en/Tracking_progress
64. University of the People. (2020, May 10). Emergency Remote Teaching Vs. Online Learning: A Comparison. https://www.uopeople.edu/blog/emergency-remote-teaching-vs-online-learning/ ’ den Haziran 2022’de erişildi.
65. Xiong, Y., ve Suen, H. K. (2018). Assessment approaches in massive open online courses: Possibilities, challenges and future directions. International Review of Education, 64(2), 241-263. https://doi.org/10.1007/s11159-018-9710-5
66. Yildirim, D., Gülbahar, Y. (2022). Implementation of Learning Analytics Indicators for Increasing Learners' Final Performance. Technology Knowledge and Learning. 27. https://doi.org/10.1007/s10758-021-09583-6
67. Yildirim, D., ve Seferoğlu, S. S. (2021). Evaluation of the effectiveness of online courses based on the community of inquiry model. Turkish Online Journal of Distance Education, 22(2), 147-163., https://doi.org/10.17718/tojde.906834
68. Yildirim, D., ve Usluel, Y. (2022). Interrelated analysis of interaction, sequential patterns and academic achievement in online learning. Australasian Journal of Educational Technology, 38(2), 181–200. https://doi.org/10.14742/ajet.7360
69. Yıldız, G., ve Çakmak, E. K. (2019). Zenginleştirilmiş E-Değerlendirme Sisteminin Ders Başarısına Etkisi ve Öğrenci Memnuniyetinin İncelenmesi. Gazi Eğitim Bilimleri Dergisi, 5, 106-139. https://dergipark.org.tr/en/pub/gebd/issue/49407/588868’ den Haziran 2022’ de erişildi.
70. Yükseköğretim Kurulu (2020). Yükseköğretim Kurumlarında Uzaktan Öğretime İlişkin Usul Ve Esaslar, https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Uzaktan_ogretim/yuksekogretim_kurumlarinda_uzaktan_ogretime_iliskin_usul_ve_esaslar.pdf ‘den Haziran 2022’de erişildi.