TY  - JOUR
T1  - SÜREÇ ODAKLI DEĞERLENDİRME TASARIMININ ANALİTİK TEMELLİ PERFORMANSA GÖRE DEĞERLENDİRMESİ
TT  - EVALUATION OF PROCESS-FOCUSED ASSESSMENT DESIGN ACCORDING TO ANALYTICS-BASED PERFORMANCE
AU  - Yıldırım, Denizer
PY  - 2022
DA  - July
DO  - 10.17943/etku.1062731
JF  - Eğitim Teknolojisi Kuram ve Uygulama
JO  - Eğitim Teknolojisi Kuram ve Uygulama (ETKU)
PB  - Tolga GÜYER
WT  - DergiPark
SN  - 2147-1908
SP  - 377
EP  - 411
VL  - 12
IS  - 2
LA  - tr
AB  - Bu araştırmanın amacı, COVID sonrası dönemde eğitimde dijital dönüşümün niteliğini artırmaya katkı sağlamak için, değerlendirmenin sürece yayıldığı bir çevrimiçi derste analitik temelli öğrenme performansı farklı olan grupların, ara sınav ve final performansını, e-değerlendirme tasarımı algısını ve genel öğrenme deneyimini incelemektir. Araştırmada öğrenme analitiği süreci yürütülmüş olup, betimleyici analitik yöntemi kullanılmıştır. Bu süreç ara sınava kadar ve finale kadar olan dönemlerde öğrenme performansı ile ilişkilendirilebilecek metriklerinin toplanmasını ve analiz edilmesini içermektedir. Çalışma grubu uzaktan eğitim programlarına kayıtlı olup Bilgi ve İletişim Teknolojileri dersini alan 285 öğrenciden oluşmaktadır. Veriler her konu için ön test, MOODLE içerisinde öğrenci izleme araçları (canlı derse katılım, çevrimiçi çalışma süresi, etkinlik tamamlama yüzdesi, öğrenme kaynaklarına erişim), e-değerlendirme tasarımı algısı ve genel öğrenme deneyimi boyutlarının kullanıldığı e-değerlendirme ölçeği ve çevrimiçi sınav (ara ve final) aracılığıyla toplanmıştır. Analitik temelli öğrenme performansını betimleyebilmek için kümeleme analizi (k-means ve hiyerarşik) kullanılmıştır. Kümelere göre ara sınav ve final performansı, e-değerlendirme tasarımı algısı ve genel öğrenme deneyimleri arasında farklılık t-testi ile analiz edilmiştir. Sonuç olarak, analitikler bakımından yüksek performans gösteren öğrencilerin akademik başarılarının daha yüksek olduğu bulunmuştur. Fakat, kurumların uzaktan eğitime ilişkin yönetmeliklerindeki sınırlılıklar nedeni ile adil bir değerlendirme sürecinin garanti edilemeyeceği tartışılmaktadır. Bu doğrultuda başarı ölçütlerinin daha iyi nasıl belirlenebileceğine odaklanılarak öğrenme performansını daha nitelikli ortaya koyabilecek uygulama örneklerinin çoğaltılması faydalı olabilir.
KW  - e-değerlendirme
KW  - öğrenme performansı
KW  - MOODLE analitikleri
KW  - kümeleme
N2  - This research aims to examine the midterm and final performance, e-assessment design perception, and general learning experiences of learners whose performance is different according to the analytics used in e-assessment.  In the research, the learning analytics process was carried out and the descriptive analytics method was used. This process includes the collection and analysis of metrics that can be associated with learning performance in the periods until the midterm and the final. The study group consists of 285 students enrolled in distance education programs and taking the Information and Communication Technology course. Data were collected through pre-test for each subject, student monitoring tools in MOODLE LMS, an online assessment scale, and midterm and final exams (midterm and final). Clustering analysis (k-means and hierarchical) was used to describe learning performance. Differences between academic achievement, e-assessment, and general learning experiences by clusters were analyzed by t-test. As a result, in an e-assessment design like the one in this study, it was found that students with high performance in terms of the variables considered had higher academic achievement. However, it is argued that due to the limitations in the regulations, a fair assessment process cannot be guaranteed. In this respect, it may be beneficial to focus on how to determine the success criteria better and how to increase the implementation examples that can demonstrate the learning performance in a more qualified way.
CR  - Adedoyin, O. B., ve Soykan, E. (2020). Covid-19 pandemic and online learning: the challenges and opportunities. Interactive Learning Environments, 1-13. https://doi.org/10.1080/10494820.2020.1813180
CR  - Ahmed, F. R. A., Ahmed, T. E., Saeed, R. A., Alhumyani, H., Abdel-Khalek, S., ve Abu-Zinadah, H. (2021). Analysis and challenges of robust E-exams performance under COVID-19. Results in Physics, 23, 103987. https://doi.org/10.1016/j.rinp.2021.103987
CR  - Ahmed, A., Zualkernan, I., ve Elghazaly, H. (2021, July). Unsupervised Clustering of Skills for an Online Learning Platform. In 2021 International Conference on Advanced Learning Technologies (ICALT) (pp. 200-202). IEEE. https://doi.org/10.1109/ICALT52272.2021.00066
CR  - Akçapınar, G., Altun, A., & Aşkar, P. (2016). Çevrimiçi Öğrenme Ortamındaki Benzer Öğrenci gruplarının Kümeleme Yöntemi ile Belirlenmesi. Eğitim Teknolojisi Kuram ve Uygulama, 6(2), 46-64. https://doi.org/10.17943/etku.91440
CR  - Akçapınar, G. ve Bayazıt, A. (2019). MoodleMiner: Moodle Öğrenme Yönetim Sistemi için Veri Madenciliği Analiz Aracı.  İlköğretim Online, 18(1). s. 406-415. https://doi.org/10.17051/ilkonline.2019.527645
CR  - Al-Hattami, A. A. (2020). E-Assessment of Students Performance During the E-Teaching and Learning. International Journal of Advanced Science and Technology, ISSN, 4238, 1537-1547.
CR  - Alsadoon, H. (2017). Students' Perceptions of E-Assessment at Saudi Electronic University. Turkish Online Journal of Educational Technology-TOJET, 16(1), 147-153.
CR  - Aranganayagi, S., ve Thangavel, K. (2007, December). Clustering categorical data using silhouette coefficient as a relocating measure. In International conference on computational intelligence and multimedia applications (ICCIMA 2007) (Vol. 2, pp. 13-17). IEEE. https://doi.org/10.1109/ICCIMA.2007.328
CR  - Bayazıt, A., ve Akçapınar, G. (2018). Çevrimiçi dersler için video analitik aracının tasarlanması ve geliştirilmesi. Elementary Education Online, 17(1). http://dergipark.gov.tr/ilkonline/issue/36274/413719
CR  - Bayrak, F., ve Yurdugül, H. (2016). Web-Tabanlı Öz-Değerlendirme Sisteminde Öğrenci Uyarı İndeksini Temel Alan Öğrenme Analitiği Modülünün Tasarlanması. Eğitim Teknolojisi Kuram ve Uygulama, 6(2), 85-99. https://doi.org/10.17943/etku.59549
CR  - Bozkurt, A., ve Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), i-vi.
CR  - Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö., E., Karadeniz, Ş., Demirel, F. (2020). Eğitimde Bilimsel Araştırma Yöntemleri. PEGEM AKADEMİ, ANKARA
CR  - Bravo-Agapito, J., Romero, S. J., ve Pamplona, S. (2021). Early prediction of undergraduate Student's academic performance in completely online learning: A five-year study. Computers in Human Behavior, 115, 106595. https://doi.org/10.1016/j.chb.2020.106595
CR  - Chowdhury, M., Demir, I., Jiang, J., ve Shahzad, N. (2021). Will Highschool Students After the Pandemic want a System of Education to be a Hybrid of Remote Learning and in Person Learning. Across The Spectrum of Socioeconomics, 4, 252. https://doi.org/10.5281/zenodo.4743649
CR  - CINECA elearning Support. (2017). Moodle plugins directory: Attendance Register: Versions. https://moodle.org/plugins/mod_attendanceregister/versions ‘ den Haziran 2022’ de erişildi.
CR  - Doğan, N., Kıbrıslıoğlu Uysal, N., Kelecioğlu, H. ve Hambleton, R. K. (2020). An overview of e-assessment. Hacettepe University Journal of Education, 35 (Special Issue), 1-5. https://doi.org/10.16986/HUJE.2020063669
CR  - Du, X., Yang, J., Shelton, B. E., Hung, J. L., ve Zhang, M. (2021). A systematic meta-review and analysis of learning analytics research. Behaviour & information technology, 40(1), 49-62. https://doi.org/10.1080/0144929X.2019.1669712
CR  - Elzainy, A., El Sadik, A., ve Al Abdulmonem, W. (2020). Experience of e-learning and online assessment during the COVID-19 pandemic at the College of Medicine, Qassim University. Journal of Taibah University Medical Sciences, 15(6), 456-462. https://doi.org/10.1016/j.jtumed.2020.09.005
CR  - Figaredo, D. D., Jaurena, I. G., ve Encina, J. M. (2022). The Impact of Rapid Adoption of Online Assessment on Students’ Performance and Perceptions: Evidence from a Distance Learning University. Electronic Journal of e-Learning, 20(3), pp224-241. https://doi.org/10.34190/ejel.20.3.2399
CR  - Ferri, F., Grifoni, P., ve Guzzo, T. (2020). Online learning and emergency remote teaching: Opportunities and challenges in emergency situations. Societies, 10(4), 86. https://doi.org/10.3390/soc10040086
CR  - Foerster, M. (2019). Framework for the quality assurance of e-assessment. Voced.edu.au; ENQA. https://www.voced.edu.au/content/ngv:84627
CR  - Glassey, J., ve Abegão, F. R. (2017, November). E-assessment and tailored feedback-are they contributing to the effectiveness of chemical engineering education?. In 2017 7th World Engineering Education Forum (WEEF) (pp. 508-512). IEEE. https://doi.org/10.1109/WEEF.2017.8467054
CR  - Guerrero-Roldán, A. E., ve Noguera, I. (2018). A model for aligning assessment with competences and learning activities in online courses. The Internet and Higher Education, 38, 36-46. https://doi.org/10.1016/j.iheduc.2018.04.005
CR  - Han, P., Wang, W., Shi, Q., ve Yue, J. (2021). A combined online-learning model with K-means clustering and GRU neural networks for trajectory prediction. Ad Hoc Networks, 117, 102476. https://doi.org/10.1016/j.adhoc.2021.102476
CR  - Harlen, W., ve James, M. (1997). Assessment and learning: differences and relationships between formative and summative assessment. Assessment in Education: Principles, Policy & Practice, 4(3), 365-379. https://doi.org/10.1080/0969594970040304
CR  - Hewson, C., ve Charlton, J. P. (2019). An investigation of the validity of course‐based online assessment methods: The role of computer‐related attitudes and assessment mode preferences. Journal of Computer Assisted Learning, 35(1), 51-60. https://doi.org/10.1111/jcal.12310
CR  - Hodges, C., Moore, S., Lockee, B., Trust, T., ve Bond, A. (2020). The difference between emergency remote teaching and online learning. Educause Review, 27, 1-12.
CR  - Holmes, W., Nguyen, Q., Zhang, J., Mavrikis, M., ve Rienties, B. (2019). Learning analytics for learning design in online distance learning. Distance Education, 40(3), 309-329. https://doi.org/10.1080/01587919.2019.1637716
CR  - Ilgaz, H., ve Adanır, G. A. (2020). Providing online exams for online learners: Does it really matter for them?. Education and Information Technologies, 25(2), 1255-1269. https://doi.org/10.1007/s10639-019-10020-6
CR  - Kearns, L. R. (2012). Student assessment in online learning: Challenges and effective practices. Journal of Online Learning and Teaching, 8(3), 198.
CR  - Kişisel Verilerin Korunması Kanunu. (2016, 7 Nisan). Resmi Gazete (Sayı: 29677). Erişim adresi:https://www.mevzuat.gov.tr/mevzuat?MevzuatNo=6698&MevzuatTur=1&MevzuatTertip=5
CR  - Knight, S., Shum, S. B., ve Littleton, K. (2014). Epistemology, assessment, pedagogy: where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2), 23-47. https://doi.org/10.18608/jla.2014.12.3
CR  - Laurillard, D. (2013). Teaching as a design science: Building pedagogical patterns for learning and technology. Routledge. https://doi.org/10.4324/9780203125083
CR  - Marín, V. I., ve Garcias, A. P. (2016). Collaborative e-Assessment as a Strategy for Scaffolding Self-Regulated Learning in Higher Education. In Formative Assessment, Learning Data Analytics and Gamification (pp. 3-24). Academic Press. https://doi.org/10.1016/B978-0-12-803637-2.00001-4
CR  - Martin, F., & Ndoye, A. (2016). Using Learning Analytics to Assess Student Learning in Online Courses. Journal of University Teaching & Learning Practice, 13(3). https://doi.org/10.53761/1.13.3.7
CR  - Mayer, R. E. (2019). Thirty years of research on online learning. Applied Cognitive Psychology, 33(2), 152-159. https://doi.org/10.1002/acp.3482
CR  - Mellar, H., Peytcheva-Forsyth, R., Kocdar, S., Karadeniz, A., ve Yovkova, B. (2018). Addressing cheating in e-assessment using student authentication and authorship checking systems: teachers’ perspectives. International Journal for Educational Integrity, 14(1), 1-21. https://doi.org/10.1007/s40979-018-0025-x
CR  - Moodle Community. (2019a). Course reports - MoodleDocs. https://docs.moodle.org/311/en/Course_reports ‘ dan Haziran 2022’de erişildi.
CR  - Moodle Community. (2019b). Grading quick guide - MoodleDocs. https://docs.moodle.org/311/en/Grading_quick_guide ‘ dan Haziran 2022’de erişildi.
CR  - Moodle Community. (2021a). Feedback activity - MoodleDocs. https://docs.moodle.org/311/en/Feedback_activity ‘ den Haziran 2022’de erişildi.
CR  - Moodle Community. (2021b). Quiz activity - MoodleDocs. https://docs.moodle.org/311/en/Quiz_activity ‘ den Haziran 2022’de erişildi.
CR  - Moodle Community. (2021c). Using Activity completion - MoodleDocs. https://docs.moodle.org/311/en/Using_Activity_completion ‘ den Haziran 2022’de erişildi.
CR  - Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and higher Education, 31(1), 53-64. https://doi.org/10.1080/03098770601167922
CR  - Ogange, B. O., Agak, J. O., Okelo, K. O., ve Kiprotich, P. (2018). Student perceptions of the effectiveness of formative assessment in an online learning environment. Open Praxis, 10(1), 29-39. https://doi.org/10.5944/openpraxis.10.1.705
CR  - Peytcheva-Forsyth, R., ve Aleksieva, L. (2021, March). Forced introduction of e-assessment during COVID-19 pandemic: How did the students feel about that? (Sofia University case). In AIP Conference Proceedings (Vol. 2333, No. 1, p. 050013). AIP Publishing LLC. https://doi.org/10.1063/5.0041862
CR  - Pishchukhina, O., ve Allen, A. (2021, September). Supporting learning in large classes: online formative assessment and automated feedback. In 2021 30th Annual Conference of the European Association for Education in Electrical and Information Engineering (EAEEIE) (pp. 1-4). IEEE. https://doi.org/10.1063/5.0041862
CR  - Rajabalee, Y. B., Santally, M. I., ve Rennie, F. (2019). The use of learning analytics to improve online learning outcomes: A systematic literature review [Working paper]. Ninth Pan-Commonwealth Forum, Edinburgh, UK. http://hdl.handle.net/11599/3275
CR  - Reeves, T. C. (2000). Alternative assessment approaches for online learning environments in higher education. Journal of Educational Computing Research, 23(1), 101-111. https://doi.org/10.2190%2FGYMQ-78FA-WMTX-J06C
CR  - Rienties, Bart and Jones, Ann (2019). Evidence -Based Learning: Futures. Using learning design and learning analytics to empower teachers to meet students’ diverse needs. In: Ferguson, Rebecca; Jones, Ann and Scanlon, Eileen eds. Educational Visions: The lessons from 40 years of innovation. London: Ubiquity Press, pp. 109–125.
CR  - Rolim, C., ve Isaias, P. (2019). Examining the use of e‐assessment in higher education: teachers and students’ viewpoints. British Journal of Educational Technology, 50(4), 1785-1800. https://doi.org/10.1111/bjet.12669
CR  - Sa'di, R., Abdelraziq, A., ve Sharadgah, T. (2021). E-Assessment at Jordan’s Universities in the Time of the COVID-19 Lockdown: Challenges and Solutions. Arab World English Journal (AWEJ) Special Issue on Covid, 19. https://dx.doi.org/10.24093/awej/covid.3
CR  - Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144. https://doi.org/10.1007/BF00117714
CR  - Sandoval, A., Gonzalez, C., Alarcon, R., Pichara, K., ve Montenegro, M. (2018). Centralized student performance prediction in large courses based on low-cost variables in an institutional context. The Internet and Higher Education, 37, 76-89. https://doi.org/10.1016/j.iheduc.2018.02.002
CR  - Saykili, A., Ozturk, A., Kumtepe, E. G., Kumtepe, A. T., ve Uğurhan, Y. Z. C. Examining the Effects of LMS Use on Academic Performance Using Learning Analytics. Hosted by UNED, Madrid (Spain), 148.
CR  - Sharadgah, T., ve Sa'di, R. (2020). Preparedness of institutions of higher education for assessment in virtual learning environments during the Covid-19 lockdown: Evidence of bona fide challenges and pragmatic solutions. Journal of Information Technology Education: Research, 19(1), 755-774. Doi: 10.28945/4615
CR  - Shavelson, R. J., Zlatkin-Troitschanskaia, O., ve Mariño, J. P. (2018). International performance assessment of learning in higher education (iPAL): Research and development. In Assessment of learning outcomes in higher education (pp. 193-214). Springer, Cham. https://doi.org/10.1007/978-3-319-74338-7_10
CR  - Soffer, T., Kahan, T., ve Livne, E. (2017). E-assessment of online academic courses via students' activities and perceptions. Studies in Educational Evaluation, 54, 83–93. https://doi.org/10.1016/j.stueduc.2016.10.001
CR  - Soffer, T., ve Nachmias, R. (2018). Effectiveness of learning in online academic courses compared with face‐to‐face courses in higher education. Journal of Computer Assisted Learning, 34(5), 534-543. https://doi.org/10.1111/jcal.12258
CR  - Soffer, T., Kahan, T., & Nachmias, R. (2019). Patterns of students’ utilization of flexibility in online academic courses and their relation to course achievement. International Review of Research in Open and Distributed Learning, 20(3). https://doi.org/10.19173/irrodl.v20i4.3949
CR  - Stödberg, U. (2012). A research review of e-assessment. Assessment & Evaluation in Higher Education, 37(5), 591-604. https://doi.org/10.1080/02602938.2011.557496
CR  - Teaching with Blackboard. (2019). Accessing the Attendance Report from a Blackboard Collaborate Session [YouTube Video]. In YouTube. https://www.youtube.com/watch?v=VdFn-NnwZP4
CR  - Tempelaar, D. (2020). Supporting the less-adaptive student: the role of learning analytics, formative assessment and blended learning. Assessment & Evaluation in Higher Education, 45(4), 579-593. https://doi.org/10.1080/02602938.2019.1677855
CR  - Tracking progress - MoodleDocs. (2020). Retrieved January 1, 2022, from Moodle.org website: https://docs.moodle.org/311/en/Tracking_progress
CR  - University of the People. (2020, May 10). Emergency Remote Teaching Vs. Online Learning: A Comparison. https://www.uopeople.edu/blog/emergency-remote-teaching-vs-online-learning/ ’ den Haziran 2022’de erişildi.
CR  - Xiong, Y., ve Suen, H. K. (2018). Assessment approaches in massive open online courses: Possibilities, challenges and future directions. International Review of Education, 64(2), 241-263. https://doi.org/10.1007/s11159-018-9710-5
CR  - Yildirim, D., Gülbahar, Y. (2022). Implementation of Learning Analytics Indicators for Increasing Learners' Final Performance. Technology Knowledge and Learning. 27. https://doi.org/10.1007/s10758-021-09583-6
CR  - Yildirim, D., ve Seferoğlu, S. S. (2021). Evaluation of the effectiveness of online courses based on the community of inquiry model. Turkish Online Journal of Distance Education, 22(2), 147-163., https://doi.org/10.17718/tojde.906834
CR  - Yildirim, D., ve Usluel, Y. (2022). Interrelated analysis of interaction, sequential patterns and academic achievement in online learning. Australasian Journal of Educational Technology, 38(2), 181–200. https://doi.org/10.14742/ajet.7360
CR  - Yıldız, G., ve Çakmak, E. K. (2019). Zenginleştirilmiş E-Değerlendirme Sisteminin Ders Başarısına Etkisi ve Öğrenci Memnuniyetinin İncelenmesi. Gazi Eğitim Bilimleri Dergisi, 5, 106-139. https://dergipark.org.tr/en/pub/gebd/issue/49407/588868’ den Haziran 2022’ de erişildi.
CR  - Yükseköğretim Kurulu (2020). Yükseköğretim Kurumlarında Uzaktan Öğretime İlişkin Usul Ve Esaslar, https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Uzaktan_ogretim/yuksekogretim_kurumlarinda_uzaktan_ogretime_iliskin_usul_ve_esaslar.pdf ‘den Haziran 2022’de erişildi.
UR  - https://doi.org/10.17943/etku.1062731
L1  - https://dergipark.org.tr/en/download/article-file/2211662
ER  -