Yüksek Öğrenimde Çevrimiçi Değerlendirmede Fakülte Üyelerinin Yaşadığı Zorluklara Yönelik Gizli Sınıf Analizi Yaklaşımı

Year 2024, Volume: 26 Issue: 2, 197 - 207, 30.06.2024

Emrah Gül

https://doi.org/10.17556/erziefd.1382191

Abstract

Çevrim içi ölçme ve değerlendirme, öğretim elemanlarının öğrenmeyi yönlendirmek ve denetlemek için bilgisayar teknolojilerini kullanmasıdır. Pandemi ve doğal afetler nedeniyle eğitimde sürdürülebilirliğin sağlanması için birçok üniversite, teknolojinin getirdiği avantajlardan yararlanarak çevrim içi ölçme ve değerlendirme uygulamalarını kullanmışlardır. Betimsel türde tasarlanan bu araştırma 105 öğretim elemanı ile yürütülmüştür. Bu amaç ile geliştirilen ölçme ve değerlendirmeye yönelik zorlukları belirleyen ölçme aracı, öğretim elemanlarına uygulanmış ve sonuçlar örtük sınıf analizi ile incelenmiştir. Araştırmaya Akaike bilgi kriteri (AIC) ve Bayesyen bilgi kriteri (BIC) değerlerine göre örtük sınıf sayısının belirlenmesi ile başlanmış ve veri yapısının iki sınıflı model ile uyumlu olduğu belirlenmiştir. Sınıf sayısı belirlendikten sonra iki sınıflı yapı test edilerek maddelere ait koşullu olasılıklar hesaplanmış ve yorumlanmıştır. Araştırmanın sonuçlarına göre çevrim içi ölçme ve değerlendirme uygulamalarında oluşan sınıflardan birincisinin zorlanan gruba (%58.7) ikincisinin ise zorlanmayan guruba (% 41.3) ait olduğu tespit edilmiştir. Koşullu olasılıklar incelendiğinde, veri yapısının iki sınıflı olmasına en büyük katkıyı sağlayan gözlenen değişkenlerin: kopya çekme, intihal yapma ve eğitim politikalarının eksikliği olduğu sonucuna ulaşılmıştır. Her iki grupta (zorlanan veya zorlanmayan) da çevrim içi ölçme ve değerlendirme uygulamalarında zorlanılan konuların başında kopya çekme, intihal, eğitim politikalarının eksiklikleri olduğu bulunmuştur.

Keywords

Çevrimiçi Değerlendirmenin Zorlukları, Gizli Sınıf Analizi, Akademik Personel, Yüksek Öğrenim

A Latent Class Analysis Approach to Challenges Experienced by Faculty Members in Online Assessment in Higher Education

Abstract

Online assessment is the use of computer technologies by faculty members to guide and check learning. Taking the advantage of technology, many universities have used online assessment applications to ensure sustainability in education due to the pandemic and natural disasters. The purpose of the current study is to explore challenges experienced by faculty members in online assessment, using latent class analysis. The descriptive design research was carried out with the participation of 105 faculty members. For the study, the number of latent classes was decided according to the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) and it was observed that the data structure was a good fit for a two-class model. According to the research results, the first class in online assessment applications was considered as the with-difficulty group (58.7 %) and the second as the without-difficulty group (41.3 %). When the conditional probabilities were examined, it was concluded that the observed variables that mostly contributed to the two-class model data structure were as follows, cheating, plagiarism and lack of education policies. It was found that the primary challenges in both groups (with or without difficulty) in online assessment applications were cheating, plagiarism and lack of education policies.

Keywords

Challenges Online Assessment, Latent Class Analysis, Academik Staff, Higher Education

Ethical Statement

This research received favourable opinion from the Hakkari University’s research ethics committee: reference: 20.12.2022-43035.

Supporting Institution

Not applicable

Thanks

Not applicable

References

• Alruwais, N., Wills, G. & Wald, M. (2018). Advantages and challenges of using e-assessment. International Journal of Information and Education Technology, 8(1), 34-37. https://www.ijiet.org/vol8/1008-JR261.pdf
• Altıntaş, Ö. & Kutlu, Ö. (2020). Investigating Measurement Invariance of Ankara University Foreign Students Selection Test According to Latent Class and Rasch Model, Education and Science, vol.45, no.203, pp.287-308, http://dx.doi.org/10.15390/EB.2020.8685
• Amante, L., Oliveira, I. R. & Gomes, M. J. (2019). E-assessment in Portuguese higher education: Framework and perceptions of teachers and students. In A. Azevedo & J. Azevedo (Eds.), Handbook of research on e-assessment in higher education (pp. 312–333). IGI Global.
• Bartholomew, P. M., Knott, M. & Moustaki, I. (2011). Latent variable models and factor analysis a unified approach 3rd edition. West Sussex: John Wiley& Sons, Ltd.
• Bearman, M., Dawson, P., Ajjawi, R., Tai, J. & Boud, D. (2020). Re-imagining university assessment in a digital world. The Enabling Power of Assessment, 7. Retrieved from https://doi.org/10.1007/978-3-030-41956-1
• Bensaid, B. & Brahimi, T. (2020). Coping with COVID-19: Higher education in the GCC countries. Springer Proceedings in Complexity, 137–153. https://doi.org/10.1007/978-3-030-62066-0_12
• Bloom, T. J., Rich, W. D., Olson, S. M. & Adams, M. L. (2018). Perceptions and performance using computer-based testing: One institution's experience. Currents in Pharmacy Teaching and Learning, 10(2), 235–242. https://doi.org/10.1016/j.cptl.2017.10.015
• Bozkurt, A. & Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), 1-6. https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/447/297
• Bull, J. & McKenna, C. (2003). A blueprint for computer-assisted assessment. Routledge.
• Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö.E., Karadeniz, Ş. & Demirel, F. (2012). Bilimsel araştırma yöntemleri. Ankara: Pegem Yayınları
• Cavus, N. (2015). Distance learning and learning management systems. Procedia-Social and Behavioral Sciences, 191(2), 872-877. https://doi.org/10.1016/j.sbspro.2015.04.611
• Celeux, G. & Soromenho, G. (1996). An entropy criterion for assessing the number of clusters in a mixture model. Journal of Classification, 13(2), 195-212. https://doi.org/10.1007/BF01246098
• Collares, C. F. & Cecilio-Fernandes, D. (2019). When I say … computerised adaptive testing. Medical Education, 53(2), 115–116. https://doi.org/10.1111/medu.13648
• Collins, L. M. & Lanza, S. T. (2010). Latent class andlatent transition analysis with application in the social, behavioral, and health sciences. New Jersey: John Wiley and Sons, Inc.
• Darling-Aduana, J. (2021). Authenticity, engagement, and performance in online high school courses: Insights from micro-interactional data. Computers & Education, 167, 104175. https://doi.org/10.1016/j.compedu.2021.104175
• Demir, E. (2014). Uzaktan eğitime genel bir bakış. Dumlupınar Üniversitesi Sosyal Bilimler Dergisi , (39) https://dergipark.org.tr/tr/download/article-file/55935
• Dermo, J. (2009). e-Assessment and the student learning experience: A survey of student perceptions of e-assessment. British Journal of Educational Technology, 40(2), 203–214. https://doi.org/10.1111/j.1467-8535.2008.00915.x
• Deutsch, T., Herrmann, K., Frese, T. & Sandholzer, H. (2012). Implementing computer-based assessment A web-based mock examination changes attitudes. Computers & Education, 58(4), 1068–1075. https://doi.org/10.1016/j.compe du.2011.11.013
• Diprose, M. (2013). Learning and assessment credibility: The design of examination strategies in a changing learning environment. Knowledge Management & E-Learning: An International Journal, 5(1), 104-116. https://doi.org/10.34105/j.kmel.2013.05.008
• Doğan-Gül, Ç. (2022). TIMSS-2015 araştırmasının dil ve cinsiyete göre ölçme değişmezliğinin gizil sınıf analizi ile incelenmesi.(Yayımlanmamış doktora tezi). Ankara Üniversitesi, Ankara, Türkiye.
• Dube, T., Zhao, Z. & Ma, M. (2009). E-assessment and design methodology management. http://eprints.hud.ac.uk/id/eprint/23208
• Eid, M. & Diener, E. (2001). Norms for affect in different cultures: Inter- and intraindividual differences. Journal of Personality and Social Psychology, 81, 869-885. https://doi.org/10.1037/0022-3514.81.5.869
• Eljinini, M. & Alsamarai, S. (2012). The impact of e-assessments system on the success of the implementation process,” Mod. Educ. Comput. Sci., vol. 4, no. 11, pp. 76–84, Dec.. https://doi.org/10.5815/ijmecs.2012.11.08
• Elsalem, L., Al-Azzam, N., Jum’ah, A., Obeidat, N., Sindiani, A. & Kheirallah, K. (2020). Stress and behavioral changes with remote E-exams during the Covid- 19 pandemic: A cross-sectional study among undergraduates of medical sciences. Elsevier, 60, 270-280. https://doi.org/10.1016/j.amsu.2020.10.058
• Fageeh, A. I. (2015). EFL student and faculty perceptions of and attitudes towards online testing in the medium of Blackboard: Promises and challenges. The JALT CALL Journal, 11(1), 41–62. https://doi.org/10.29140/jaltc all.v11n1.183
• Finch, H. (2015). A comparison of statistics for assessing model invariance in latent class analysis. Open Journal of Statistics, 5, 191-210. https://www.doi.org/10.4236/ojs.2015.53022
• Flavin M., (2022). A Disruptive Innovation perspective on students’ opinions of online assessment. Research in Learning Technology, 2021, 29: 2611 http://dx.doi.org/10.25304/rlt.v29.2611
• Gilbert, L., Whitelock, D. & Gale, V. (2011) “Synthesis report on assessment and feedback with technology enhancement,” Southampton.
• Goodman, L. A. (1974). Exploratory latent structure analysis using identifiable models, Biometrika, 61(2), 215-231. https://doi.org/10.2307/2334349
• Guangul, F. M., Suhail, A. H., Khalit, M. I. & Khidhir, B. A. (2020). Challenges of remote assessment in higher education in the context of COVID-19: a case study of Middle East College. Educational Assessment, Evaluation and Accountability, 32(4), 519–535. https://doi.org/10.1007/s11092-020-09340-w
• Güngör, D., Korkmaz, M. & Somer, O. (2013). Çoklu-Grup Örtük Sınıf Analizi ve Ölçme Eşdeğerliği. Türk Psikoloji Dergisi, 28 (72), 48-57. https://www.tpd.com.tr/tr/yayinlar/dergiler/1031828/tpd1300443320130000m000061.pdf
• Güngör-Çulha, D. (2012). Örtük sınıf analizlerinde ölçme eşdeğerliğinin incelenmesi. (Yayımlanmamış doktora tezi). Ege Üniversitesi, İzmir, Türkiye.
• Hamsatu, P., Yusufu, G. & Mohammed, H. A. (2016). Teachers' perceptions and undergraduate students' experience in e-exam in higher institution in Nigeria. Journal of Education and Practice, 7(23), 158–166. https://files.eric.ed.gov/fullt ext/EJ111 2920.pdf
• Heritage, M. (2007). Formative assessment: What do teachers need to know and do? Phi Delta Kappan, 89(2), 140-145. https://doi.org/10.1177/003172170708900210
• Jamil, M., Tariq, R. H. & Shami, P. A. (2012). Computer-based vs paper-based examinations: Perceptions of university teachers. Turkish Online Journal of Educational Technology, 11(4), 371–381. https://files.eric.ed.gov/fulltext/EJ989302.pdf
• Jawaid, M., Moosa, F. A., Jaleel, F. & Ashraf, J. (2014). Computer based assessment (CBA): Perception of residents at Dow University of Health Sciences. Pakistan Journal of Medical Sciences, 30(4), 688. doi: http://dx.doi.org/10.12669/pjms.304.5444
• Jordan, S. & Mitchell, T. (2009). E-assessment for learning? The probability of short-answer free-text questions with tailored feedback. Br. J. Educ. Technol., vol. 40, no. 2, pp. 371–385, Mar. 2009. https://doi.org/10.1111/j.1467-8535.2008.00928.x
• Jordan, S. (2013). E-assessment: Past, present and future. New Directions in the Teaching of Physical Sciences, (9), 87-106. https://doi.org/10.29311/ndtps.v0i9.504
• Kankaras, M. & Moors, G. (2009). Measurement equivalence in solidarity attitudes in Europe. Insights from a multiple group latent class factor approach. International Sociology, 24(4), 557-579. https://doi.org/10.1177/0268580909334502
• Kankaras, M., Moors, G.B.D. & Vermunt, J.K. (2010). Testing for measurement invariance with latent class analysis. In E. Davidov, P. Schmidt, & J. Billiet (Ed.), Cross-cultural analysis. Methods and applications (ss. 359-384). Taylor & Francis Group: Routledge.
• Karasar, N. (2015). Araştırmalarda Rapor Hazırlama (19. Basım). Nobel Yayınları, ISBN: 978-605-5426-15-6, İstanbul, 152 s.
• Khan, S. & Khan, R. A. (2019). Online assessments: Exploring perspectives of university students. Education and Information Technologies (2019) 24:661–677 https://doi.org/10.1007/s10639-018-9797-0
• Khare, A. & Lam, H. (2008). Assessing student achievement and progress with online examinations: some pedagogical and technical issues. International Journal on E-Learning, 7(3), 383–402. https://www.learntechlib.org/primary/p/23620/.
• Kumar, L. R., Bedra, A. & Karkera, R. (2013). Perception of medical students on e-assessment conducted through Yengage portal. Archives of Medicine and Health Sciences, 1(1), 61. https://doi.org/10.4103/232 1-4848.113577
• Lazerfeld, P. F. (1950). The logical and mathematics foundatitions of latent structure analysis. in measurement and prediction (ed. Stouffer at all.) Princeton University Press.
• Lin, H. T. (2006). A comparison of model selection indices for nested latent class models. Monte Carlo Methods and Applications, 12(3), 239-259. https://www.doi.org/10.1515/156939606778705164
• Linzer, D. A. & Lewis, J. (2007). poLCA: Polytomous Variable Latent Class Analysis. R. JSS Journal of Statistical Software, VV(II), https://www.sscnet.ucla.edu/polisci/faculty/lewis/pdf/poLCA-JSS-final.pdf
• Lockman, A. S. & Schirmer, B. R. (2020). Online instruction in higher education: Promising, research-based, and evidence-based practices. Journal of Education and E-Learning Research, 7(2), 130–152. https://doi.org/10.20448/journal.509.2020.72.130.152
• McCutcheon, A. L. (1987). Latent class analysis. Sage University Papers Series.
• Middleton, K. V. (2022). Considerations for Future Online Testing and Assessment in Colleges and Universities. Educational Measurement: Issues and Practice. Spring 2022, Vol. 41, No. 1, pp. 51–53. https://doi.org/10.1111/emip.12497
• Mirza, H. (2021). University Teachers’ Perceptions of Online Assessment during the Covid-19 Pandemic in Lebanon, American Academic & Scholarly Research Journal, 13(1). https://aasrc.org/aasrj/index.php/aasrj/article/view/2064/1187
• Mitchell, T., Aldridge, N., Williamson, W. & Broomhead, P. (2003). Computer based testing of medical knowledge. Presented at the 7th Computer Assisted Assessment Conference.
• Momeni, A. (2022). Online Assessment in Times of COVID-19 Lockdown: Iranian EFL Teachers' Perceptions, International Journal of Language Testing. Vol. 12, No. 2, October 2022. https://files.eric.ed.gov/fulltext/EJ1363125.pdf
• Montenegro-Rueda, M., Luque-de la Rosa, A., Sarasola Sánchez-Serrano, J.L. & FernándezCerero, J. (2021). Assessment in Higher Education during the COVID-19 Pandemic: A Systematic Review. Sustainability, 13, 10509. https://doi.org/10.3390/su131910509
• Moors, G. & Wennekers, C. (2003). Comparing moral values in western european countries between 1981 and 1999. A multigroup latent-class factor approach. International Journal of Comparative Sociology, 44(1):155-172. https://doi.org/10.1177/002071520304400203
• Oosterhof, A., Conrad, R. M. & Ely, D. P. (2008). Assessing learners online. New Jersey: Pearson.
• Osman, M. E. (2020). Global impact of COVID-19 on education systems: The emergency remote teaching at Sultan Qaboos University. Journal of Education for Teaching, 46(4), 463-471. https://doi.org/10.1080/02607476.2020.1802583
• Peterson, D. (2013). Five steps to better tests. Questionmark White Paper. Retrieve from https://www.questionmark.com/test-design-and-delivery-overview/
• R Development Core Team (2007). R: a language and environment for statistical computing, r foundation for statistical computing, Vienna, Austria. ISBN 3-900051-07-0.
• Ridgway, J., McCusker, S. & Pead, D. (2004). “Literature review of e-assessment,” Bristol. http://www.futurelab.org.uk/resources/publications-reports-articles/literature-reviews/Literature-Review204
• Rolim, C. & Isaias, P. (2018). Examining the use of e-assessment in higher education: Teachers and students’ viewpoints. British Journal of Educational Technology, 50(4), 1785–1800. https://doi.org/10.1111/bjet.12669
• Sa'di, R. A., Abdelraziq, A. & Sharadgah, T. A. (2021). E-Assessment at Jordan’s Universities in the Time of the COVID-19 Lockdown: Challenges and Solutions. Arab World English Journal (AWEJ) Special Issue on Covid 19 Challenges (1) 37-54. DOI: https://dx.doi.org/10.24093/awej/covid.3
• Schneberger, S., Amoroso, D. L. & Durfee, A. (2008). Factors that influence the performance of computer-based assessments: An extension of the technology acceptance model. Journal of Computer Information Systems, 48(2), 74–90. https://doi.org/10.1080/08874417.2008.11646011
• Şenel, S. & Şenel, H. C. (2021). Remote Assessment in Higher Education during COVID-19 Pandemic. International Journal of Assessment Tools in Education, 8 (2), 181-199 . DOI: 10.21449/ijate.820140
• Silvia, P. J., Kaufman, J. C. & Pretz, J. E. (2009). Is creativity domain-specific? Latent class models of creative accomplishments and creative self-descriptions. Psychology of Aesthetics, Creativity, and the Arts, 139-148. https://libres.uncg.edu/ir/uncg/f/P_Silvia_Is_2008.pdf
• Sim, G., Holifield, P. & Brown, M. (2004). Implementation of computer assisted assessment: Lessons from the literature. ALT-J, 12(3), 215-229. https://doi.org/10.3402/rlt.v12i3.11255
• Stiggins, R.J. (1992) High quality classroom assessment: what does it really mean? Educ Meas Issues Pract 11(2):35–39. https://doi.org/10.1111/j.1745-3992.1992.tb00241.x
• Stone, D. E. & Zheng, G. (2014). Learning management systems in a changing environment. In Handbook of research on education and technology in a changing society (pp. 756-767). IGI Global. https://www.doi.or/10.4018/978-1-4666-6046-5.ch056
• Terzis, V. & Economides, A. A. (2011). Computer based assessment: Gender differences in perceptions and acceptance. Computers in Human Behavior, 27(6), 2108–2122. https://doi.org/10.1016/j.chb.2011.06.005
• Terzis, V., Moridis, C. N. & Economides, A. A. (2013a). Continuance acceptance of computer-based assessment through the integration of user's expectations and perceptions. Computers & Education, 62, 50–61. https://doi.org/10.1016/j.compe du.2012.10.018
• Terzis, V., Moridis, C. N., Economides, A. A. & Mendez, G. R. (2013b). Computer based assessment acceptance: A cross-cultural study in Greece and Mexico. Journal of Educational Technology & Society, 16(3), 411–424. https://www.jstor.org/stabl e/jeduc techs oci.16.3.411
• Vermunt, J. K. & Madigson, J. (2004). Local Indepence. In A. B. M.S. Lewis Beck, The Sage Encyclopedia of Social Sciences Research Methods. 732-733.
• Vurdien, R. & Puranen, P. (2022). Teacher attitudes toward online assessment in challenging times. In B. Arnbjörnsdóttir, B. Bédi, L. Bradley, K. Friðriksdóttir, H. Garðarsdóttir, S. Thouësny, & M. J. Whelpton (Eds), Intelligent CALL, granular systems, and learner data: short papers from EUROCALL 2022 (pp. 370-374). https://doi.org/10.14705/rpnet.2022.61.1486
• Wang M.-C., Deng Q., Bi X., Ye H. & Yang W. (2017). Performance of the entropy as an index of classification accuracy in latent profile analysis: A Monte Carlo simulation study. Acta Psychologica Sinica, 49(11), 1473-1482. https://doi.org/10.3724/SP.J.1041.2017.01473
• Wang, W. & Kingston, N. (2019). Adaptive testing with a hierarchical item response theory model. Applied Psychological Measurement, 43(1), 51–67. https://doi.org/10.1177/01466 21618 765714
• Way A. (2012). “The use of e-assessments in the Nigerian higher education system,” Turkish Online J. Distance Educ., vol. 13, no. 1, pp. 140–152,
• Weleschuk, A., Dyjur, P. & Kelly, P. (2019). Online assessment in higher education. Taylor Institute for Teaching and Learning Guide Series. https://taylorinstitute.ucalgary.ca/sites/default/files/TI%20Guides/Online%20Assessment%20Guide-2019-10-24.pdf
• Whitelock, D. M., Mackenzie, D., Whitehouse, C., Ruedel, C. & Rae, S. (2006). Identifying innovative and effective practice in e-assessment: findings from seventeen UK case studies. In:Danson, Myles ed. Proceedings of the 10th CAA International Computer Assisted Assessment Conference. Loughborough, UK: Professional Development, Loughborough University, pp. 505–511.
• Williams, M. L. & Pabrock, K. (1999). Distance Learning: The Essencial Guide. Thousand Oaks. CA: Sage Publications. Inc. (2008).
• Yandı, A., Köse, İ. A. & Uysal, Ö. (2017). Farklı Yöntemlerle Ölçme Değişmezliğinin İncelenmesi: Pisa 2012 Örneği. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 2017; 13(1): 243-253. https://doi.org/10.17860/mersinefd.305952
• Zhai, X., Haudek, K., Shi, L., Nehm, R. & Urban‐Lurain, M. (2020). From substitution to redefinition: A framework of machine learning‐based science assessment. J. Res. Sci. Teach, 57(9), 1430-1459. https://doi.org/10.1002/tea.21658