A Latent Class Analysis Approach to Challenges Experienced by Faculty Members in Online Assessment in Higher Education

Öz

Online assessment is the use of computer technologies by faculty members to guide and check learning. Taking the advantage of technology, many universities have used online assessment applications to ensure sustainability in education due to the pandemic and natural disasters. The purpose of the current study is to explore challenges experienced by faculty members in online assessment, using latent class analysis. The descriptive design research was carried out with the participation of 105 faculty members. For the study, the number of latent classes was decided according to the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) and it was observed that the data structure was a good fit for a two-class model. According to the research results, the first class in online assessment applications was considered as the with-difficulty group (58.7 %) and the second as the without-difficulty group (41.3 %). When the conditional probabilities were examined, it was concluded that the observed variables that mostly contributed to the two-class model data structure were as follows, cheating, plagiarism and lack of education policies. It was found that the primary challenges in both groups (with or without difficulty) in online assessment applications were cheating, plagiarism and lack of education policies.

Anahtar Kelimeler

• Challenges Online Assessment
• Latent Class Analysis
• Academik Staff
• Higher Education

Destekleyen Kurum

Not applicable

Etik Beyan

This research received favourable opinion from the Hakkari University’s research ethics committee: reference: 20.12.2022-43035.

Teşekkür

Not applicable

Yüksek Öğrenimde Çevrimiçi Değerlendirmede Fakülte Üyelerinin Yaşadığı Zorluklara Yönelik Gizli Sınıf Analizi Yaklaşımı

Öz

Çevrim içi ölçme ve değerlendirme, öğretim elemanlarının öğrenmeyi yönlendirmek ve denetlemek için bilgisayar teknolojilerini kullanmasıdır. Pandemi ve doğal afetler nedeniyle eğitimde sürdürülebilirliğin sağlanması için birçok üniversite, teknolojinin getirdiği avantajlardan yararlanarak çevrim içi ölçme ve değerlendirme uygulamalarını kullanmışlardır. Betimsel türde tasarlanan bu araştırma 105 öğretim elemanı ile yürütülmüştür. Bu amaç ile geliştirilen ölçme ve değerlendirmeye yönelik zorlukları belirleyen ölçme aracı, öğretim elemanlarına uygulanmış ve sonuçlar örtük sınıf analizi ile incelenmiştir. Araştırmaya Akaike bilgi kriteri (AIC) ve Bayesyen bilgi kriteri (BIC) değerlerine göre örtük sınıf sayısının belirlenmesi ile başlanmış ve veri yapısının iki sınıflı model ile uyumlu olduğu belirlenmiştir. Sınıf sayısı belirlendikten sonra iki sınıflı yapı test edilerek maddelere ait koşullu olasılıklar hesaplanmış ve yorumlanmıştır. Araştırmanın sonuçlarına göre çevrim içi ölçme ve değerlendirme uygulamalarında oluşan sınıflardan birincisinin zorlanan gruba (%58.7) ikincisinin ise zorlanmayan guruba (% 41.3) ait olduğu tespit edilmiştir. Koşullu olasılıklar incelendiğinde, veri yapısının iki sınıflı olmasına en büyük katkıyı sağlayan gözlenen değişkenlerin: kopya çekme, intihal yapma ve eğitim politikalarının eksikliği olduğu sonucuna ulaşılmıştır. Her iki grupta (zorlanan veya zorlanmayan) da çevrim içi ölçme ve değerlendirme uygulamalarında zorlanılan konuların başında kopya çekme, intihal, eğitim politikalarının eksiklikleri olduğu bulunmuştur.

Anahtar Kelimeler

• Çevrimiçi Değerlendirmenin Zorlukları
• Gizli Sınıf Analizi
• Akademik Personel
• Yüksek Öğrenim

Kaynakça

1. Alruwais, N., Wills, G. & Wald, M. (2018). Advantages and challenges of using e-assessment. International Journal of Information and Education Technology, 8(1), 34-37. https://www.ijiet.org/vol8/1008-JR261.pdf
2. Altıntaş, Ö. & Kutlu, Ö. (2020). Investigating Measurement Invariance of Ankara University Foreign Students Selection Test According to Latent Class and Rasch Model, Education and Science, vol.45, no.203, pp.287-308, http://dx.doi.org/10.15390/EB.2020.8685
3. Amante, L., Oliveira, I. R. & Gomes, M. J. (2019). E-assessment in Portuguese higher education: Framework and perceptions of teachers and students. In A. Azevedo & J. Azevedo (Eds.), Handbook of research on e-assessment in higher education (pp. 312–333). IGI Global.
4. Bartholomew, P. M., Knott, M. & Moustaki, I. (2011). Latent variable models and factor analysis a unified approach 3rd edition. West Sussex: John Wiley& Sons, Ltd.
5. Bearman, M., Dawson, P., Ajjawi, R., Tai, J. & Boud, D. (2020). Re-imagining university assessment in a digital world. The Enabling Power of Assessment, 7. Retrieved from https://doi.org/10.1007/978-3-030-41956-1
6. Bensaid, B. & Brahimi, T. (2020). Coping with COVID-19: Higher education in the GCC countries. Springer Proceedings in Complexity, 137–153. https://doi.org/10.1007/978-3-030-62066-0_12
7. Bloom, T. J., Rich, W. D., Olson, S. M. & Adams, M. L. (2018). Perceptions and performance using computer-based testing: One institution's experience. Currents in Pharmacy Teaching and Learning, 10(2), 235–242. https://doi.org/10.1016/j.cptl.2017.10.015
8. Bozkurt, A. & Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), 1-6. https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/447/297

9. Bull, J. & McKenna, C. (2003). A blueprint for computer-assisted assessment. Routledge.
10. Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö.E., Karadeniz, Ş. & Demirel, F. (2012). Bilimsel araştırma yöntemleri. Ankara: Pegem Yayınları
11. Cavus, N. (2015). Distance learning and learning management systems. Procedia-Social and Behavioral Sciences, 191(2), 872-877. https://doi.org/10.1016/j.sbspro.2015.04.611
12. Celeux, G. & Soromenho, G. (1996). An entropy criterion for assessing the number of clusters in a mixture model. Journal of Classification, 13(2), 195-212. https://doi.org/10.1007/BF01246098
13. Collares, C. F. & Cecilio-Fernandes, D. (2019). When I say … computerised adaptive testing. Medical Education, 53(2), 115–116. https://doi.org/10.1111/medu.13648
14. Collins, L. M. & Lanza, S. T. (2010). Latent class andlatent transition analysis with application in the social, behavioral, and health sciences. New Jersey: John Wiley and Sons, Inc.
15. Darling-Aduana, J. (2021). Authenticity, engagement, and performance in online high school courses: Insights from micro-interactional data. Computers & Education, 167, 104175. https://doi.org/10.1016/j.compedu.2021.104175
16. Demir, E. (2014). Uzaktan eğitime genel bir bakış. Dumlupınar Üniversitesi Sosyal Bilimler Dergisi , (39) https://dergipark.org.tr/tr/download/article-file/55935
17. Dermo, J. (2009). e-Assessment and the student learning experience: A survey of student perceptions of e-assessment. British Journal of Educational Technology, 40(2), 203–214. https://doi.org/10.1111/j.1467-8535.2008.00915.x
18. Deutsch, T., Herrmann, K., Frese, T. & Sandholzer, H. (2012). Implementing computer-based assessment A web-based mock examination changes attitudes. Computers & Education, 58(4), 1068–1075. https://doi.org/10.1016/j.compe du.2011.11.013
19. Diprose, M. (2013). Learning and assessment credibility: The design of examination strategies in a changing learning environment. Knowledge Management & E-Learning: An International Journal, 5(1), 104-116. https://doi.org/10.34105/j.kmel.2013.05.008
20. Doğan-Gül, Ç. (2022). TIMSS-2015 araştırmasının dil ve cinsiyete göre ölçme değişmezliğinin gizil sınıf analizi ile incelenmesi.(Yayımlanmamış doktora tezi). Ankara Üniversitesi, Ankara, Türkiye.
21. Dube, T., Zhao, Z. & Ma, M. (2009). E-assessment and design methodology management. http://eprints.hud.ac.uk/id/eprint/23208
22. Eid, M. & Diener, E. (2001). Norms for affect in different cultures: Inter- and intraindividual differences. Journal of Personality and Social Psychology, 81, 869-885. https://doi.org/10.1037/0022-3514.81.5.869
23. Eljinini, M. & Alsamarai, S. (2012). The impact of e-assessments system on the success of the implementation process,” Mod. Educ. Comput. Sci., vol. 4, no. 11, pp. 76–84, Dec.. https://doi.org/10.5815/ijmecs.2012.11.08
24. Elsalem, L., Al-Azzam, N., Jum’ah, A., Obeidat, N., Sindiani, A. & Kheirallah, K. (2020). Stress and behavioral changes with remote E-exams during the Covid- 19 pandemic: A cross-sectional study among undergraduates of medical sciences. Elsevier, 60, 270-280. https://doi.org/10.1016/j.amsu.2020.10.058
25. Fageeh, A. I. (2015). EFL student and faculty perceptions of and attitudes towards online testing in the medium of Blackboard: Promises and challenges. The JALT CALL Journal, 11(1), 41–62. https://doi.org/10.29140/jaltc all.v11n1.183
26. Finch, H. (2015). A comparison of statistics for assessing model invariance in latent class analysis. Open Journal of Statistics, 5, 191-210. https://www.doi.org/10.4236/ojs.2015.53022
27. Flavin M., (2022). A Disruptive Innovation perspective on students’ opinions of online assessment. Research in Learning Technology, 2021, 29: 2611 http://dx.doi.org/10.25304/rlt.v29.2611
28. Gilbert, L., Whitelock, D. & Gale, V. (2011) “Synthesis report on assessment and feedback with technology enhancement,” Southampton.
29. Goodman, L. A. (1974). Exploratory latent structure analysis using identifiable models, Biometrika, 61(2), 215-231. https://doi.org/10.2307/2334349
30. Guangul, F. M., Suhail, A. H., Khalit, M. I. & Khidhir, B. A. (2020). Challenges of remote assessment in higher education in the context of COVID-19: a case study of Middle East College. Educational Assessment, Evaluation and Accountability, 32(4), 519–535. https://doi.org/10.1007/s11092-020-09340-w
31. Güngör, D., Korkmaz, M. & Somer, O. (2013). Çoklu-Grup Örtük Sınıf Analizi ve Ölçme Eşdeğerliği. Türk Psikoloji Dergisi, 28 (72), 48-57. https://www.tpd.com.tr/tr/yayinlar/dergiler/1031828/tpd1300443320130000m000061.pdf
32. Güngör-Çulha, D. (2012). Örtük sınıf analizlerinde ölçme eşdeğerliğinin incelenmesi. (Yayımlanmamış doktora tezi). Ege Üniversitesi, İzmir, Türkiye.
33. Hamsatu, P., Yusufu, G. & Mohammed, H. A. (2016). Teachers' perceptions and undergraduate students' experience in e-exam in higher institution in Nigeria. Journal of Education and Practice, 7(23), 158–166. https://files.eric.ed.gov/fullt ext/EJ111 2920.pdf
34. Heritage, M. (2007). Formative assessment: What do teachers need to know and do? Phi Delta Kappan, 89(2), 140-145. https://doi.org/10.1177/003172170708900210
35. Jamil, M., Tariq, R. H. & Shami, P. A. (2012). Computer-based vs paper-based examinations: Perceptions of university teachers. Turkish Online Journal of Educational Technology, 11(4), 371–381. https://files.eric.ed.gov/fulltext/EJ989302.pdf
36. Jawaid, M., Moosa, F. A., Jaleel, F. & Ashraf, J. (2014). Computer based assessment (CBA): Perception of residents at Dow University of Health Sciences. Pakistan Journal of Medical Sciences, 30(4), 688. doi: http://dx.doi.org/10.12669/pjms.304.5444
37. Jordan, S. & Mitchell, T. (2009). E-assessment for learning? The probability of short-answer free-text questions with tailored feedback. Br. J. Educ. Technol., vol. 40, no. 2, pp. 371–385, Mar. 2009. https://doi.org/10.1111/j.1467-8535.2008.00928.x
38. Jordan, S. (2013). E-assessment: Past, present and future. New Directions in the Teaching of Physical Sciences, (9), 87-106. https://doi.org/10.29311/ndtps.v0i9.504
39. Kankaras, M. & Moors, G. (2009). Measurement equivalence in solidarity attitudes in Europe. Insights from a multiple group latent class factor approach. International Sociology, 24(4), 557-579. https://doi.org/10.1177/0268580909334502
40. Kankaras, M., Moors, G.B.D. & Vermunt, J.K. (2010). Testing for measurement invariance with latent class analysis. In E. Davidov, P. Schmidt, & J. Billiet (Ed.), Cross-cultural analysis. Methods and applications (ss. 359-384). Taylor & Francis Group: Routledge.
41. Karasar, N. (2015). Araştırmalarda Rapor Hazırlama (19. Basım). Nobel Yayınları, ISBN: 978-605-5426-15-6, İstanbul, 152 s.
42. Khan, S. & Khan, R. A. (2019). Online assessments: Exploring perspectives of university students. Education and Information Technologies (2019) 24:661–677 https://doi.org/10.1007/s10639-018-9797-0
43. Khare, A. & Lam, H. (2008). Assessing student achievement and progress with online examinations: some pedagogical and technical issues. International Journal on E-Learning, 7(3), 383–402. https://www.learntechlib.org/primary/p/23620/.
44. Kumar, L. R., Bedra, A. & Karkera, R. (2013). Perception of medical students on e-assessment conducted through Yengage portal. Archives of Medicine and Health Sciences, 1(1), 61. https://doi.org/10.4103/232 1-4848.113577
45. Lazerfeld, P. F. (1950). The logical and mathematics foundatitions of latent structure analysis. in measurement and prediction (ed. Stouffer at all.) Princeton University Press.
46. Lin, H. T. (2006). A comparison of model selection indices for nested latent class models. Monte Carlo Methods and Applications, 12(3), 239-259. https://www.doi.org/10.1515/156939606778705164
47. Linzer, D. A. & Lewis, J. (2007). poLCA: Polytomous Variable Latent Class Analysis. R. JSS Journal of Statistical Software, VV(II), https://www.sscnet.ucla.edu/polisci/faculty/lewis/pdf/poLCA-JSS-final.pdf
48. Lockman, A. S. & Schirmer, B. R. (2020). Online instruction in higher education: Promising, research-based, and evidence-based practices. Journal of Education and E-Learning Research, 7(2), 130–152. https://doi.org/10.20448/journal.509.2020.72.130.152
49. McCutcheon, A. L. (1987). Latent class analysis. Sage University Papers Series.
50. Middleton, K. V. (2022). Considerations for Future Online Testing and Assessment in Colleges and Universities. Educational Measurement: Issues and Practice. Spring 2022, Vol. 41, No. 1, pp. 51–53. https://doi.org/10.1111/emip.12497
51. Mirza, H. (2021). University Teachers’ Perceptions of Online Assessment during the Covid-19 Pandemic in Lebanon, American Academic & Scholarly Research Journal, 13(1). https://aasrc.org/aasrj/index.php/aasrj/article/view/2064/1187
52. Mitchell, T., Aldridge, N., Williamson, W. & Broomhead, P. (2003). Computer based testing of medical knowledge. Presented at the 7th Computer Assisted Assessment Conference.
53. Momeni, A. (2022). Online Assessment in Times of COVID-19 Lockdown: Iranian EFL Teachers' Perceptions, International Journal of Language Testing. Vol. 12, No. 2, October 2022. https://files.eric.ed.gov/fulltext/EJ1363125.pdf
54. Montenegro-Rueda, M., Luque-de la Rosa, A., Sarasola Sánchez-Serrano, J.L. & FernándezCerero, J. (2021). Assessment in Higher Education during the COVID-19 Pandemic: A Systematic Review. Sustainability, 13, 10509. https://doi.org/10.3390/su131910509
55. Moors, G. & Wennekers, C. (2003). Comparing moral values in western european countries between 1981 and 1999. A multigroup latent-class factor approach. International Journal of Comparative Sociology, 44(1):155-172. https://doi.org/10.1177/002071520304400203
56. Oosterhof, A., Conrad, R. M. & Ely, D. P. (2008). Assessing learners online. New Jersey: Pearson.
57. Osman, M. E. (2020). Global impact of COVID-19 on education systems: The emergency remote teaching at Sultan Qaboos University. Journal of Education for Teaching, 46(4), 463-471. https://doi.org/10.1080/02607476.2020.1802583
58. Peterson, D. (2013). Five steps to better tests. Questionmark White Paper. Retrieve from https://www.questionmark.com/test-design-and-delivery-overview/
59. R Development Core Team (2007). R: a language and environment for statistical computing, r foundation for statistical computing, Vienna, Austria. ISBN 3-900051-07-0.
60. Ridgway, J., McCusker, S. & Pead, D. (2004). “Literature review of e-assessment,” Bristol. http://www.futurelab.org.uk/resources/publications-reports-articles/literature-reviews/Literature-Review204
61. Rolim, C. & Isaias, P. (2018). Examining the use of e-assessment in higher education: Teachers and students’ viewpoints. British Journal of Educational Technology, 50(4), 1785–1800. https://doi.org/10.1111/bjet.12669
62. Sa'di, R. A., Abdelraziq, A. & Sharadgah, T. A. (2021). E-Assessment at Jordan’s Universities in the Time of the COVID-19 Lockdown: Challenges and Solutions. Arab World English Journal (AWEJ) Special Issue on Covid 19 Challenges (1) 37-54. DOI: https://dx.doi.org/10.24093/awej/covid.3
63. Schneberger, S., Amoroso, D. L. & Durfee, A. (2008). Factors that influence the performance of computer-based assessments: An extension of the technology acceptance model. Journal of Computer Information Systems, 48(2), 74–90. https://doi.org/10.1080/08874417.2008.11646011
64. Şenel, S. & Şenel, H. C. (2021). Remote Assessment in Higher Education during COVID-19 Pandemic. International Journal of Assessment Tools in Education, 8 (2), 181-199 . DOI: 10.21449/ijate.820140
65. Silvia, P. J., Kaufman, J. C. & Pretz, J. E. (2009). Is creativity domain-specific? Latent class models of creative accomplishments and creative self-descriptions. Psychology of Aesthetics, Creativity, and the Arts, 139-148. https://libres.uncg.edu/ir/uncg/f/P_Silvia_Is_2008.pdf
66. Sim, G., Holifield, P. & Brown, M. (2004). Implementation of computer assisted assessment: Lessons from the literature. ALT-J, 12(3), 215-229. https://doi.org/10.3402/rlt.v12i3.11255
67. Stiggins, R.J. (1992) High quality classroom assessment: what does it really mean? Educ Meas Issues Pract 11(2):35–39. https://doi.org/10.1111/j.1745-3992.1992.tb00241.x
68. Stone, D. E. & Zheng, G. (2014). Learning management systems in a changing environment. In Handbook of research on education and technology in a changing society (pp. 756-767). IGI Global. https://www.doi.or/10.4018/978-1-4666-6046-5.ch056
69. Terzis, V. & Economides, A. A. (2011). Computer based assessment: Gender differences in perceptions and acceptance. Computers in Human Behavior, 27(6), 2108–2122. https://doi.org/10.1016/j.chb.2011.06.005
70. Terzis, V., Moridis, C. N. & Economides, A. A. (2013a). Continuance acceptance of computer-based assessment through the integration of user's expectations and perceptions. Computers & Education, 62, 50–61. https://doi.org/10.1016/j.compe du.2012.10.018
71. Terzis, V., Moridis, C. N., Economides, A. A. & Mendez, G. R. (2013b). Computer based assessment acceptance: A cross-cultural study in Greece and Mexico. Journal of Educational Technology & Society, 16(3), 411–424. https://www.jstor.org/stabl e/jeduc techs oci.16.3.411
72. Vermunt, J. K. & Madigson, J. (2004). Local Indepence. In A. B. M.S. Lewis Beck, The Sage Encyclopedia of Social Sciences Research Methods. 732-733.
73. Vurdien, R. & Puranen, P. (2022). Teacher attitudes toward online assessment in challenging times. In B. Arnbjörnsdóttir, B. Bédi, L. Bradley, K. Friðriksdóttir, H. Garðarsdóttir, S. Thouësny, & M. J. Whelpton (Eds), Intelligent CALL, granular systems, and learner data: short papers from EUROCALL 2022 (pp. 370-374). https://doi.org/10.14705/rpnet.2022.61.1486
74. Wang M.-C., Deng Q., Bi X., Ye H. & Yang W. (2017). Performance of the entropy as an index of classification accuracy in latent profile analysis: A Monte Carlo simulation study. Acta Psychologica Sinica, 49(11), 1473-1482. https://doi.org/10.3724/SP.J.1041.2017.01473
75. Wang, W. & Kingston, N. (2019). Adaptive testing with a hierarchical item response theory model. Applied Psychological Measurement, 43(1), 51–67. https://doi.org/10.1177/01466 21618 765714
76. Way A. (2012). “The use of e-assessments in the Nigerian higher education system,” Turkish Online J. Distance Educ., vol. 13, no. 1, pp. 140–152,
77. Weleschuk, A., Dyjur, P. & Kelly, P. (2019). Online assessment in higher education. Taylor Institute for Teaching and Learning Guide Series. https://taylorinstitute.ucalgary.ca/sites/default/files/TI%20Guides/Online%20Assessment%20Guide-2019-10-24.pdf
78. Whitelock, D. M., Mackenzie, D., Whitehouse, C., Ruedel, C. & Rae, S. (2006). Identifying innovative and effective practice in e-assessment: findings from seventeen UK case studies. In:Danson, Myles ed. Proceedings of the 10th CAA International Computer Assisted Assessment Conference. Loughborough, UK: Professional Development, Loughborough University, pp. 505–511.
79. Williams, M. L. & Pabrock, K. (1999). Distance Learning: The Essencial Guide. Thousand Oaks. CA: Sage Publications. Inc. (2008).
80. Yandı, A., Köse, İ. A. & Uysal, Ö. (2017). Farklı Yöntemlerle Ölçme Değişmezliğinin İncelenmesi: Pisa 2012 Örneği. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 2017; 13(1): 243-253. https://doi.org/10.17860/mersinefd.305952
81. Zhai, X., Haudek, K., Shi, L., Nehm, R. & Urban‐Lurain, M. (2020). From substitution to redefinition: A framework of machine learning‐based science assessment. J. Res. Sci. Teach, 57(9), 1430-1459. https://doi.org/10.1002/tea.21658