Use of take-home exam for remote assessment: A case study from Turkey

Abstract

COVID-19 has changed the way we teach. Today, we have become far more experienced in the delivery of distance education and use of online tools. However, the quality of distance education and learning outcomes have become a matter of ongoing debate. Just as higher education aims to develop high-level skills in its students, researchers are seeking ways to perform valid and reliable assessment in distance education. Institutions and educators are also in search of assessment tools that can help prevent instances of cheating and plagiarism. However, performance-based assessment tools may also offer options to measure both high-level skills and in limiting cheating behaviors. In this study, we used the take-home exam as a formative remote assessment tool as a local case in Turkey. We surveyed the views of 43 undergraduate students about the quality of take-home exams as a remote assessment tool. The results showed that participants had a high quality perception about the use of take-home exam, especially with regards to being kept informed about evaluation and scoring, rapid assessment, the provision of feedback, and consistency of scope between assessment and course content. Whereas students highly perceived the use of take-home exams, they reported more moderate views regarding take-home exams increasing the level of interaction with their peers. The results of this study suggest that the use of take-home exams is significantly preferred by higher education students, that it is a reliable and distinctive way to measure students’ academic performance, and may increase student-teacher interaction through its formative use.

Keywords

• Remote assessment
• Take-home exam
• COVID-19
• Formative assessment
• E-assessment
• Online assessment

References

1. Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. M. (2011). Interaction in distance education and online learning: Using evidence and theory to improve practice. Journal of Computing in Higher Education, https://doi.org/10.1007/s12528-011-9043-x
2. Al-Balas, M., Al-Balas, H. I., Jaber, H. M., Obeidat, K., Al-Balas, H., Aborajooh, E. A., et al. (2020). Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: Current situation, challenges, and perspectives. BMC Medical Education, https://doi.org/10.1186/s12909-020-02257-4
3. Atılgan, H., Kan, A., & Doğan, N. (2009). Eğitimde ölçme ve değerlendirme [Measurement and evaluation in education]. Ankara, Turkey: Anı Yayıncılık.
4. Bengtsson, L. (2019). Take-Home Exams in Higher Education: A Systematic Review. Education Sciences, https://doi.org/10.3390/educsci9040267
5. Bıkmaz Bilgen, Ö., & Doğan, N. (2017). The Comparison of Interrater Reliability Estimating Techniques. Journal of Measurement and Evaluation in Education and Psychology, https://doi.org/10.21031/epod.294847
6. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, https://doi.org/10.1007/s11092-008-9068-5
7. Boud, D., Cohen, R., & Sampson, J. (1999). Peer learning and assessment. Assessment and Evaluation in Higher Education, https://doi.org/10.1080/0260293990240405
8. Bouhajeb, M., Mefteh, H., & Ben Ammar, R. (2018). Higher education and economic growth: the importance of innovation. Atlantic Review of Economics, 1(2).

9. Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, https://doi.org/10.1111/j.2044-8279.1988.tb00874.x
10. Butler-Henderson, K., & Crawford, J. (2020). A systematic review of online examinations: A pedagogical innovation for scalable authentication and integrity. Computers and Education, https://doi.org/10.1016/j.compedu.2020.104024
11. Chaiyo, Y., & Nokham, R. (2017). The effect of Kahoot, Quizizz and Google Forms on the student's perception in the classrooms response system. In 2nd Joint International Conference on Digital Arts, Media and Technology 2017: Digital Economy for Sustainable Growth (pp. 178-182). IEEE. https://doi.org/10.1109/ICDAMT.2017.7904957
12. Crawford, J., Butler-Henderson, K., Rudolph, J., Malkawi, B., Glowatz, M., Burton, R., et al. (2020). COVID-19: 20 countries' higher education intra-period digital pedagogy responses. Journal of Applied Learning & Teaching, http://dx.doi.org/10.37074/jalt.2020.3.1.7
13. Dawson, P. (2020). Strategies for using online invigilated exams. Tertiary Education Quality and Standards Agency. https://tlu.cit.ie/contentFiles/files/strategies-for-using-online-invigilated-exams.pdf
14. Dendir, S., & Maxwell, R. S. (2020). Cheating in online courses: Evidence from online proctoring. Computers in Human Behavior Reports, https://doi.org/10.1016/j.chbr.2020.100033
15. Erkuş, A. (2006). Sınıf öğretmenleri için ölçme ve değerlendirme: Kavramlar ve uygulamalar [Measurement and evaluation for classroom teachers: Concepts and practices]. Ankara, Turkey: Ekinoks.
16. Frey, B. B. (2018). Test security. In B. B. Frey (Ed.), The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781506326139.n695
17. Gharib, A., Phillips, W., & Mathew, N. (2012). Cheat Sheet or Open-Book? A Comparison of the Effects of Exam Types on Performance, Retention, and Anxiety. Psychology Research, https://doi.org/10.17265/2159-5542/2012.08.004
18. Hall, L. (2001). Take-Home Tests: Educational Fast Food for the New Millennium? Journal of Management & Organization, https://doi.org/10.5172/jmo.2001.7.2.50
19. Harmon, O., Lambrinos, J., & Buffolino, J. (2010). Is the Cheating Risk Always Higher in Online Instruction Compared to Face-to-Face Instruction? Working Papers 2008-14, University of Connecticut, Department of Economics. https://ideas.repec.org/p/uct/uconnp/2008-14.html
20. Harper, B. (2018). Technology and Teacher–Student Interactions: A Review of Empirical Research. Journal of Research on Technology in Education, https://doi.org/10.1080/15391523.2018.1450690
21. Higgins, R., Hartley, P., & Skelton, A. (2002). The conscientious consumer: Reconsidering the role of assessment feedback in student learning. Studies in Higher Education, https://doi.org/10.1080/03075070120099368
22. Howell, S. L., Sorensen, D., & Tippets, H. R. (2009). The New (and Old) News about Cheating for Distance Educators. Online Journal of Distance Learning Administration, 12(3).
23. Hricko, M., & Howell, S. L. (Eds.) (2005). Online assessment and measurement: Foundations and challenges. IGI Global. https://doi.org/10.4018/978-1-59140-720-1
24. Institute of Education Sciences. (2020). Best Practices for Creating Take-Home Packets to Support Distance Learning. https://doi.org/10.17226/24677
25. Ilgaz, H., Afacan Adanır, G. (2020). Providing online exams for online learners: Does it really matter for them?. Educ Inf Technol. https://doi.org/10.1007/s10639-019-10020-6
26. Jain, A. K., Bolle, R., & Pankanti, S. (Eds.) (2006). Biometrics: personal identification in networked society (Vol. 479). New York, NY: Springer.
27. Johnson, C. M., Green, K. A., Galbraith, B. J., & Anelli, C. M. (2015). Assessing and Refining Group Take-Home Exams as Authentic, Effective Learning Experiences. Journal of College Science Teaching, 44(5), 61–71.
28. Jung, I., Choi, S., Lim, C., & Leem, J. (2002). Effects of different types of interaction on learning achievement, satisfaction and participation in web-based instruction. Innovations in Education and Teaching International, https://doi.org/10.1080/14703290252934603
29. Knight, P. T. (2002). Studies in Higher Education Summative Assessment in Higher Education : Practices in disarray. Studies in Higher Education, https://doi.org/10.1080/03075070220000662
30. Krippendorff, K. (1995). On the reliability of unitizing continuous data. Sociological Methodology, https://doi.org/10.2307/271061
31. Kutlu, Ö., Doğan, C. D., & Karakaya, İ. (2008). Öğrenci başarısının belirlenmesi: Performansa ve portfolyoya dayalı durum belirleme [Determining student success: Determination based on performance and portfolio]. Ankara, Turkey: Pegem Akademi.
32. Kyllonen, P. C. (2012). Measurement of 21st century skills within the Common Core State Standards. Invitational Research Symposium on Technology Enhanced Assessments. https://oei.org.ar/ibertic/evaluacion/sites/default/files/biblioteca/11_measurement_of_21stcenturyskills.pdf
33. London School of Economics and Political Science. (2020). Take-home assessment. https://info.lse.ac.uk/staff/divisions/Eden-Centre/Assessment-Toolkit/Assessment-conditions/Take-home-assessment
34. López, D., Cruz, J. L., Sánchez, F., & Fernández, A. (2011). A take-home exam to assess professional skills. In Proceedings - Frontiers in Education Conference. https://doi.org/10.1109/FIE.2011.6142797
35. Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass.
36. Mohanna, K., & Patel, A. (2016). Overview of open book-open web exam over blackboard under e-learning system. In Proceedings - 2015 5th International Conference on e-Learning (pp. 396-402). New York, NY: IEEE. https://doi.org/10.1109/ECONF.2015.81
37. Nutbrown, S., Higgins, C., & Beesley, S. (2016). Measuring the impact of high quality instant feedback on learning. Practitioner Research in Higher Education, 10(1), 130-139.
38. Özdin, S., & Bayrak Özdin, Ş. (2020). Levels and predictors of anxiety, depression and health anxiety during COVID-19 pandemic in Turkish society: The importance of gender. International Journal of Social Psychiatry, https://doi.org/10.1177/0020764020927051
39. Peterson, J. (2019). An analysis of academic dishonesty in online classes. Mid-Western Educational Researcher, 31(1), 24-36.
40. Rich, J. D., Colon, A. N., Mines, D., & Jivers, K. L. (2014). Creating learner-centered assessment strategies or promoting greater student retention and class participation. Frontiers in Psychology, https://doi.org/10.3389/fpsyg.2014.00595
41. Rovai, A. P. (2000). Online and traditional assessments: What is the difference? Internet and Higher Education, https://doi.org/10.1016/S1096-7516(01)00028-8
42. Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: SAGE Publications.
43. Sankar, J. R. K., John, J., Menon, N., Elumalai, K., Alqahtani, M., & Abumelha, M. (2020). Factors Affecting the Quality of E-Learning During the COVID-19 Pandemic from the Perspective of Higher Education Students. Journal of Information Technology Education: Research, 19(1), 731-753.
44. Schweig, J., McEachin, A., & Kuhfeld, M. (2020). Addressing COVID-19’s Disruption of Student Assessment. Inside IES Research. https://ies.ed.gov/blogs/research/post/addressing-covid-19-s-disruption-of-student-assessment
45. Şenel, S. (2020). Ölçme ve değerlendirmede teknoloji kullanımı [Technology use in measurement and evaluation]. In A. M. Güneş & E. Yünkül (Eds.), Dijital Çağda Lider Sınıf Öğretmeni Olmak [Becoming a leading classroom teacher in digital age] (pp. 223-271). Ankara, Turkey: Nobel Academic.
46. Senel, S., & Senel, H. C. (2021). Remote Assessment in Higher Education during COVID-19 Pandemic. International Journal of Assessment Tools in Education, https://doi.org/10.21449/ijate.820140
47. Seow, T. K., Kit, S., & Soong, A. (2014). Students’ perceptions of BYOD open-book examinations in a large class: a pilot study. In Proceedings of Rhetoric and Reality: Critical perspectives on educational technology (pp. 604-608). Dunedin, New Zealand: Ascilite2014.
48. Stödberg, U. (2012). A research review of e-assessment. Assessment & Evaluation in Higher Education, https://doi.org/10.1080/02602938.2011.557496
49. Sullivan, D. P. (2016). An Integrated Approach to Preempt Cheating on Asynchronous, Objective, Online Assessments in Graduate Business Classes. Online Learning, http://dx.doi.org/10.24059/olj.v20i3.650
50. Test Security - Conference on Test Security. (n.d.). Retrieved January 3, 2021, from https://conferenceontestsecurity.org/ University of Guelph-Office of Teaching and Learning. (2020). Top 5 Tips for Take Home Exams. https://otl.uoguelph.ca/top-5-tips-take-home-exams
51. Vazquez, J. J., Chiang, E. P., & Sarmiento-Barbieri, I. (2021). Can we stay one step ahead of cheaters? A field experiment in proctoring online open book exams. Journal of Behavioral and Experimental Economics, https://doi.org/10.1016/j.socec.2020.101653
52. Woldeab, D., Brothen, T. (2019). 21st Century assessment: Online proctoring, test anxiety, and student performance. The International Journal of E-Learning & Distance Education, 34(1), 1-10.
53. Yin, R. K. (2002). Case study research: Design and methods. Thousand Oaks, CA: SAGE Publications.
54. Zoller, U., & Ben-Chaim, D. (1989). Interaction between examination type, anxiety state, and academic achievement in college science; an action-oriented research. Journal of Research in Science Teaching, https://doi.org/10.1002/tea.3660260107