Using structural equation modeling to understand the determinants that drive instructors’ use of online proctoring

Abstract

Despite the increasing use of online proctoring, prior research has not fully explored the factors influencing instructors’ adoption of these tools, nor clarified their role in promoting academic integrity. This study addresses these gaps by investigating instructors’ perceptions of online proctoring to uncover factors that underpin decisions to adopt online proctoring as an academic integrity tool. Using the Unified Theory of Acceptance and Use of Technology framework to examine determinants that influence the intended use of online proctoring, an online survey was completed by 158 instructors at various higher education institutions. Using structural equation modeling, the study found that performance expectancy is the primary determinant of an instructor’s intention to use online proctoring while effort expectancy has no significant impact. Interestingly, social influence also has a significant impact but only for instructors who have moderate to no online teaching experience. These findings suggest that institutions should focus on communicating the integrity benefits of online proctoring, provide clear guidelines for its implementation, and offer support for interpreting proctoring results. Additionally, institutions should address student privacy and anxiety concerns, especially when supporting novice online instructors. By tailoring policies and resources to these determinants, educators and institutions can make more informed decisions about the adoption and management of online proctoring.

Keywords

• online practoring
• academic integrity
• online teaching
• technology acceptance

References

1. Alessio, H. M., & Messinger, J. D. (2021). Faculty and student perceptions of academic integrity in technology-assisted learning and testing. Frontiers in Education, 6(2021), 1–6. https://doi.org/10.3389/feduc.2021.629220
2. Amigud, A., & Lancaster, T. (2019). 246 reasons to cheat: An analysis of students’ reasons for seeking to outsource academic work. Computers & Education, 134, 98–107. https://doi.org/10.1016/j.compedu.2019.01.017
3. Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94.
4. Bedford, W., Gregg, J., & Clinton, S. (2009). Implementing technology to prevent online cheating: A case study at a small southern regional university (SSRU). MERLOT Journal of Online Learning and Teaching, 5(2), 230–238.
5. Bedford, W., Gregg, J., & Clinton, S. (2011). Preventing online cheating with technology: A pilot study of Remote Proctor and an update of its use. Journal of Higher Education Theory and Practice, 11(2), 41–59.
6. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588.
7. Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford Publications.
8. Casey, K., Casey, M., & Griffin, K. (2018). Academic integrity in the online environment: Teaching strategies and software that encourage ethical behavior. International Conference Proceedings of the Institute for Global Business Research, 2, 58–66.

9. Dunn, T. P., Meine, M. F., & McCarley, J. (2010). The remote proctor: An innovative technological solution for online course integrity. International Journal of Technology, Knowledge & Society, 6(1), 1–7. https://doi.org/10.18848/1832-3669/CGP/v06i01/56033
10. Faucher, D., & Caves, S. (2009). Academic dishonesty: Innovative cheating techniques and the detection and prevention of them. Teaching and Learning in Nursing, 4(2), 37–41. https://doi.org/10.1016/j.teln.2008.09.003
11. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed). McGraw-Hill Humanities/Social Sciences/Languages.
12. Grijalva, T. C., Nowell, C., & Kerkvliet, J. (2006). Academic honesty and online courses. College Student Journal, 40(1), 180–185.
13. Hair, J. F., Sarstedt, M., Pieper, T. M., & Ringle, C. M. (2012). The use of partial least squares structural equation modeling in strategic management research: A review of past practices and recommendations for future applications. Long Range Planning, 45(5–6), 320–340.
14. Harris, L., Harrison, D., McNally, D., & Ford, C. (2020). Academic integrity in an online culture: Do McCabe’s findings hold true for online, adult learners? Journal of Academic Ethics, 18(2020), 419–434. https://doi.org/10.1007/s10805-019-09335-3
15. Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135.
16. Hollister, K. K., & Berenson, M. L. (2009). Proctored versus unproctored online exams: Studying the impact of exam environment on student performance. Decision Sciences Journal of Innovative Education, 7(1), 271–294.
17. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424.
18. King, C. G., Guyette, R. W., & Piotrowski, C. (2009). Online exams and cheating: An empirical analysis of business students’ views. Journal of Educators Online, 6(1). https://eric.ed.gov/?id=EJ904058
19. Kitahara, R., Westfall, F., & Mankelwicz, J. (2011). New, multi-faceted hybrid approaches to ensuring academic integrity. Journal of Academic and Business Ethics, 3(1), 1–12.
20. Lanier, M. (2006). Academic integrity and distance learning. Journal of Criminal Justice Education, 17(2), 244–261. https://doi.org/10.1080/10511250600866166
21. LoSchiavo, F. M., & Shatz, M. A. (2011). The impact of an honor code on cheating in online courses. Journal of Online Learning and Teaching, 7(2), 179–184.
22. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130.
23. Marsh, H. W., & Hocevar, D. (1985). Application of confirmatory factor analysis to the study of self-concept: First-and higher order factor models and their invariance across groups. Psychological Bulletin, 97(3), 562.
24. McMurtrie, B. (2023, February 2). Rethinking research papers and other responses to ChatGPT. The Chronicle of Higher Education. https://www.chronicle.com/newsletter/teaching/2023-02-02
25. McNabb, L., & Olmstead, A. (2009). Communities of integrity in online courses: Faculty member beliefs and strategies. Journal of Online Learning and Teaching, 5(2), 208–221.
26. Medina, M. S., & Castleberry, A. N. (2016). Proctoring strategies for computer-based and paper-based tests. American Journal of Health-System Pharmacy, 73(5), 274–277. https://doi.org/10.2146/ajhp150678
27. Mintz, S. (2023, January 16). ChatGPT: Threat or menace? Inside HigherEd. https://www.insidehighered.com/blogs/higher-ed-gamma/chatgpt-threat-or-menace
28. Mitrano, T. (2023, January 17). Coping with ChatGPT. Inside HigherEd. https://www.insidehighered.com/blogs/law-policy%E2%80%94and-it/coping-chatgpt
29. Nadelson, S. (2006). The role of the environment in student ethical behavior. Journal of College and Character, 7(5). https://doi.org/10.2202/1940-1639.1195
30. Nelson, D. (2021). How online business school instructors address academic integrity violations. Journal of Educators Online, 18(3). https://www.thejeo.com/archive/2021_18_3/nelson
31. Newton, D. (2015, November 4). Cheating in online classes is now big business. The Atlantic. https://www.theatlantic.com/education/archive/2015/11/cheating-through-online-courses/413770/
32. Raman, R., Sairam, B., Veena, G., Vachharajani, H., & Nedungadi, P. (2021). Adoption of online proctored examinations by university students during COVID-19: Innovation diffusion study. Education and Information Technologies, 26(2021), 7339–7358. https://doi.org/10.1007/s10639-021-10581-5
33. Rodchua, S., Yiadom-Boakye, G., & Woolsey, R. (2011). Student verification system for online assessments: Bolstering quality and integrity of distance learning. Journal of Industrial Technology, 27(3), 1–8.
34. Rogers, E. (1995). Diffusion of innovations (4th ed.). Free Press.
35. Rowe, N. C. (2004). Cheating in online student assessment: Beyond plagiarism. On-Line Journal of Distance Learning Administration, Summer, 1–8.
36. Schaffhauser, D. (2017, July 26). Nobody’s watching: Proctoring in online learning. Campus Technology. https://campustechnology.com/articles/2017/07/26/nobodys-watching-proctoring-in-online-learning.aspx
37. Soper, D. S. (2022). Structural equation model sample size calculator [Computer software]. https://www.danielsoper.com/statcalc
38. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478. https://doi.org/10.2307/30036540
39. Verhoef, A. H., & Coetser, Y. M. (2021). Academic integrity of university students during emergency remote online assessment: An exploration of student voices. Transformation in Higher Education, 6(0), a132. https://doi.org/10.4102/the.v6i0.132
40. Vinzi, V. E., Chin, W. W., Henseler, J., & Wang, H. (Eds.). (2010). Handbook of partial least squares (Vol. 201). Springer.
41. Williams, M. D., Rana, N. P., & Dwivedi, Y. K. (2015). The unified theory of acceptance and use of technology (UTAUT): A literature review. Journal of Enterprise Information Management, 28(3), 443–488. https://doi.org/10.1108/JEIM-09-2014-0088
42. Worthen, B. R., White, K. R., Fan, X., & Sudweeks, R. R. (1998). Measurement and assessment in schools (2nd ed.). Allyn & Bacon.
43. Wuthisatian, R. (2020). Student exam performance in different proctored environments: Evidence from an online economics course. International Review of Economics Education, 35(November 2020), 100196. https://doi.org/10.1016/j.iree.2020.100196
44. Yates, R. W., & Beaudrie, B. (2009). The impact of online assessment on grades in community college distance education mathematics courses. American Journal of Distance Education, 23(2), 62–70. .