COMPARISON OF OPINIONS OF STUDENTS AND UNIVERSITY TEACHERS FROM MEDICAL UNIVERSITY OF WARSAW ON E-ASSESSMENT – A PRELIMINARY REPORT

Year 2016, Volume: 5 , 1 - 9, 01.09.2016

Joanna Gotlib Mariusz Panczyk Piotr Gębski Aleksander Zarzeka Jarosława Belowska Marcin Malczyk

Abstract

Background Until now, test exams at Warsaw Medical University were mostly
carried out in the paper-and-pen test form. Beginning in the academic year
2014-2015, it is possible to get a credit and pass a test exam on the ASK
Systems e-exam platform. In the academic year 2014-2015 there were three
e-exams organised in WMU.The aim of the study The study aimed to compare
opinion of students and university teachers from Medical University of Warsaw
on e-assessment.Material Study group: 181 persons - 148 students: students, who
took part in e-assessment on ASK Systems E-exam platfrom, (Group 1 - 59
students), and students, who did not take part in e-assessment (Group 2 - 89
students) and 33 university teachers (Group 3). Group 1 - 59 women, mean age:
24,96 (min.21, max.44, SD: 5,54). 25 of students has alredy took part in an
e-exam apart from university. Group 2 – 55 women, 25 men, mean age: 22,53;
min.19, max.32; SD: 2,15, 80 of students has alredy took part in an e-exam
apart from university. Group 3 - mean age: 46,57; min. 29, max. 61, SD=10,99;
14 of women, 15 teachers took part in e-exam. Methods Participation in the
study (February 22-27, 2015) was voluntary. An anonymus questionnaire survey.
E-questionnaire consisted of 58 statements in 5 thematic domains, 5 step Likert
scale. Links to the questionnaire were placed on an external server, which
ensured anonymity. Reliability assessment of the questionnaire: Cronbach's
Alpha coefficient. Analysis of significance of differences between Groups 1, 2,
and 3: non-parametric Kruskal-Wallis test, α=0.05. Results Among the study
groups, statistically significant differences of opinions and attitudes towards
e-exams concerned the following: phrasing questions that would be impossible to
phrase in the case of a traditional pen-and-paper test (H=9.661,p<0.008),
additional intellectual effort required from students during an e-exam
(H=13.283,p<0.001), trends in methods of testing knowledge of students
(H=6.366, p<0.042), results of course credits earned on an e-exam platform
(H=6.531, p<0.038), possibility of technical error in the examination system
(H=7.834, p<0.020), increase of examination stress during an e-exam
(H=17.689, p<0.001), concentration problems when reading a question on a
computer screen (H=6.604, p<0.037), drawing an individual set of test
questions (H=11.634, p<0.003), using multimedia materials in test questions
(H=9.798, p<0.008), reducing the opportunity of cheating on exams (H=10.483,
p<0.005), and increasing chances for a better result in a test (H=6.891, p<0.032).
Conclusions:Due to the fact that students had correct conception of
organization of e-exams and positive attitudes towards EE, there is no need of
organising special preparation for students regarding computer-aided
examinations. It seems to be essential to develop procedures of grading
computer-aided exams even before the introduction of this form of examination.
Moreover, it also seems necessary to conduct training for university teachers
on possibilities that are provided by the ASK Systems e-exam platform with
reference to test exam structuring and phrasing test questions, with particular
focus on multimedia elements. Owing to a not numerous and non-representative
study group, the present study results may be only of a pilot nature and the
study needs to be continued in consecutive years.

Keywords

E-assessment, quality of assessment, modern technologies, health sciences students, university teachers

References

• Cantillon P, Irish B, Sales D. (2004). Using computers for assessment in medicine. BMJ; 329: 606-609. Conole G, Warburton B. (2005). A review of computer-assisted assessment ALT-J. Research in Learning Technology; 13(1): 17-31. Dennick R, Wilkinson S, Purcell N. (2009). Online e-Assessment: AMEE Guide No. 39 Medical Teacher; 31: 192-206. Hewson C. (2012). Can online course-based assessment methods be fair and equitable? Relationships between students’ preferences and performance within online and offline assessments. Journal of Computer Assisted Learning; 28: 488-498. Mooney GA, Bligh JG, Leinster SJ. (1998). Some techniques for computer-based assessment in medical education. Medical Teacher; 20(6): 560-566. Feldt LS. (1969). A test of hypothesis that Cronbachs alpha or Kuder-Richardson coefficent 20 is same for 2 tests. Psychometrika;34(3):363. Cronbach LJ, Meehl PE. (1955) Construct validity in psychological tests. Psychological bulletin.;52(4):281. Nunnally JC, Bernstein IH. (1967). Psychometric theory. 3 ed. New York: McGraw-Hill. Jankowski K, Zajenkowski M. (2009). Metody szacowania rzetelności pomiaru testem. In: Fronczyk K, editor. Psychometria - podstawowe zagadnienia. Warszawa: Vizja Press & IT; p. 84-110. Jawaid M, Moosa FA, Jaleel F, Ashraf J. (2014). Computer Based Assessment (CBA): Perception of residents at Dow University of Health Sciences. Pak J Med Sci; 30(4): 688-691 Hassanien MA, Al-Hayani A, Abu-Kamer R, Almazrooa A. (2013). A six step approach for developing computer based assessment in medical education. Medical Teacher; 35: 15-19. Rudland JR, Schwartz P, Ali A. (2011). Moving a formative test from a paper-based to a computer-based format. A student viewpoint. Medical Teacher; 33: 738-743. Dermo J. (2009). e-Assessment and the student learning experience: A survey of student perceptions of e-assessment. British Journal of Educational Technology; 40(2): 203-214. Hochlehnert A, Brass K, Moeltner A, Juenger J. (2011). Does Medical Students’ Preference of Test Format (Computer-based vs. Paper-based) have an Influence on Performance? BMC Medical Education; 11 (89): 1-6. Gotlib J, Zarzeka A, Panczyk M, Malczyk M. (2015). Zaliczenie testowe z przedmiotu „Prawo w medycynie” dla studentów Wydziału Nauki o Zdrowiu na platformie egzaminów elektronicznych ASK Systems – doświadczenia własne. Medycyna Dydaktyka Wychowanie; 1: 28-30. Gotlib J, Panczyk M, Gębski P, Zarzeka A, Iwanow L, Dąbrowski F, Dykowska G, Malczyk M. (2015). Analiza opinii studentów Warszawskiego Uniwersytetu Medycznego na temat udziału w zaliczeniach i egzaminach elektronicznych – doniesienie wstępne. Polish Journal of Polish Health; 1: in print. Gotlib J, Panczyk M, Gębski P, Zarzeka A, Iwanow L, Dąbrowski F, Dykowska G, Malczyk M. (2015) Analiza porównawcza opinii studentów Warszawskiego Uniwersytetu Medycznego na temat egzaminów i zaliczeń elektronicznych zależności od ich udziału w takiej formie sprawdzania wiedzy – doniesienie wstępne. Polish Journal of Polish Health; 1: in print.