An evaluation of the small group study in the critical appraisal special study module

Abstract

INTRODUCTION Critical appraisal is one of the steps towards evidence-based medicine. Today, the production of new knowledge is quite rapid and its evaluation is becoming increasingly difficult. To overcome these difficulties, educational activities incorporating critical appraisal practices in medical education programmes have found a place in curricula around the world. The aim of this study is to determine the opinions of trainers and students about the practice that is included in the Critical Appraisal Special Study Module (CA-SSM), which is carried out as a structured programme in Dokuz Eylül University Faculty of Medicine (DEUFM) and is carried out as a small group study. METHOD The population of the study consisted of 417 students who took CA-SSM in DEUFM Term 2 in the academic year 2023-2024 and 32 trainers who participated in the exercises. The opinions of the students and trainers were collected electronically using two different forms. They were asked to give a score between 0 and 5 for each item. Percentage distributions, means and standard deviations were calculated for both groups. The Mann-Whitney U tests were used in the analyses. p < 0.05 was set as the statistical significance level. Thematic analysis was used for qualitative data. RESULTS The response rate for students was 51.8% and 62.5% for trainers. The mean for all items evaluated by both groups was above three points. The highest mean belongs to the item "level of preparation of trainers for practice" with 4.56 for the students and to the item "sufficiency of time allocated for practice" with 4.55 for the trainers. There was no statistically significant difference between the mean scores of the students according to their gender for any of the items (p>0.05 for all). The most common theme in the written feedback was that the application was instructive and useful for the students, while for the trainers the placement of the application within the programme affected the students' performance. There was no statistically significant difference between the students and the trainers in their evaluation of the articles used in the application. However, there was a statistically significant difference between the groups in terms of student participation and the evaluation of the adequacy of the time allocated for implementation (M-W U=1532,000 p=.023 and M-W U=1342,000 p=.003, respectively). DISCUSSION Students and trainers find the CA-SSM implementation valuable. Although the feedback from students and trainers on the CA practice is generally positive, it is important to make arrangements in line with the data obtained to increase the efficiency effectiveness of the practice. In order to enrich the programme and increase its educational impact, it would be appropriate to carry out studies to evaluate all its components.

Keywords

• Critical appraisal
• evidence-based medicine
• medical education
• student feedback
• trainer feedback

References

1. 1. Cook DJ, Jaeschke R, Guyatt GH. Critical Appraisal of Therapeutic Interventions in the Intensive Care Unit: Human Monoclonal Antibody Treatment in Sepsis. Journal of Intensive Care Medicine. 1992;7(6):275-282. doi:10.1177/088506669200700601).
2. 2. Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005 Dec;5(1)
3. 3. Godwin M, Seguin R. Critical appraisal skills of family physicians in Ontario, Canada. BMC Med Educ. 2003 Dec 2;3(1)
4. 4. HAYNES RB. How to Keep Up with the Medical Literature: I. Why Try to Keep Up and How to Get Started. Ann Intern Med. 1986 Jul 1;105(1):149.
5. 5. Ibbotson T, Grimshaw J, Grant A. Evaluation of a programme of workshops for promoting the teaching of critical appraisal skills. Med Educ. 1998 Sep;32(5):486-91. doi: 10.1046/j.1365-2923.1998.00256.x. PMID: 10211289.
6. 6. Taylor RS, Reeves BC, Ewings PE, Taylor RJ. Critical appraisal skills training for health care professionals: a randomized controlled trial [ISRCTN46272378]. BMC Med Educ. 2004 Dec;4(1)
7. 7. Taylor R, Reeves B, Ewings P, Binns S, Keast J, Mears R. A systematic review of the effectiveness of critical appraisal skills training for clinicians. Med Educ. 2000 Feb;34(2):120-5. doi: 10.1046/j.1365-2923.2000.00574.x. PMID: 10652064.
8. 8. Sasannia S, Amini M, Moosavi M, Askarinejad A, Moghadami M, Ziaee H, Vara F. Critical appraisal skills training to undergraduate medical students: A Randomized Control Study. J Adv Med Educ Prof. 2022 Oct;10(4):253-258. doi: 10.30476/JAMP.2022.94852.1610. PMID: 36310666; PMCID: PMC9589073.

9. 9. Coomarasamy A, Taylor R, Khan K. A systematic review of postgraduate teaching in evidence-based medicine and critical appraisal. Medical Teacher. 2003 Jan;25(1):77-81.
10. 10. Parkes J, Hyde C, Deeks J, Milne R. Teaching critical appraisal skills in health care settings. Cochrane Database Syst Rev. 2001;3:CD001270. DOI: 10.1002/14651858.
11. 11. Horsley T, Hyde C, Santesso N, Parkes J, Milne R, Stewart R. Teaching critical appraisal skills in healthcare settings. The Cochrane Database of Systematic Reviews 2011; Issue 11. Art. No.: CD001270. DOI: 10.1002/14651858.CD001270.pub2.
12. 12. Gupta A. Importance of critical appraisal skills training in medical students. Natl Med J India 2022;35:370–1.
13. 13. Hong B, Plugge E. Critical appraisal skills teaching in UK dental schools. Br Dent J. 2017 Feb 10;222(3):209-213. doi: 10.1038/sj.bdj.2017.126. PMID: 28184065.
14. 14. Meats E, Heneghan C, Crilly M, Glasziou P. Evidence-based medicine teaching in UK medical schools. Med Teach. 2009 Apr;31(4):332-7. doi: 10.1080/01421590802572791. PMID: 19404893.
15. 15. Daou D, Chakhtoura M, El-Yazbi A, Mukherji D, Sbaity E, Refaat MM, et al. Teaching critical appraisal to large classes of undergraduate medical students using team-based learning versus group discussions: a randomized controlled trial. BMC Med Educ. 2022 Dec;22(1).
16. 16. Siemens DR, Punnen S, Wong J, Kanji N. A survey on the attitudes towards research in medical school. BMC medical education. 2010;10(1):4.
17. 17. Constantinou C, Wijnen-Meijer M. Student evaluations of teaching and the development of a comprehensive measure of teaching effectiveness for medical schools. BMC Med Educ. 2022 Feb 19;22(1).