AI-ASSISTED AND TRADITIONAL ASSESSMENT OF THERAPEUTIC COMMUNICATION SKILLS: A QUASI-EXPERIMENTAL STUDY WITH STANDARDIZED PATIENT-BASED EDUCATION

Öz

Objective: This study aims to examine the effectiveness of standardized patient (SP) based therapeutic communication training and the accuracy, reliability, and feasibility of artificial intelligence (AI) supported assessment in improving nursing students' communication competencies. The premise of the study is to determine to what extent AI can assess complex clinical skills, such as interpersonal communication, in harmony with human raters. Method: This quasi-experimental pre-test/post-test study, including a three-month follow-up measurement, was completed with 64 students (intervention group n=33, control group n=31) after eight of the initially randomized 72 students left the study during the process. A five-session SP-supported therapeutic communication training was applied to the intervention group, while the control group received standard education. Communication performances were scored at baseline, post-training, and follow-up by both human raters and an AI-based assessment system analyzing anonymized transcripts containing coded nonverbal behaviors. Data were analyzed using mixed-design ANOVA, independent samples t-tests, Pearson correlations, and intraclass correlation coefficients (ICC). Results: Students in the intervention group showed significant improvement in therapeutic communication skills compared to the control group at all measurement times (p<.001), and this improvement was maintained at the three-month follow-up. AI-supported assessments showed near-perfect agreement with human evaluations at all time points (ICC=0.97-0.99). This finding demonstrates that AI can offer consistent, objective, and high-reliability communication assessment. Conclusion: The integration of SP-based training and AI-supported assessment effectively improves nursing students' interpersonal communication competencies and supports the retention of learning. AI makes a strong contribution to competency-based education models by providing scalable, unbiased, and detailed feedback. The findings emphasize the importance of integrating structured simulation experiences and AI-based assessment approaches into nursing education. Future research is recommended to focus on long-term follow-up, multi-center samples, and ethical dimensions regarding AI use in education.

Anahtar Kelimeler

• Artificial Intelligence
• Patient Simulation
• Nursing Education
• Nursing Students
• Quasi-Experimental Study

YAPAY ZEKÂ DESTEKLİ VE GELENEKSEL DEĞERLENDİRME İLE TERAPÖTİK İLETİŞİM BECERİLERİNİN DEĞERLENDİRİLMESİ: STANDARDİZE HASTA TEMELLİ YARI DENEYSEL BİR ÇALIŞMA

Öz

Amaç: Bu çalışma, standardize hasta (SH) temelli terapötik iletişim eğitiminin etkililiğini ve hemşirelik öğrencilerinin iletişim yeterliklerini geliştirmede yapay zekâ (YZ) destekli değerlendirmenin doğruluğu, güvenilirliği ve uygulanabilirliğini incelemeyi amaçlamaktadır. Çalışmanın çıkış noktası, YZ’nin kişilerarası iletişim gibi karmaşık klinik becerileri insan değerlendiricilerle ne ölçüde uyumlu biçimde değerlendirebildiğini belirlemektir. Yöntem: Üç aylık takip ölçümünü içeren bu yarı deneysel ön test-son test çalışma, başlangıçta randomize edilen 72 öğrenciden sekizinin süreç içinde çalışmadan ayrılması sonrası 64 öğrenciyle tamamlandı (müdahale grubu n=33, kontrol grubu n=31). Müdahale grubuna beş oturumluk SH destekli terapötik iletişim eğitimi uygulandı ve kontrol grubu ise standart eğitim aldı. İletişim performansları başlangıçta, eğitim sonrası ve takipte hem insan değerlendiriciler hem de kodlanmış nonverbal davranışları içeren anonimleştirilmiş transkriptleri analiz eden YZ tabanlı değerlendirme sistemiyle puanlandı. Veriler karma desenli ANOVA, bağımsız örneklemler t-testi, Pearson korelasyonları ve sınıf içi korelasyon katsayıları (ICC) ile analiz edildi. Bulgular: Müdahale grubundaki öğrenciler, tüm ölçüm zamanlarında kontrol grubuna kıyasla terapötik iletişim becerilerinde anlamlı gelişme gösterdi (p<.001) ve bu gelişim üç aylık takipte de korundu. YZ destekli değerlendirmeler, tüm zaman noktalarında insan değerledirmeleriyle mükemmele yakın düzeyde uyum gösterdi (ICC=0.97-0.99). Bu bulgu, YZ’nin tutarlı, nesnel ve yüksek güvenilirlikte iletişim değerlendirmesi sunabildiğini gösterdi. Sonuç: SH temelli eğitim ile YZ destekli değerlendirmenin entegrasyonu, hemşirelik öğrencilerinin kişilerarası iletişim yetkinliklerini etkili biçimde geliştirmekte ve öğrenmenin kalıcılığını desteklemektedir. YZ; ölçeklenebilir, tarafsız ve ayrıntılı geri bildirim sağlayarak yetkinliğe dayalı eğitim modellerine güçlü bir katkı sunmaktadır. Bulgular, yapılandırılmış simülasyon deneyimlerinin ve YZ tabanlı değerlendirme yaklaşımlarının hemşirelik eğitimine entegrasyonunun önemini vurgulamaktadır. Gelecek araştırmaların uzun dönemli izlem, çok merkezli örneklemler ve eğitimde YZ kullanımına ilişkin etik boyutlara odaklanması önerilmektedir.

Anahtar Kelimeler

• Yapay Zekâ
• Hasta Simülasyonu
• Hemşirelik Eğitimi
• Hemşirelik Öğrencileri
• Yarı Deneysel Çalışma

Kaynakça

1. Molu B. Improving nursing students’ learning outcomes in neonatal resuscitation: a quasi‐experimental study comparing AI‐assisted care plan learning with traditional instruction. J Eval Clin Pract. 2024;31(1).
2. Seibert K, Domhoff D, Bruch D, et al. Application scenarios for artificial intelligence in nursing care: rapid review. J Med Internet Res. 2021;23(11):e26522.
3. Reed J, Dodson TM. Generative AI backstories for simulation preparation. Nurse Educ. 2023;49(4):184-188.
4. Ramírez‐Baraldes E, García‐Gutiérrez D, García‐Salido C. Artificial intelligence in nursing: new opportunities and challenges. Eur J Educ. 2025;60(1).
5. Buchanan C, Howitt ML, Wilson R, Booth R, Risling T, Bamford M. Predicted influences of artificial intelligence on nursing education: scoping review. Jmir Nurs. 2021;4(1):e23933.
6. Andersen BL, Jørnø RLV, Nortvig AM. Blending adaptive learning technology into nursing education: a scoping review. Contemp Educ Technol. 2021;14(1):ep333.
7. Topaz M, Peltonen L, Michalowski M, et al. The ChatGPT effect: nursing education and generative artificial intelligence. J Nurs Educ. 2025;64(6).
8. Kim J, Lee HK, Cho YH. Learning design to support student-AI collaboration: perspectives of leading teachers for AI in education. Educ Inf Technol. 2022;27(5):6069-6104.

9. Han S, Kang HS, Gimber P, Lim S. Nursing students’ perceptions and use of generative artificial intelligence in nursing education. Nurs Rep. 2025;15(2):68.
10. Syed W, Al-Rawi MBA. Assessment of awareness, perceptions, and opinions towards artificial intelligence among healthcare students in Riyadh, Saudi Arabia. Medicina (Mex). 2023;59(5):828.
11. Gado S, Kempen R, Lingelbach K, Bipp T. Artificial intelligence in psychology: how can we enable psychology students to accept and use artificial intelligence? Psychol Learn Teach. 2021;21(1):37-56.
12. Jung SY. Challenges for future directions for artificial intelligence integrated nursing simulation education. Korean J Women Health Nurs. 2023;29(3):239-242.
13. Abujaber AA, Abd‐Alrazaq A, Al‐Qudimat AR, Nashwan AJ. A strengths, weaknesses, opportunities, and threats (SWOT) analysis of ChatGPT integration in nursing education: a narrative review. Cureus. 2023.
14. Gagné JCD. The state of artificial intelligence in nursing education: past, present, and future directions. Int J Environ Res Public Health. 2023;20(6):4884.
15. Liaw SY, Tan JZ, Bin Rusli KD, et al. Artificial intelligence versus human-controlled doctor in virtual reality simulation for sepsis team training: randomized controlled study. J Med Internet Res. 2023;25:e47748.
16. Yu L, Yu Z. Qualitative and quantitative analyses of artificial intelligence ethics in education using VOSviewer and CitNetExplorer. Front Psychol. 2023;14.
17. Gagné JCD, Hwang H, Jung D. Cyberethics in nursing education: ethical implications of artificial intelligence. Nurs Ethics. 2023;31(6):1021-1030.
18. RANDOM.ORG. https://www.random.org/lists/
19. Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods. 2009;41(4):1149-1160.
20. Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340(mar23 1):c332-c332. doi:10.1136/bmj.c332
21. Decker S, Alinier G, Crawford SB, Gordon RM, Jenkins D, Wilson CA. Healthcare simulation standards of best practiceTM the debriefing process. Clin Simul Nurs. 2021;58:27-32.
22. Kolb DA. Experimental learning: experience as the source of learning and development. Prentice-Hall; 1984.
23. Polit DF, Beck CT. Nursing research: generating and assessing evidence for nursing practice. 10. edition, international edition. Wolters Kluwer; 2017.
24. MacLean S, Kelly M, Geddes F, Della P. Use of simulated patients to develop communication skills in nursing education: an integrative review. Nurse Educ Today. 2017;48:90-98.
25. Adnan AI. Effectiveness of communication skills training in medical students using simulated patients or volunteer outpatients. Cureus. 2022.
26. Epstein RM. Defining and assessing professional competence. JAMA. 2002;287(2):226.
27. Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-645.
28. Alhakami H, Alsubait T, Alhakami W, et al. Advancing sustainable healthcare through enhanced therapeutic communication with elderly patients in the Kingdom of Saudi Arabia. Sustainability. 2023;15(22):15778.
29. Han S, Yoo JH, Kang K. Development and validation of the therapeutic communication scale in nursing students. Healthcare. 2024;12(3):394.
30. Geng J, Liu M, Zhang H, et al. Application of the six-step standard communication process in the communication training for newly recruited nurses in cancer specialist hospitals. Front Surg. 2022;9.
31. Yang S, Kang MK. Efficacy testing of a multi-access metaverse-based early onset schizophrenia nursing simulation program: a quasi-experimental study. Int J Environ Res Public Health. 2022;20(1):449.
32. Kartika IR, Rezkiki F, Wulandari S, Rida RO. Training on therapeutic communication and patient safety to improve the quality indicator of nursing care at the government hospital in Bukittinggi, West Sumatera. Salus Publica J Community Serv. 2023;1(1):18-22.
33. Mahyuvi T, Masqurotin M, Rumpiati R. Therapeutic communication with patient anxiety levels during operation preparation: a cross-sectional study. Int J Nurs Health Serv. 2023;6(4):257-265.
34. Dermani DB, Garbuio DC, Carvalho EC d. Knowledge, applicability and importance attributed by nursing undergraduates to communicative strategies. Rev Bras Enferm. 2020;73(6).
35. Wulandari RA, Asmaningrum N, Ardiana A. Transcultural communication strategies in nursing with multicultural clients in hospital settings: a systematic literature review. J Pendidik Keperawatan Indones. 2022;8(2):91-106.
36. Gul S, Saddique H, Jabeen R. Knowledge, attitude and practice of nurses regarding therapeutic communication at tertiary care hospital. JHRR. 2024;4(2):144-148.
37. Yıldırım S, Kazandı E, Cirit K, Yağız H. The effects of communication skills on resilience in undergraduate nursing students in Turkey. Perspect Psychiatr Care. 2020;57(3):1120-1125.
38. Xuto P, Prasitwattanaseree P, Chaiboonruang T, et al. Development and evaluation of an AI-assisted answer assessment (4A) for cognitive assessments in nursing education. Nurs Rep. 2025;15(3):80.