Quality and Readability of AI-Generated Post-Myocardial Infarction Patient Information
Abstract
Objective: Large language models are increasingly used to obtain health-related information; however, the quality and readability of their responses to practical questions after myocardial infarction (MI) remain uncertain. This study compared ChatGPT and Google Gemini with guideline-based responses for post-MI patient information.
Methods: Ten commonly searched questions regarding post-MI recovery and secondary prevention were evaluated. Identical questions were submitted to ChatGPT and Google Gemini, and corresponding reference responses were derived from contemporary European Society of Cardiology guidelines. Two blinded cardiologists independently evaluated response quality using the CLEAR Tool. Readability was assessed using the Flesch Reading Ease Score (FRES) and Flesch–Kincaid Grade Level (FKGL).
Results: Thirty responses were evaluated. Inter-rater reliability was excellent (ICC=0.916; 95% CI, 0.821–0.960). Total CLEAR scores differed significantly among the three sources (P=0.002). The median total CLEAR score was highest for ChatGPT [24.00 (23.63–24.38)], followed by guideline-based responses [23.25 (23.00–23.88)] and Google Gemini [19.00 (18.00–20.13)]. ChatGPT and guideline-based responses did not differ significantly in total CLEAR score (P=0.164), whereas Google Gemini scored significantly lower than both. Readability also differed significantly among sources (both FRES and FKGL, P<0.001). Google Gemini had the highest FRES [48.8 (41.0–51.6)] and lowest FKGL [11.5 (10.8–12.1)], indicating greater readability.
Conclusion: ChatGPT provided post-MI information with overall quality comparable to guideline-based responses, whereas Google Gemini generated more readable but lower-quality responses. These findings suggest a potential trade-off between readability and informational quality and support the use of LLMs as adjuncts rather than substitutes for clinician-led post-MI patient education.
Keywords
Myocardial infarction, ChatGPT, Google Gemini, Large language models, Patient education, Readability
Ethical Statement
References
- Salari N, Morddarvanjoghi F, Abdolmaleki A, Rasoulpoor S, Khaleghi AA, Hezarkhani LA, et al. The global prevalence of myocardial infarction: a systematic review and meta-analysis. BMC Cardiovasc Disord. 2023;23:206. doi:10.1186/s12872-023-03231-w.
- Christensen DM, Schjerning AM, Smedegaard L, Charlot MG, Ravn PB, Ruwald AC, et al. Long-term mortality, cardiovascular events, and bleeding in stable patients 1 year after myocardial infarction: a Danish nationwide study. Eur Heart J. 2023;44(6):488-98. doi:10.1093/eurheartj/ehac667.
- Gouda P, Savu A, Bainey KR, Kaul P, Welsh RC. Long-term risk of death and recurrent cardiovascular events following acute coronary syndromes. PLoS One. 2021;16(7):e0254008. doi:10.1371/journal.pone.0254008.
- Yun HS, Bickmore T. Online health information-seeking in the era of large language models: cross-sectional web-based survey study. J Med Internet Res. 2025;27:e68560. doi:10.2196/68560.
- Ferreira Santos J, Ladeiras-Lopes R, Leite F, Dores H. Applications of large language models in cardiovascular disease: a systematic review. Eur Heart J Digit Health. 2025;6(4):540-53. doi:10.1093/ehjdh/ztaf028.
- Wang L, Wan Z, Ni C, Song Q, Li Y, Clayton E, et al. Applications and concerns of ChatGPT and other conversational large language models in health care: systematic review. J Med Internet Res. 2024;26:e22769. doi:10.2196/22769.
- Sallam M, Barakat M, Sallam M. Pilot testing of a tool to standardize the assessment of the quality of health information generated by artificial intelligence-based models. Cureus. 2023;15(11):e49373. doi:10.7759/cureus.49373.
- Korkmaz İ. Accuracy and reliability of AI models in emergency myocardial infarction education. Emerg Med Int. 2026;2026:5530861. doi:10.1155/emmi/5530861.
- Jiang X, Liu J, Dai S, Mao X, Cha R, Wu L. Evaluating the effectiveness of generative AI for the creation of patient education materials on coronary heart disease: a comparative study. JMIR Form Res. 2025;9:e78816. doi:10.2196/78816.
- Byrne RA, Rossello X, Coughlan JJ, Barbato E, Berry C, Chieffo A, et al. 2023 ESC Guidelines for the management of acute coronary syndromes. Eur Heart J. 2023;44(38):3720-826. doi:10.1093/eurheartj/ehad191.