BÜYÜK DİL MODELLERİ İLE KAROTİS DOPPLER RAPORLARININ SINIFLANDIRILMASI: KISA BİR GÖZLEM

Abstract

CLASSIFICATION OF CAROTID DOPPLER REPORTS BY LARGE LANGUAGE MODELS: A BRIEF OBSERVATION

Abstract

Dear Editor, Large language models (LLMs) are increasingly important in clinical decision support systems and medical education due to their ability to analyse medical texts (1). This brief observation evaluates the performance of four LLMs — ChatGPT-4o (OpenAI), Claude 3.7 Sonnet (Anthropic), Gemini 1.5 Pro (Google DeepMind), and Grok-3 (xAI) — in classifying internal carotid artery (ICA) stenosis according to the Society of Radiologists in Ultrasound (SRU) criteria, using velocity parameters in carotid Doppler ultrasonography (USG) reports (2). A total of 40 USG reports were used, all containing identical velocity data but presented in two distinct formats. Each report included the peak systolic velocity (PSV), end diastolic velocity (EDV), and internal carotid artery/common carotid artery (ICA/CCA) PSV ratio for both the right and left ICA. The first 20 reports included non-directive descriptive statements. In the remaining 20, the same velocity values were retained, but directive phrases such as “plaques not causing significant stenosis” and “no haemodynamically significant stenosis detected” were added.

References

1. Lecler A, Duron L, Soyer P. Revolutionizing radiology with GPT-based models: Current applications, future possibilities and limitations of ChatGPT. Diagn Interv Imaging 2023;104(6):269-74. google scholar
2. Grant EG, Benson CB, Moneta GL, Alexandrov AV, Baker JD, Bluth EI, et al. Carotid artery stenosis: gray-scale and Doppler US diagnosis—Society of Radiologists in Ultrasound Consensus Conference. Radiology 2003;229(2):340-6. google scholar
3. Biswas M, Saba L, Omerzu T, Johri AM, Khanna NN, Viskovic K, et al. A review on joint carotid intima-media thickness and plaque area measurement in ultrasound for cardiovascular/stroke risk monitoring: Artificial intelligence framework. J Digit Imaging 2021;34(3):581-604. google scholar
4. Jain PK, Sharma N, Saba L, Paraskevas KI, Kalra MK, Johri A, et al. Unseen artificial intelligence-deep learning paradigm for segmentation of low atherosclerotic plaque in carotid ultrasound: A multicenter cardiovascular study. Diagnostics (Basel) 2021;11(12):2257. google scholar
5. Sacoransky E, Kwan BYM, Soboleski D. ChatGPT and assistive AI in structured radiology reporting: A systematic review. Curr Probl Diagn Radiol 2024;53(6):728-37. google scholar