Application of machine learning algorithms for predicting internal carotid artery stenosis and comparing their value to duplex Doppler ultrasonography criteria

Öz

Purpose: There is a discrepancy between duplex Doppler ultrasonography (DUS) and digital subtraction angiography (DSA) for determining internal carotid artery (ICA) stenosis. We aim to train machine learning algorithms (MLAs) with DUS velocity values for predicting ICA stenosis and comparing their success to DUS criteria.
Materials and methods: DUS values (peak systolic velocity (PSV) and end-diastolic velocity of the common carotid artery (CCA) and ICA) and DSA studies of 159 ICA stenoses were reviewed retrospectively. Stenoses were classified as <50%, 50-69%, ≥70% by each modality. Linear regression models with descriptive and predictive analysis and MLAs; LightGBM, XgBoost, KNeighbors, Support Vector Machine (SVM), Decision Tree, Random Forest were trained with DUS values for predicting DSA stenosis.
Results: Predicted values of regression models have a linear relationship with DSA stenosis between 0-60%. LightGBM and SVM achieved the highest classification accuracy (69%), while all algorithms failed in the 50-69% interval. DUS criteria outperformed all MLAs in predicting DSA stenosis of ≥70% (sensitivity:0.91). Both MLAs and DUS criteria were unsuccessful in the 50-69% interval where DUS mostly overestimates and MLAs underestimate. MLAs using ICA PSV/CCA PSV ratio had higher accuracy for predicting DSA stenosis <50%.
Conclusion: DUS criteria could be considered as the sole diagnostic tool for ICA stenosis over 70%. Improved DUS criteria or wider training datasets for MLAs are warranted to detect 50-69% stenosis accurately.

Anahtar Kelimeler

• Carotid artery
• duplex Doppler ultrasonography
• digital subtraction angiography
• machine learning
• stenosis

İnternal karotid arter darlığını tahmin etmede makine öğrenme algoritmalarının kullanımı ve öngörüm başarısının dubleks Doppler ultrasonografi kriterleriyle karşılaştırılması

Öz

Amaç: İnternal karotid arter (İKA) darlığını belirlemede, dupleks Doppler ultrasonografi (DUS) ile dijital subtraksiyon anjiyografi (DSA) arasında tutarsızlık bildirilmiştir. DUS hız değerleri ile eğitilmiş makine öğrenme algoritmalarının (MÖA), İKA darlığını tahmin etme performansını araştırmayı amaçlıyoruz.
Gereç ve yöntem: İKA darlığı olan 159 karotid bifurkasyonunun, ortak karotid arter (OKA) ve İKA'dan elde olunmuş DUS hız değerleri (pik sistolik hız (PSH) ve diyastol sonu hızı) ve DSA tetkikleri retrospektif olarak incelendi. Darlık derecesi her modaliteye göre <%50, %50-69, ≥%70 olarak sınıflandırıldı. Tanımlayıcı ve kestirimci analizler içeren doğrusal regresyon modelleri ve çeşitli MÖA’lar (LightGBM, XgBoost, KNeighbors, Support Vector Machine (SVM), Decision Tree, Random Forest) DSA’da saptanan darlık derecesini tahmin etmek için DUS hız değerleri ile eğitildi.
Bulgular: Regresyon modellerinin tahmin ettiği darlık değerleri ve asıl DSA darlık değerleri, %0-60 arasında doğrusal bir ilişkiye sahipti. MÖA’lar arasında LightGBM ve SVM en yüksek sınıflandırma doğruluğunu (%69) elde ederken, tüm algoritmalar %50-69 darlık aralığında başarısız oldu. DUS kriterleri, ≥%70'lik DSA darlığını tahmin etmede tüm MÖA’lardan daha iyi performans gösterdi (duyarlılık:0,91). Hem MÖA'lar hem de DUS kriterleri %50-69 darlık  aralığında başarısız olup, DUS darlığı olduğundan fazla, MÖA’lar darlığı olduğundan az
olarak tahmin etti. İKA PSH/OKA PSH oranını kullanan MÖA’lar, <%50 DSA darlığını öngörmede daha yüksek doğruluğa sahipti. 

Sonuç: DUS kriterleri, %70'in üzerinde İKA darlığı için tek tanı aracı olarak kabul edilebilir. Geliştirilmiş DUS kriterleri veya MÖA'lar için daha geniş eğitim veri setleri sağlanması, %50-69 darlık aralığının daha yüksek doğrulukla tespit edilmesini sağlayabilir.

Anahtar Kelimeler

• Darlık
• dijital subtraksiyon anjiyografi
• dupleks Doppler ultrasonografi
• karotid arter
• makine öğrenmesi

Kaynakça

1. 1. North American Symptomatic Carotid Endarterectomy Trial Collaborators, Barnett HJM, Taylor DW, et al. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med 1991; 325:445–453. https://doi.org/10.1056/NEJM199108153250701
2. 2. European Carotid Trialists' Collaborative Group. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet 1998;351:1379–1387.
3. 3. Endarterectomy for asymptomatic carotid artery stenosis. Executive Committee for the Asymptomatic Carotid Atherosclerosis Study. JAMA 1995;273:1421–1428.
4. 4. Arning C, Widder B, von Reutern GM, et al. Revision of DEGUM ultrasound criteria for grading internal carotid artery stenoses and transfer to NASCET measurement. Ultraschall Med 2010;31:251–257. https://doi.org/10.1055/s-0029-1245336
5. 5. Hathout GM, Fink JR, El-Saden SM, Grant EG. Sonographic NASCET index: a new doppler parameter for assessment of internal carotid artery stenosis. AJNR Am J Neuroradiol 2005;26:68–75.
6. 6. Grant EG, Benson CB, Moneta GL, et al. Carotid artery stenosis: gray-scale and Doppler US diagnosis--Society of Radiologists in Ultrasound Consensus Conference. Radiology 2003;229:340–346. https://doi.org/10.1148/radiol.2292030516
7. 7. Eliasziw M, Rankin RN, Fox AJ, et al. Accuracy and prognostic consequences of ultrasonography in identifying severe carotid artery stenosis. North American Symptomatic Carotid Endarterectomy Trial (NASCET) Group. Stroke 1995;26:1747–1752. https://doi.org/10.1161/01.str.26.10.1747
8. 8. Ballard JL, Fleig K, De Lange M, Killeen JD. The diagnostic accuracy of duplex ultrasonography for evaluating carotid bifurcation. Am J Surg 1994;168:123–126; discussion 130. https://doi.org/10.1016/s0002-9610(94)80050-2

9. 9. Howard G, Baker WH, Chambless LE, et al. An approach for the use of Doppler ultrasound as a screening tool for hemodynamically significant stenosis (despite heterogeneity of Doppler performance). A multicenter experience. Asymptomatic Carotid Atherosclerosis Study Investigators. Stroke 1996;27:1951–1957. https://doi.org/10.1161/01.str.27.11.1951
10. 10. Boyko M, Kalashyan H, Becher H, et al. Comparison of Carotid Doppler Ultrasound to Other Angiographic Modalities in the Measurement of Carotid Artery Stenosis. J Neuroimaging 2018;28:683-687. doi: 10.1111/jon.12532
11. 11. Barlinn K, Rickmann H, Kitzler H, et al. Validation of Multiparametric Ultrasonography Criteria with Digital Subtraction Angiography in Carotid Artery Disease: A Prospective Multicenter Study. Ultraschall Med 2018;39:535–543. https://doi.org/10.1055/s-0043-119355
12. 12. Dean N, Lari H, Saqqur M, et al. Reliability of carotid doppler performed in a dedicated stroke prevention clinic. Can J Neurol Sci 2005;32:327–331. https://doi.org/10.1017/s0317167100004212
13. 13. Wardlaw JM, Chappell FM, Best JJK, et al. Non-invasive imaging compared with intra-arterial angiography in the diagnosis of symptomatic carotid stenosis: a meta-analysis. Lancet 2006;367:1503–1512. https://doi.org/10.1016/S0140-6736(06)68650-9
14. 14. Gough MJ. Preprocedural imaging strategies in symptomatic carotid artery stenosis. J Vasc Surg 2011;54:1215–1218. https://doi.org/10.1016/j.jvs.2011.05.101
15. 15. Zou KH, Tuncali K, Silverman SG. Correlation and simple linear regression. Radiology 2003;227:617–622. https://doi.org/10.1148/radiol.2273011499
16. 16. Carnicelli AP, Stone JJ, Doyle A, et al. Predictive multivariate regression to increase the specificity of carotid duplex ultrasound for high-grade stenosis in asymptomatic patients. Ann Vasc Surg 2014;28:1548–1555. https://doi.org/10.1016/j.avsg.2014.02.010
17. 17. Hathout G, Nayak N, Abdulla A, Huang J. The Revised Sonographic NASCET Index: A New Hemodynamic Parameter for the Assessment of Internal Carotid Artery Stenosis. Ultraschall Med 2015;36:362–368. https://doi.org/10.1055/s-0034-1385070
18. 18. Polak A, Polak JF. Internal to Common Carotid Artery Peak Systolic Velocity Ratios for Predicting North American Symptomatic Carotid Endarterectomy Trial Stenosis: Derivation/Validation Study Using a Machine Learning Technique. J Vasc Ultrasound 2019;43:182–185. https://doi.org/10.1177/1544316719874576
19. 19. AbuRahma AF, Srivastava M, Stone PA, et al. Critical appraisal of the Carotid Duplex Consensus criteria in the diagnosis of carotid artery stenosis. J Vasc Surg 2011;53:53–59; discussion 59-60. https://doi.org/10.1016/j.jvs.2010.07.045
20. 20. Beach KW, Leotta DF, Zierler RE. Carotid Doppler velocity measurements and anatomic stenosis: correlation is futile. Vasc Endovascular Surg 2012;46:466–474. https://doi.org/10.1177/1538574412452159
21. 21. Goldstein LB, Bushnell CD, Adams RJ, et al. Guidelines for the primary prevention of stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2011;42:517–584. https://doi.org/10.1161/STR.0b013e3181fcb238
22. 22. Brott TG, Halperin JL, Abbara S, et al. 2011 ASA/ACCF/AHA/AANN/AANS/ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/SVM/SVS guideline on the management of patients with extracranial carotid and vertebral artery disease: executive summary. Circulation 2011;124:489–532. https://doi.org/10.1161/CIR.0b013e31820d8d78