Reliability of AI Chatbots in Categorising Foods by Oxalate Content/ Yapay Zeka Araçlarının Gıdaları Oksalat İçeriğine Göre Sınıflandırmadaki Güvenilirliği

Abstract

Aim: AI chatbots have shown promise in food classification tasks, but their accuracy in categorising foods based on specific nutritional content, such as oxalates, has not been thoroughly evaluated in the Turkish language. This study assesses the performance of three AI chatbots—ChatGPT 4.0, Gemini, and Microsoft Copilot—in classifying foods according to their oxalate content. Materials and Methods: A dataset of 63 diverse food items, including commonly consumed Turkish foods, was used to evaluate the chatbots’ accuracy across five oxalate categories: little or none, low, moderate, high, and very high. The performance of each model was analysed, and commonly correct and incorrect classifications were identified. Results: ChatGPT 4.0 demonstrated the highest overall accuracy (69.8%), significantly outperforming Gemini (36.5%) and Microsoft Copilot (26.9%). Foods such as spinach and cocoa were consistently classified correctly, while foods like carrot and walnut were commonly misclassified. Statistical analysis using Cochran’s Q test revealed significant differences in accuracy among the chatbots (p-value <0.05). Conclusion: This study highlights the potential of AI chatbots in dietary management, particularly in supporting clinicians who recommend low oxalate diets for patients with conditions such as hyperoxaluria or kidney disease stones. However, it emphasises the need for further refinement to improve accuracy, especially in classifying foods with regional variations or complex compositions commonly encountered in clinical settings.

Keywords

• Artificial Intelligence
• Hyperoxaluria
• Kidney Stones
• Renal Diet
• Oxalate

Reliability of AI Chatbots in Categorising Foods by Oxalate Content/ Yapay Zeka Araçlarının Gıdaları Oksalat İçeriğine Göre Sınıflandırmadaki Güvenilirliği

Abstract

Amaç: Yapay zeka sohbet botları, gıda sınıflandırma görevlerinde umut vaat etmiştir; ancak oksalat gibi belirli besin içeriklerine göre gıdaları sınıflandırmadaki doğrulukları, Türkçe dilinde yeterince değerlendirilmemiştir. Bu çalışma, üç yapay zeka sohbet botunun—ChatGPT 4.0, Gemini ve Microsoft Copilot—oksalat içeriğine göre gıdaları sınıflandırma performansını değerlendirmektedir. Gereç ve Yöntem: Yaygın tüketilen Türk gıdalarını içeren 63 farklı gıda maddesinden oluşan bir veri seti, beş oksalat kategorisinde (çok az veya yok, düşük, orta, yüksek ve çok yüksek) sohbet botlarının doğruluğunu değerlendirmek için kullanılmıştır. Her modelin performansı analiz edilmiş ve yaygın doğru ve yanlış sınıflandırmalar belirlenmiştir. Bulgular: ChatGPT 4.0, genel doğruluk açısından en yüksek performansı (%69,8) sergileyerek Gemini (%36,5) ve Microsoft Copilot'un (%26,9) önüne geçmiştir. Ispanak ve kakao gibi gıdalar tutarlı bir şekilde doğru sınıflandırılırken, havuç ve ceviz gibi gıdalar genellikle yanlış sınıflandırılmıştır. Cochran’s Q testi kullanılarak yapılan istatistiksel analiz, sohbet botları arasındaki doğruluk farklılıklarının anlamlı olduğunu göstermiştir (p <0,05). Sonuç: Bu çalışma, özellikle hiperoksalüri veya böbrek taşı gibi durumları olan hastalar için düşük oksalat diyeti önerilen durumlarda, diyet yönetiminde yapay zeka araçlarının potansiyelini vurgulamaktadır. Ancak, bölgesel farklılıklar veya karmaşık bileşimlere sahip gıdaların sınıflandırılmasında doğruluğun artırılması gerektiğini vurgulamaktadır.

Keywords

• Yapay Zeka
• Hiperoksalüri
• Böbrek Taşı
• Renal Diyet
• Oksalat

References

1. Lang J, Narendrula A, El-Zawahry A, Sindhwani P, Ekwenna O. Global trends in incidence and burden of urolithiasis from 1990 to 2019: an analysis of global burden of disease study data. Eur Urol Suppl. 2022;35:37-46. https://doi.org/10.1016/j.euros.2021.10.008
2. Abufaraj M, Al Karmi J, Yang L. Prevalence and trends of urolithiasis among adults. Curr Opin Urol. 2022;32(4):425-32.https://doi.org/10.1097/mou.0000000000000994
3. Abbas W, Akram M, Sharif A. Nephrolithiasis; prevalence, risk factors and therapeutic strategies: a review. Med J Islam Repub Iran. 2019;3(3):90-5. https://doi.org/10.18689/mjiem-1000120
4. Hill AJ, Basourakos SP, Lewicki P, Wu X, Arenas-Gallo C, Chuang D, et al. Incidence of kidney stones in the United States: the continuous national health and nutrition examination survey. J Urol. 2022;207(4):851-6. https://doi.org/10.1097/ju.0000000000002331
5. Thongprayoon C, Krambeck AE, Rule AD. Determining the true burden of kidney stone disease. Nat Rev Nephrol.2020;16(12):736-46. https://doi.org/10.1038/s41581-020-0320-7
6. Shastri S, Patel J, Sambandam KK, Lederer ED. Kidney stone pathophysiology, evaluation and management: core curriculum 2023. Am J Kidney Dis. 2023;82(5):617-34. https://doi.org/10.1053/j.ajkd.2023.03.017
7. Khan SR, Pearle MS, Robertson WG, Gambaro G, Canales BK, Doizi S, et al. Kidney stones. Nat Rev Dis Primers. 2016;2(1):1-23. https://doi.org/10.1038/nrdp.2016.8
8. Pearle MS, Goldfarb DS, Assimos DG, Curhan G, Denu-Ciocca CJ, Matlaga BR, et al. Medical management of kidney stones:AUA guideline. J Urol. 2014;192(2):316-24. https://doi.org/10.1016/j.juro.2014.05.006

9. Morgan MS, Pearle MS. Medical management of renal stones. BMJ. 2016;352. https://doi.org/10.1136/bmj.i52
10. Balawender K, Łuszczki E, Mazur A, Wyszyńska J. The Multidisciplinary Approach in the Management of Patients with Kidney Stone Disease—A State-of-the-Art Review. Nutrients. 2024;16(12):1932. https://doi.org/10.3390/nu16121932
11. Eryildirim B, Sahin C, Tuncer M, Sabuncu K, Cetinel C, Tarhan F, et al. Effect of medical expulsive therapy on the healthrelated quality of life of patients with ureteral stones: a critical evaluation. Int Urol Nephrol. 2015;47:1271-5.https://doi.org/10.1007/s11255-015-1036-7
12. 12. Diniz DH, Blay SL, Schor N. Quality of life of patients with nephrolithiasis and recurrent painful renal colic. Nephrol Dial Transplant. 2007;106(3):c91-c7. https://doi.org/10.1159/000102995 Kalaitzidis RG, Damigos D, Siamopoulos KC. Environmental and stressful factors affecting the occurrence of kidney stones and the kidney colic. Int Urol Nephrol. 2014;46:1779-84. https://doi.org/10.1007/s11255-014-0758-2
13. Khopekar F, Nabi S, Shiva M, Stewart M, Rajendran B, Nabi G. Cost-effectiveness of quality improvement intervention to reduce time between CT-detection and ureteroscopic laser fragmentation in acute symptomatic ureteric stones management. World J Urol. 2024;42(1):144. https://doi.org/10.1007/s00345-023-04694-4
14. Neumaier ER. Effects of an integrated behavioral health intervention on the health outcomes and quality of life of patients with kidney stone disease [dissertation]. The University of Wisconsin-Madison;2012.
15. Néill EN, Richards HL, Hennessey D, Ryan EM, Fortune DG. Psychological distress in patients with urolithiasis: a systematic review and meta-analysis. J Urol. 2023;209(1):58-70.https://doi.org/10.1097/ju.0000000000003032
16. Huang Y, Zhang YH, Chi ZP, Huang R, Huang H, Liu G, et al. The handling of oxalate in the body and the origin of oxalate in calcium oxalate stones. Urol Res. 2020;104(3-4):167-76. https://doi.org/10.1159/000504417
17. Singh P, Enders FT, Vaughan LE, Bergstralh EJ, Knoedler JJ, Krambeck AE, et al. Stone composition among first-time symptomatic kidney stone formers in the community Mayo Clin Proc. Elsevier. 2015;90(10):1356-65. https://doi.org/10.1016/j.mayocp.2015.07.016
18. Trinchieri A. Epidemiology of urolithiasis: an update. Clinical cases in mineral and bone metabolism. [Internet]. 2008[cited 2024 Nov 27];5(2):101-106. Available from: https://pmc.ncbi.nlm.nih.gov/articles/PMC2781200/
19. Taylor EN, Curhan GC. Role of nutrition in the formation of calcium-containing kidney stones. Nephrol Dial Transplant.2004;98(2):55-63. https://doi.org/10.1159/000080265
20. Lin BB, Lin ME, Huang RH, Hong YK, Lin BL, He XJ. Dietary and lifestyle factors for primary prevention of nephrolithiasis: a systematic review and meta-analysis. BMC Nephrol. 2020;21:1-13. https://doi.org/10.1186/s12882-020-01925-3
21. Willett WC, Koplan JP, Nugent R, Dusenbury C, Puska P, Gaziano TA. Prevention of chronic disease by means of diet and lifestyle changes. Dis Control Priorities Dev Ctries [Internet]. 2006[cited 2024 Nov 28];3(2):599-616. Available from: https://www.ncbi.nlm.nih.gov/books/NBK11795/
22. Mitchell T, Kumar P, Reddy T, Wood KD, Knight J, Assimos DG, et al. Dietary oxalate and kidney stone formation. Am J Physiol Renal Physiol. 2019;316(3):F409-F13. https://doi.org/10.1152/ajprenal.00373.2018
23. Franceschi VR, Nakata PA. Calcium oxalate in plants: Formation and function. Annu Rev Plant Biol. 2005;56(1):41-71. https://doi.org/10.1146/annurev.arplant.56.032604.144106
24. Massey LK. Dietary influences on urinary oxalate and risk of kidney stones. Front Biosci. 2003;8:584-94. https://doi.org/10.2741/1082
25. Holmes RP, Knight J, Assimos DG. Lowering urinary oxalate excretion to decrease calcium oxalate stone disease. Urolithiasis.2016;44(1):27-32. https://doi.org/10.1007/s00240-015-0839-4
26. Lo CYZ, Khor QH, Abdullatif VA, Delgado C, Lu Y, Katz J, et al. Systematic review of pharmacological, complementary and alternative therapies for the prevention of calcium oxalate stones. Asian J Urol. [Article in press]. 2024 [cited 2024 Nov 30] Available from: https://doi.org/10.1016/j.ajur.2024.04.006
27. Siener R, Ernsten C, Welchowski T, Hesse A. Metabolic Profile of Calcium Oxalate Stone Patients with Enteric Hyperoxaluria and Impact of Dietary Intervention. Nutrients. 2024;16(16):2688. https://doi.org/10.3390/nu16162688
28. Taylor EN, Curhan GC. Oxalate intake and the risk for nephrolithiasis. J Am Soc Nephrol. 2007;18(7):2198-204. https://doi.org/10.1681/asn.2007020219
29. Meschi T, Maggiore U, Fiaccadori E, Schianchi T, Bosi S, Adorni G, et al. The effect of fruits and vegetables on urinary stone risk factors. Kidney Int. 2004;66(6):2402-10. https://doi.org/10.1111/j.1523-1755.2004.66029.x
30. Noonan SC, Savage GP. Oxalate content of foods and its effect on humans. Asia Pac J Clin Nutr [Internet]. 1999 [cited 2024 Nov 30];8(1):64-74. Available from: https://pubmed.ncbi.nlm.nih.gov/24393738/
31. Aiumtrakul N, Thongprayoon C, Arayangkool C, Vo KB, Wannaphut C, Suppadungsuk S, et al. Personalized Medicine in Urolithiasis: AI Chatbot-Assisted Dietary Management of Oxalate for Kidney Stone Prevention. J Pers Med. 2024;14(1):107.https://doi.org/10.3390/jpm14010107
32. Attalla K, De S, Monga M. Oxalate content of food: a tangled web. Urology. 2014;84(3):555-60. https://doi.org/10.1016/j.urology. 2014.03.053
33. Udegbe FC, Ebulue OR, Ebulue CC, Ekesiobi CS. The role of artificial intelligence in healthcare: A systematic review of applications and challenges. Int Med Sci Rev J. 2024;4(4):500-8. https://doi.org/10.51594/imsrj.v4i4.1052
34. Poalelungi DG, Musat CL, Fulga A, Neagu M, Neagu AI, Piraianu AI, et al. Advancing patient care: how artificial intelligence is transforming healthcare. Healthcare (Basel). 2023;13(8):1214. https://doi.org/10.3390/jpm13081214
35. Talyshinskii A, Naik N, Hameed BZ, Juliebø-Jones P, Somani BK. Potential of AI-driven chatbots in urology: revolutionizing patient care through artificial intelligence. Curr Urol Rep. 2024;25(1):9-18. https://doi.org/10.1007/s11934-023-01184-3
36. Zhang J, Oh YJ, Lange P, Yu Z, Fukuoka Y. Artificial intelligence chatbot behavior change model for designing artificial intelligence chatbots to promote physical activity and a healthy diet. J Med Internet Res. 2020;22(9):e22845. https://doi.org/10.2196/22845
37. Qarajeh A, Tangpanithandee S, Thongprayoon C, Suppadungsuk S, Krisanapan P, Aiumtrakul N, et al. AI-Powered Renal Diet Support: Performance of ChatGPT, Bard AI, and Bing Chat. J Pers Med. 2023;13(5):1160-72. https://doi.org/10.3390/clinpract 13050104
38. Sharma SK, Gaur S. Optimizing Nutritional Outcomes: The Role of AI in Personalized Diet Planning. International Journal for Research Publication and Seminar. Int J Res Pub Seminar. 2024. https://doi.org/10.36676/jrps.v15.i2.15
39. Alhur A. Redefining healthcare with artificial intelligence (AI): the contributions of ChatGPT, Gemini, and Co-pilot. Curr Opin Clin Nutr Metab Care. 2024;16(4). https://doi.org/10.7759/cureus.57795
40. Oh YJ, Zhang J, Fang M-L, Fukuoka Y. A systematic review of artificial intelligence chatbots for promoting physical activity, healthy diet, and weight loss. Int J Behav Nutr Phys Act. 2021;18:1-25. https://doi.org/10.1186/s12966-021-01224-6
41. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. Nat Biomed Eng. 2019;17:1-9. https://doi.org/10.1186/s12916-019-1426-2
42. Miao J, Thongprayoon C, Suppadungsuk S, Krisanapan P, Radhakrishnan Y, Cheungpasitporn W. Chain of thought utilization in large language models and application in nephrology. J Med AI Nephrol. 2024;60(1):148. https://doi.org/10.3390/medicina60010148
43. ChatGPT 4.0 [Internet]. [cited 2024 Nov 13]. Available from: https://chatgpt.com.
44. Gemini [Internet]. [cited 2024 Nov 14]. Available from: https://gemini.google.com/app.
45. Copilot [Internet]. [cited 2024 Nov 15]. Available from: https://copilot.microsoft.com/.
46. Traver MA, Passman CM, LeRoy T, Passmore L, Assimos DG. Is the Internet a reliable source for dietary recommendations for stone formers?. J Endourol. 2009;23(4):715-7. https://doi.org/10.1089/end.2008.0490
47. Lambert K, Mullan J, Mansfield K, Koukomous A, Mesiti L. Evaluation of the quality and health literacy demand of online renal diet information. J Hum Nutr Diet. 2017;30(5):634-45. https://doi.org/10.1111/jhn.12466