Assessing AI-based chatbots accuracy in caloric estimation: A focus on traditional Turkish foods

Year 2025, Volume: 11 Issue: 5, 922 - 933, 04.09.2025

Hüsna Kaya Kaçar , Buse Sarıkaya

https://doi.org/10.18621/eurj.1684936

Abstract

Objectives: To evaluate the accuracy of three widely used AI-based chatbots - ChatGPT 4.0, Microsoft Copilot, and Gemini - in estimating the caloric values of traditional Turkish foods.

Methods: The accuracy of caloric information provided by the chatbots for 71 traditional Turkish foods selected from the Türkomp National Food Composition Database was assessed. Each chatbot was queried in Turkish using the standardised prompt: "What is the caloric content of [food name] per 100 grams?" Responses were recorded and, when necessary, converted to a per-100-gram basis to ensure consistency. Accuracy percentages were calculated by comparing chatbot responses to Türkomp reference values, and extreme deviations were adjusted to 0%. Mean accuracy scores and distribution across predefined accuracy intervals were analysed for each chatbot. Statistical analysis was conducted to determine the mean accuracy percentages and identify differences among the chatbots.

Results: ChatGPT 4.0 achieved the highest mean accuracy (81.62%±20.6%), followed closely by Microsoft Copilot (81.23%±20.7%), while Gemini demonstrated lower accuracy (70.99%±30.2) (P<0.05). A one-way ANOVA showed a statistically significant difference in the mean accuracy percentages among the chatbots (F(2, 210)=4.39, P=0.0136). Foods such as kefir, tahini halva, and walnut baklava were estimated with over 90% accuracy by all three chatbots, suggesting strengths in their training datasets and the relatively simple or standardised nutrient composition of these foods. However, significant discrepancies in caloric estimations were observed across the chatbots, likely due to differences in algorithms and database integrations.

Conclusions: The findings suggest that AI-based chatbots have the potential to serve as culturally relevant tools for dietary assessment. However, the results also emphasise the need for careful use and further development. While ChatGPT 4.0 and Microsoft Copilot performed better than Gemini, the study shows the need for improved algorithms and expanded training datasets to enhance the accuracy and reliability of chatbots in nutritional evaluation. This study contributes to the growing body of research on AI applications in dietetics and public health; however, addressing these limitations is crucial to ensure their practical utility. Optimising chatbot design for real-world use will require interdisciplinary collaboration among AI developers, nutritional scientists, and healthcare professionals.

Keywords

Artificial intelligence , caloric estimation , chatbot accuracy , dietary assessment , nutrition technology , traditional Turkish foods

Ethical Statement

This study did not involve human or animal subjects and used publicly accessible data. Therefore, Ethics Committee approval was not required. This study does not require informed consent.

References

• 1. Laranjo L, Dunn AG, Tong HL, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc. 2018;25(9):1248-1258. doi: 10.1093/jamia/ocy072.
• 2. Xu L, Sanders L, Li K, Chow JCL. Chatbot for Health Care and Oncology Applications Using Artificial Intelligence and Machine Learning: Systematic Review. JMIR Cancer. 2021;7(4):e27850. doi: 10.2196/27850.
• 3. Kulkarni R, Jaiswal CR. A survey on AI Chatbots. Int J Res Appl Sci Eng Technol. 2023;11(9):1738-1744. doi: 10.22214/ijraset.2023.55911.
• 4. Tsolakidis D, Gymnopoulos LP, Dimitropoulos K. Artificial Intelligence and Machine Learning Technologies for Personalized Nutrition: A Review. Informatics 2024;11(3):62. doi: 10.3390/informatics11030062
• 5. Abeltino A, Riente A, Bianchetti G, et al. Digital applications for diet monitoring, planning, and precision nutrition for citizens and professionals: a state of the art. Nutr Rev. 2025;83(2):e574-e601. doi: 10.1093/nutrit/nuae035.
• 6. Zhang J, Oh YJ, Lange P, Yu Z, Fukuoka Y. Artificial Intelligence Chatbot Behavior Change Model for Designing Artificial Intelligence Chatbots to Promote Physical Activity and a Healthy Diet: Viewpoint. J Med Internet Res. 2020;22(9):e22845. doi: 10.2196/22845.
• 7. Johnson KB, Wei WQ, Weeraratne D, et al. Precision Medicine, AI, and the Future of Personalized Health Care. Clin Transl Sci. 2021;14(1):86-93. doi: 10.1111/cts.12884.
• 8. Ahmed Z, Mohamed K, Zeeshan S, Dong X. Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine. Database (Oxford). 2020;2020:baaa010. doi: 10.1093/database/baaa010.
• 9. Budhwar P, Chowdhury S, Wood G, et al. Human resource management in the age of generative artificial intelligence: Perspectives and research directions on ChatGPT. Hum Resour Manag J. 2023;33(3):606-659. doi: 10.1111/1748-8583.12524.
• 10. Detopoulou P, Voulgaridou G, Moschos P, et al. Artificial intelligence, nutrition, and ethical issues: A mini-review. Clin Nutr Open Sci. 2023;50:46-56. doi: 10.1016/j.nutos.2023.07.001.
• 11. Côté M, Lamarche B. Artificial intelligence in nutrition research: perspectives on current and future applications. Appl Physiol Nutr Metab. 2022;47(1):1-8. doi: 10.1139/apnm-2021-0448.
• 12. Ponzo V, Goitre I, Favaro E, et al. Is ChatGPT an Effective Tool for Providing Dietary Advice? Nutrients. 2024;16(4):469. doi: 10.3390/nu16040469.
• 13. Hoang YN, Chen YL, Ho DKN, et al. Consistency and Accuracy of Artificial Intelligence for Providing Nutritional Information. JAMA Netw Open. 2023;6(12):e2350367. doi: 10.1001/jamanetworkopen.2023.50367.
• 14. Liao LL, Chang LC, Lai IJ. Assessing the Quality of ChatGPT's Dietary Advice for College Students from Dietitians' Perspectives. Nutrients. 2024;16(12):1939. doi: 10.3390/nu16121939.
• 15. Wang L, Wan Z, Ni C, et al. Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review. J Med Internet Res. 2024;26:e22769. doi: 10.2196/22769.
• 16. Chew HSJ. The Use of Artificial Intelligence-Based Conversational Agents (Chatbots) for Weight Loss: Scoping Review and Practical Recommendations. JMIR Med Inform. 2022;10(4):e32578. doi: 10.2196/32578.
• 17. Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29(8):1930-1940. doi: 10.1038/s41591-023-02448-8.
• 18. Fadhil A, Gabrielli S. Addressing challenges in promoting healthy lifestyles: the al-chatbot approach. PervasiveHealth '17: Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare. 2017:261-265. doi: 10.1145/3154862.3154914.
• 19. Bonan M, Ross H, et al. Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey. ACM Computing Surveys. 2023;56(2):1-40. doi: 10.1145/3605943.
• 20. Hagos DH, Battle R, Rawat DB. Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives. IEEE Trans Artif Intell. 2024;5(2):5873-5893. doi: 10.1109/TAI.2024.3444742.
• 21. Hadi MU, Qureshi R, Shah A, Irfan M, Zafar A, Shaikh MB, et al. Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects. Authorea preprints. 2023;1(3):1-26. https://www.techrxiv.org/users/618307/articles/682263-large-language-models-a-comprehensive-survey-of-its-applications-challenges-limitations-and-future-prospects.
• 22. Al-Surmi A, Bashiri M, Koliousis I. AI based decision making: combining strategies to improve operational performance. Int J Prod Res. 2022;60(14):4464-4486. doi: 10.1080/00207543.2021.1966540.
• 23. Khlaif ZN, Mousa A, Hattab MK, et al. The Potential and Concerns of Using AI in Scientific Research: ChatGPT Performance Evaluation. JMIR Med Educ. 2023;9:e47049. doi: 10.2196/47049.
• 24. Umoga UJ, Sodiya EO, Ugwuanyi ED, et al. Exploring the potential of AI-driven optimization in enhancing network performance and efficiency. Magna Scientia Advanced Research and Reviews. 2024;10(1):368-78. doi: 10.30574/msarr.2024.10.1.0028
• 25. Wang C, Liu X, Yue Y, et al. Survey on factuality in large language models: Knowledge, retrieval and domain-specificity. arXiv preprint arXiv:231007521. 2023. doi: 10.48550/arXiv.2310.07521.
• 26. Huang Y, Tang K, Chen M, Wang B. A comprehensive survey on evaluating large language model applications in the medical industry. arXiv preprint arXiv:240415777. 2024. doi:10.48550/arXiv.2404.15777
• 27. Annepaka Y, Pakray P. Large language models: a survey of their development, capabilities, and applications. Knowl Inf Syst. 2025;67(3):2967-3022. doi: 10.1007/s10115-024-02310-4
• 28. Giannakopoulos K, Kavadella A, Aaqel Salim A, Stamatopoulos V, Kaklamanos EG. Evaluation of the performance of generative AI large language models ChatGPT, Google Bard, and Microsoft Bing Chat in supporting evidence-based dentistry: comparative mixed methods study. J Med Internet Res. 2023;25:e51580. doi: 10.2196/51580
• 29. Gichoya JW, Banerjee I, Bhimireddy AR, et al. AI recognition of patient race in medical imaging: a modelling study. Lancet Digit Health. 2022;4(6):e406-e414. doi: 10.1016/S2589-7500(22)00063-2
• 30. Collenette J, Atkinson K, Bench-Capon T. Explainable AI tools for legal reasoning about cases: A study on the European Court of Human Rights. Artificial Intelligence. 2023;317:103861. doi: 10.1016/j.artint.2023.103861.
• 31. Zhang C, Meng Y. Bayesian deep learning: An enhanced AI framework for legal reasoning alignment. Comput Law Security Rev. 2024;55:106073. doi: 10.1016/j.clsr.2024.106073.
• 32. Spaić B, Jovanović M. Artificial Reason and Artificial Intelligence: The Legal Reasoning Capabilities of GPT-4. Annals of the Faculty of Law in Belgrade. 2024;72(3). doi: 10.51204/Anali_PFBU_24302A.
• 33. Shanmuganathan M. Behavioural finance in an era of artificial intelligence: Longitudinal case study of robo-advisors in investment decisions. J Behav Exp Finance. 2020;27:100297. doi: 10.1016/j.jbef.2020.100297.
• 34. Sultan MS, Sultan MS. Enhancing Corporate Financial Performance Through AI: A Novel AI Model for Forecasting Organizational Risk Management, CRM, and Operational Efficiency. Inter J Sci Res (IJSR). 2024;13(8):1758-1765. doi: 10.21275/SR24828091042.
• 35. Carels RA, Harper J, Konrad K. Qualitative perceptions and caloric estimations of healthy and unhealthy foods by behvioral weight loss participants. Appetite. 2006;46(2):199-206. doi: 10.1016/j.appet.2005.12.002.
• 36. Wang Z, Wang L, Hou Y, et al. Precision calories: A promising strategy for personalized health interventions in the precision nutrition framework. Trends Food Sci Technol. 2024;153:104727. doi: 10.1016/j.tifs.2024.104727
• 37. Almasi N, Rakıcıoğlu N. The Estimation of Standard Portion Sizes Using Food Photographic Booklet Among University Students in Turkey. Clin Exp Health Sci 2021;11(4):775-781. doi: 10.33808/clinexphealthsci.886274.
• 38. Çakmak M, Sarıışık M. An Investigation on the Basic Contents of the Main Dishes of the Turkish Cuisine. Rev Anais Bras de Est Tur (ABET). 2019;9(1, 2 e3):1-13. doi: 10.34019/2238-2925.2019.v9.27149.
• 39. Biringen Loker G, Ozkoc SO, Amoutzopoulos B, et al. Establishing a food composition database for Turkey based on European standards. Nutr Bull. 2011;36(2):254-258. doi: 10.1111/j.1467-3010.2011.01898.x.
• 40. Biringen Löker G, Amoutzopoulos B, Özge Özkoç S, Özer H, Şatir G, Bakan A. A pilot study on food composition of five Turkish traditional foods. Br Food J. 2013;115(3):394-408. doi: 10.1108/00070701311314219.
• 41. Kirk D, Catal C, Tekinerdogan B. Precision nutrition: A systematic literature review. Comput Biol Med. 2021;133:104365. doi: 10.1016/j.compbiomed.2021.104365.
• 42. Gami SJ, Dhamodharan B, Dutta PK, Gupta V, Whig P. Data Science for Personalized Nutrition Harnessing Big Data for Tailored Dietary Recommendations. In: Gami SJ, Dutta PK, Whing, eds., Nutrition Controversies and Advances in Autoimmune Disease. IGI Global Publishing. 2024: pp. 606-630.
• 43. Prabhakaran V, Qadri R, Hutchinson B. Cultural incongruencies in artificial intelligence. arXiv preprint arXiv:2211.13069. 2022. doi: 10.48550/arXiv.2211.13069.
• 44. Zhou L, Karidi T, Liu W, et al. Does mapo tofu contain coffee? probing llms for food-related cultural knowledge. arXiv preprint arXiv:24040.6833. 2024. doi: 10.48550/arXiv.2404.06833.
• 45. Winata GI, Hudi F, Irawan PA, et al. Worldcuisines: A massive-scale benchmark for multilingual and multicultural visual question answering on global cuisines. arXiv preprint arXiv:24101.2705. 2024. doi: 10.48550/arXiv.2410.12705.
• 46. Amoutzopoulos B, Page P, Roberts C, et al. Portion size estimation in dietary assessment: a systematic review of existing tools, their strengths and limitations. Nutr Rev. 2020;78(11):885-900. doi: 10.1093/nutrit/nuz107.
• 47. Abdusalomov A, Mukhiddinov M, Djuraev O, Khamdamov U, Abdullaev U, editors. AI-Based Estimation from Images of Food Portion Size and Calories for Healthcare Systems. International Conference on Intelligent Human Computer Interaction; 2023: Springer. doi: 10.1007/978-3-031-53830-8_2
• 48. Konstantakopoulos FS, Georga EI, Fotiadis DI. A review of image-based food recognition and volume estimation artificial intelligence systems. IEEE Rev Biomed Eng. 2023;17:136-52. doi:10.1109/RBME.2023.3283149.
• 49. ChatGPT 4.0. Open AI ChatGPT. Availabe online: https://chatgpt.com/ (accessed on 20.11.2024).
• 50. Copilot. Microsoft Copilot. Availabe online: https://copilot.microsoft.com/ (accessed on 23.11.2024).
• 51. Gemini. Google Gemini App. Availabe online: https://gemini.google.com/app (accessed on 10.11.2024).
• 52. Roumeliotis KI, Tselikas ND. Chatgpt and open-ai models: A preliminary review. Future Internet. 2023;15(6):192. doi: 10.3390/fi15060192.
• 53. Jaffe S, Parikh N, Butler JL, Farach A, Cambon A, Hecht B, et al., editors. Generative AI in Real-World Workplaces: The Second Microsoft Report on AI and Productivity Research. 2024.
• 54. Šarac M, editor Microsoft Copilot as a Transformative Tool in Business: Opportunities and Challenges 2025: Singidunum University. doi: 10.15308/Sinteza-2025-347-353
• 55. Akhtar ZB. From bard to Gemini: An investigative exploration journey through Google’s evolution in conversational AI and generative AI. Comput Artif Intell. 2024;2(1):1378. doi: 10.59400/cai.v2i1.1378.
• 56. Database, N.F.C. Turkish. Availabe online: https://turkomp.tarimorman.gov.tr/main (accessed on 10.11.2024).
• 57. Aiumtrakul N, Thongprayoon C, Arayangkool C, et al. Personalized Medicine in Urolithiasis: AI Chatbot-Assisted Dietary Management of Oxalate for Kidney Stone Prevention. J Pers Med. 2024;14(1):107. doi: 10.3390/jpm14010107.
• 58. Bayram HM, Ozturkcan SA. AI showdown: info accuracy on protein quality content in foods from ChatGPT 3.5, ChatGPT 4, bard AI and bing chat. Br Food J. 2024;126(9):3335-3346. doi: 10.1108/bfj-02-2024-0158.
• 59. Naja F, Taktouk M, Matbouli D, et al. Artificial intelligence chatbots for the nutrition management of diabetes and the metabolic syndrome. Eur J Clin Nutr. 2024;78(10):887-896. doi: 10.1038/s41430-024-01476-y.
• 60. Qarajeh A, Tangpanithandee S, Thongprayoon C, et al. AI-Powered Renal Diet Support: Performance of ChatGPT, Bard AI, and Bing Chat. Clin Pract. 2023;13(5):1160-1172. doi: 10.3390/clinpract13050104.
• 61. Hieronimus B, Hammann S, Podszun MC. Can the AI tools ChatGPT and Bard generate energy, macro- and micro-nutrient sufficient meal plans for different dietary patterns? Nutr Res. 2024;128:105-114. doi: 10.1016/j.nutres.2024.07.002.
• 62. Salazar GZ, Zúñiga D, Vindel CL, et al. Efficacy of AI Chats to Determine an Emergency: A Comparison Between OpenAI's ChatGPT, Google Bard, and Microsoft Bing AI Chat. Cureus. 2023;15(9):e45473. doi: 10.7759/cureus.45473.
• 63. Safitri S, Mantoro T, Bhakti MAC, Wandy W. Cooking and Food Information Chatbot System using GPT-3. In Proceedings of 2023 IEEE 9th International Conference on Computing, Engineering and Design (ICCED); 2023:1-6. doi: 10.1109/ICCED60214.2023.10425553.
• 64. Cathcart A, Wiggins M, Kelly W, et al. Recognition of Culturally Diverse Cuisine in Popular Nutrition Apps. Curr Dev Nutr. 2022;6:872. doi: 10.1093/cdn/nzac066.002
• 65. Li X, Yin A, Choi HY, Chan V, Allman-Farinelli M, Chen J. Evaluating the Quality and Comparative Validity of Manual Food Logging and Artificial Intelligence-Enabled Food Image Recognition in Apps for Nutrition Care. Nutrients. 2024;16(15):2573. doi: 10.3390/nu16152573.
• 66. Thames Q, Karpur A, Norris W, et al. Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 8899-8907. doi: 10.1109/CVPR46437.2021.00879.
• 67. Yang Z, Khatibi E, Nagesh N. ChatDiet: Empowering personalized nutrition-oriented food recommender chatbots through an LLM-augmented framework. Smart Health. 2024; 32(1):100465. doi: 10.1016/j.smhl.2024.100465
• 68. Fadhil A. Can a chatbot determine my diet?: Addressing challenges of chatbot application for meal recommendation. ArXiv Preprint ArXiv:1802.09100. 2018. doi: 10.48550/arXiv.1802.09100
• 69. Shonkoff E, Cara KC, Pei X, et al. The State of the Science on Artificial Intelligence-Based Dietary Assessment Methods That Use Digital Images: A Scoping Review. Curr Dev Nutr. 2022;6(Suppl 1):534. doi: 10.1093/cdn/nzac077.037.
• 70. Limketkai BN, Mauldin K, Manitius N, Jalilian L, Salonen BR. The Age of Artificial Intelligence: Use of Digital Technology in Clinical Nutrition. Curr Surg Rep. 2021;9(7):20. doi: 10.1007/s40137-021-00297-3.
• 71. Sharma SK, Gaur S. Optimizing Nutritional Outcomes: The Role of AI in Personalized Diet Planning. Int J Res Publication Sem. 2024;15(2)::107-116. doi: 10.36676/jrps.v15.i2.15.
• 72. Lo FP, Qiu J, Wang Z, et al. Dietary Assessment with Multimodal ChatGPT: A Systematic Analysis. IEEE J Biomed Health Inform. 2024;28(12):7577-7587. doi: 10.1109/JBHI.2024.3417280.
• 73. Haman M, Školník M, Lošťák M. AI dietician: Unveiling the accuracy of ChatGPT's nutritional estimations. Nutrition. 2024;119:112325. doi: 10.1016/j.nut.2023.112325.
• 74. Hu X, Chen J, Li X, et al. Do large language models know about facts? arXiv preprint arXiv:2310.05177. 2023. doi: 10.48550/arXiv.2310.05177.
• 75. Venkatasubramanian V. Do large language models “understand” their knowledge? AIChE J. 2025;71(3):e18661. doi: 10.1002/aic.18661.
• 76. Azimi I, Qi M, Wang L, Rahmani AM, Li Y. Evaluation of LLMs accuracy and consistency in the registered dietitian exam through prompt engineering and knowledge retrieval. Sci Rep. 2025;15(1):1506. doi: 10.1038/s41598-024-85003-w.
• 77. Sclar M, Choi Y, Tsvetkov Y, Suhr A. Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324. 2023. doi: 10.48550/arXiv.2310.11324.
• 78. Guan B, Roosta T, Passban P, Rezagholizadeh M. The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMs. arXiv preprint arXiv:2502.04134. 2025. doi: 10.48550/arXiv.2502.04134.