Research Article
BibTex RIS Cite

Üretken Yapay Zekânın Satın Alma Süreçlerinde Karar Destek Aracı Olarak Kullanılması: ChatGPT, CoPilot ve Gemini Araçlarının Karşılaştırılması

Year 2025, Volume: 5 Issue: 2, 101 - 112
https://doi.org/10.70101/ussmad.1758962

Abstract

İşletmelerde satın alma taleplerinin doğru önceliklendirilmesi, iş sürekliliği ve kaynak yönetimi açısından kritik öneme sahiptir. Gün içinde farklı departmanlarca oluşturulan talepler genellikle satın alma birimi tarafından sübjektif olarak sıralanmakta, bu da bazı acil taleplerin geri planda kalmasına yol açabilmektedir. Sürecin insan kontrolünde yürütülmesi hem zaman kaybına hem de hatalı önceliklendirmelere neden olmaktadır. Bu çalışmada, bir üretim işletmesinin ERP sistemine API aracılığıyla entegre edilen üç üretken yapay zekâ aracı (ChatGPT-4.5, Microsoft CoPilot ve Google Gemini) kullanılarak 100 satın alma talebi önce “Acil”, “Normal” ve “Acil Değil”, ardından “Acil” ve “Normal” biçiminde sınıflandırılmıştır. Yapay zekâ modellerinin sonuçları, satın alma personelinin sınıflandırmalarıyla karşılaştırılarak doğruluk, Cohen’s Kappa, precision, recall ve F1-score metrikleri üzerinden değerlendirilmiştir. Ayrıca üretken yapay zeka araçlarının doğru yanıt verme performansları Pearson Ki-kare testiyle incelenmiş; sonuçlar, araçlar arasında anlamlı bir karşılıklı bağımlılık olduğunu ve özellikle Copilot ile Gemini’nin hem üçlü hem de ikili sınıflandırmalarda yüksek düzeyde uyum sergilediğini göstermiştir. Bulgular, her üç modelin ikili sınıflandırmada başarılı performans gösterdiğini; özellikle CoPilot’un diğer modellere göre daha yüksek doğruluk sağladığını ortaya koymuştur. Çalışma, üretken yapay zekâ araçlarının satın alma süreçlerinde ön sınıflandırma, hız ve iş gücü tasarrufu açısından etkili bir karar destek aracı olarak kullanılabileceğini göstermektedir.

References

  • Ahmadi, S. (2023). Open AI and its impact on fraud detection in the financial industry. Journal of Knowledge Learning and Science Technology ISSN: 2959–6386 (Online), 2(3), 263–281.
  • Alhur, A. (2024). Redefining healthcare with artificial intelligence (AI): The contributions of ChatGPT, Gemini, and Co-Pilot. Cureus, 16(4): e57795. https://doi.org/10.7759/cureus.57795
  • Al‐Worafi, Y., Hermansyah, A., Tan, C., Choo, C., Bouyahya, A., Paneerselvam, G., & Ming, L. (2023). Applications, benefits, and risks of ChatGPT in medical and health sciences research: An experimental study. Progress in Microbes & Molecular Biology, 6(1), 1–13. https://doi.org/10.36877/pmmb.a0000337
  • Aydın, F. O., Aksoy, B. K., & Ceylan, A. (2024). Refraktif cerrahide sık sorulan sorular için ChatGPT 3.5, ChatGPT 4.0, Gemini ve Microsoft CoPilot tarafından oluşturulan yanıtların okunabilirliği ve uygunluğu. Turkish Journal of Ophthalmology, 54(6), 313-317. https://doi:10.4274/tjo.galenos.2024.28234.
  • Ayhan, M., & Kılıç, Z. (2024). Yapay zekâ modellerinin hadis tarihi sorularına verdiği yanıtların karşılaştırmalı analizi: ChatGPT ve Gemini örneği. Dinbilimleri Akademik Araştırma Dergisi, 24(3), 137-159.
  • Banimelhem, O., & Amayreh, W. (2023). The performance of ChatGPT in emotion classification. The 14th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan.
  • Boduroğlu, E., Koç, O., & Yiğiter, M. S. (2023). Madde güçlüklerinin tahmin edilmesinde uzman görüşleri ve chatgpt performansının karşılaştırılması / comparison of expert opinions and ChatGPT performance in predicting item difficulties. Disiplinlerarası Eğitim Araştırmaları Dergisi, 7(15), 202-210. https://doi.org/10.57135/jier.1296255
  • Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1), 198-204. https://doi.org/10.5281/ zenodo.7716416
  • Brigo, F., Broggi, S., Strigaro, G., Olivo, S., Tommasini, V., Massar, M., ... & Zaboli, A. (2025). Artificial intelligence (ChatGPT 4.0) vs. Human expertise for epileptic seizure and epilepsy diagnosis and classification in Adults: An exploratory study. Epilepsy & Behavior, 166, 110364.
  • Brynjolfsson, E., Li, D., & Raymond, L. R. (2023). Generative AI at work (No. w31161). National Bureau of Economic Research.
  • Bulut, C. (2025). Sağlık kurumlarında yapay zekâ destekli karar destek sistemlerinin kullanımı. Uluslararası Sağlık Yönetimi ve Stratejileri Araştırma Dergisi, 11(1), 27-37.
  • Butean, O., Felek, T., Erkal, D., & Er, K. (2025). Comparison of responses from different large language models on dens invaginatus cases: a pilot study. Recep Tayyip Erdoğan Üniversitesi Diş Hekimliği Fakültesi 3. Ulusal Öğrenci Kongresi, Rize, Turkey.
  • Chatterjee, S., Liu, C., Rowland, G., & Hogarth, T. (2024). The impact of AI tools on engineering at ANZ Bank: An empirical study on GitHub CoPilot within a corporate environment. 10th International Conference on Software Engineering (SEC 2024), Melbourne, Australia. https://doi.org/10.5121/csit.2024.140702
  • Chu, M. N. (2023). Assessing the benefits of ChatGPT for business: an empirical study on organizational performance. IEEE Access, 11, 76427–76436.
  • Coşar, M. (2024). Kurumsal bilgi güvenliği yönetiminde yapay zekâ destekli risk analizi. Denetişim, 31(1), 144-155. https://doi.org/10.58348/denetisim.1519578
  • Crewson, P. E. (2005). Reader agreement studies. American Journal of Roentgenology, 184(5), 1391–1397.
  • Çeber, B. (2024). Reklam ajanslarında yapay zekâ kullanımı: sektör profesyonellerinin ChatGPT ve Midjourney deneyimlerine yönelik bir araştırma. Erciyes İletişim Dergisi, 11(2), 583-606. https://doi.org/10.17680/erciyesiletisim.1439479
  • Çelik, A., Balcıoğlu, Y., & Altındağ, E. (2025). Understanding user engagement with Gemini through sentiment analysis and concept mapping. International Journal of Business Analytics, 12(1), 1-14. https://doi.org/10.4018/ijban.375429
  • Dabbas, W. F., Odeibat, Y. M., Alhazaimeh, M., Hiasat, M. Y., Alomari, A. A., Marji, A., ... & Momani, G. M. (2024). Accuracy of ChatGPT in neurolocalization. Cureus 16(4): e59143. https://doi.org/10.7759/cureus.59143
  • Dalgıç, A. (2023). Restoran işletmelerine yapılan olumsuz yorumları ChatGPT değerlendirebilir mi? TripAdvisor’da bir uygulama. İşletme Araştırmaları Dergisi, 15(4), 3069-3080.
  • Darlan, D., Ajani, O. S., An, J. W., Bae, N. Y., Lee, B., Park, T., & Mallipeddi, R. (2025). SmartBerry for AI-based growth stage classification and precision nutrition management in strawberry cultivation. Scientific Reports, 15(1), 14019. https://doi.org/10.1038/s41598-025-97168-z
  • Das, D., Kumar, N., Longjam, L.A., Sinha, R., Deb Roy, A., Mondal, H., et al., (2023). Assessing the capability of ChatGPT in answering first- and second-order knowledge questions on microbiology as per competency-based medical education curriculum. Cureus, 15(3), e36034 (Mar).
  • Dave, T., Athaluri, S., & Singh, S. (2023). ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Frontiers in Artificial Intelligence, 6. https://doi.org/10.3389/frai.2023.1169595
  • Deloitte AI Institute. (2023). A new frontier in artificial intelligence: Implications of Generative AI for businesses. Deloitte. https://www.deloitte.com/content/dam/assets-zone3/us/en/docs/services/consulting/2024/us-ai-institute-generative-artificial-intelligence.pdf
  • Dis, E., Bollen, J., Zuidema, W., Rooij, R., & Bockting, C. (2023). Chatgpt: five priorities for research. Nature, 614(7947), 224-226. https://doi.org/10.1038/d41586-023-00288-7
  • Duarte, F. (2025). Number of ChatGPT users (July 2025). Exploding Topics. https://explodingtopics.com/blog/chatgpt-users
  • Emekli, E., Özel, B., & Emekli, E., (2025). Psikiyatri eğitiminde klinik akıl yürütme becerilerinin geliştirilmesi: chatgpt ile otomatik senaryo ve soru üretimi. 10. Psikiyatri Zirvesi ve 17. Anksiyete Kongresi, Antalya, Turkey.
  • En, L., Tha, N., & Thuan, T. (2024). Exploring the application of ChatGPT in learning English of students – a study on perception of sophomores at Nguyen Tat Thanh University. Journal of Science and Technology, 7(3), 41-47. https://doi.org/10.55401/fg15by35
  • Erdem, E. (2024a). Yapay zeka uygulamalarinin sosyal bilim alaninda yapilan çalişmalarda uygulanabilirliği: Chatgpt, Bing ve Youchat örneği. İletişim Bilimi Araştırmaları Dergisi, 4(3), 218-234.
  • Erdem, M. B. (2024b). Lojistik Firma Web Sitelerinin ChatGPT İle Analiz Edilmesi. Erciyes Akademi, 38(4), 1085-1099. https://doi.org/10.48070/erciyesakademi.1596551
  • Fabijan, A., Polis, B., Fabijan, R., Zakrzewski, K., Nowosławska, E., & Zawadzka-Fabijan, A. (2023). Artificial ıntelligence in scoliosis classification: an ınvestigation of language-based models. Journal of Personalized Medicine, 13(12), 1695. https://doi.org/10.3390/jpm13121695
  • Gelmiş, M., Ayten, A., Özsoy, Ç., Bulut, B., & Köse, M. G. (2025). Comparing ChatGPT and Google Gemini in urology: Which AI model provides superior patient education on penile prosthesis?. Androloji Bülteni, 27(1), 8-12.
  • Göktaş, L. S. (2023). ChatGPT uzaktan eğitim sınavlarında başarılı olabilir mi? Turizm alanında doğruluk ve doğrulama üzerine bir araştırma. Journal of Tourism and Gastronomy Studies, 11(2), 892-905. https://doi.org/10.21325/jotags.2023.1224
  • Gupta, R., Hamid, A., Jhaveri, M., Patel, N., & Suthar, P. (2024). Comparative evaluation of AI models such as ChatGPT 3.5, ChatGPT 4.0, and Google Gemini in neuroradiology diagnostics. Cureus.16(8): e67766 https://doi.org/10.7759/cureus.67766
  • Güney, A., & Özcan, E. (2025). Yapay zekanın muhasebede kullanımı üzerine bir araştırma. Journal of Accounting Institute, (72), 1-13. https://doi.org/10.26650/MED.1492095
  • Hacıbey, İ. & Halis, A. (2025). Assessment of artificial intelligence performance in answering questions on onabotulinum toxin and sacral neuromodulation. Investigative and Clinical Urology, 66(3), 188-193. https://doi.org/10.4111/icu.20250040
  • Harada, T., Sato, S. & Nishiura, M. (2025). Using Generative AI to Improve Library Book Classification Accuracy: The Role of Increased Training Data. Oliver, G., Frings-Hessami, V., Du, J.T., Tezuka, T. (eds) Sustainability and Empowerment in the Context of Digital Libraries. ICADL 2024. Lecture Notes in Computer Science, vol 15494. Springer, Singapore. https://doi.org/10.1007/978-981-96-0868-3_20
  • Huang, S., Grady, P. & GPT-3. (2022, September 19). Generative AI: A creative new world. Sequoia. https://www.sequoiacap.com/article/generative-ai-a-creative-new-world/
  • Hussain, A., & Ahmed, A. (2023). Auto-classification of harmonized tariff codes using ChatGPT. 4th International Conference on Distributed Sensing and Intelligent Systems (ICDSIS 2023), Dubai, UAE.
  • İbiş, S. (2025). Yapay zekâ teknolojilerinin gastronomi turizminde kullanımı: ChatGPT örneği. Güncel Turizm Araştırmaları Dergisi, 9(1), 109-131. https://doi.org/10.32572/guntad.1518594
  • Jalil, S., Rafi, S., LaToza, T. D., Moran, K., & Lam, W. (2023). ChatGPT and software testing education: Promises & perils. IEEE International Conference on Software Testing, Verification, and Validation Workshops (ICSTW), Dublin, Ireland.
  • Kaftan, A., Hussain, M., & Naser, F. (2024). Response accuracy of ChatGPT 3.5 CoPilot and Gemini in interpreting biochemical laboratory data: A pilot study. Scientific Reports, 14(1), https://doi.org/10.1038/s41598-024-58964-1
  • Kaiser, K., Hughes, A., Yang, A., Turk, A., Mohanty, S., Gonzalez, A., … & Ellis, R. (2024). Accuracy and consistency of publicly available large language models as clinical decision support tools for the management of colon cancer. Journal of Surgical Oncology, 130(5), 1104–1110. https://doi.org/10.1002/jso.27821
  • Kaplan, A., & Haenlein, M. (2019). Siri, Siri, in my hand: Who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business Horizons, 62(1), 15–25.
  • Koubaa, A. (2023). GPT-4 vs. GPT-3.5: A concise showdown. Prerprints. https://doi.org/10.20944/preprints202303.0422.v1
  • Kutlucan, E., & Seferoğlu, S. S. (2024). Eğitimde yapay zekâ kullanımı: ChatGPT’nin KEFE ve PEST analizi. Türk Eğitim Bilimleri Dergisi, 22(2), 1059-1083. https://doi.org/10.37217/tebd.1368821
  • Küçük, D., & Fazlı, C. (2024) Hukuki metinlerin otomatik işlenmesinde yapay zekâ teknolojilerinin kullanımı. Bilişim Hukuku Dergisi, 6(1):1-23. https://doi.org/10.55009/bilisimhukukudergisi.1450588.
  • Küçüker, M. (2023). Muhasebede yapay zekâ uygulamaları: ChatGPT’nin muhasebe sınavı. Firat University Journal of Social Sciences, 33(2), 875-888. https://doi.org/10.18069/firatsbed.1289885
  • Lawton, G. (2023, July 20). What is generative AI? Everything you. Techtarget. https://www.techtarget.com/searchenterpriseai/definition/generative-AI
  • Lee, K., Kessler, D., Caglič, I., Kuo, Y., Shaida, N., & Barrett, T. (2024). Assessing the performance of ChatGPT and Bard/Gemini against radiologists for prostate imaging-reporting and data system classification based on prostate multiparametric MRI text reports. British Journal of Radiology, 98(1167), 368–374. https://doi.org/10.1093/bjr/tqae236
  • Li, M., & Guenier, A. (2024). ChatGPT and health communication. International Journal of E-Health and Medical Communications, 15(1), 1–26. https://doi.org/10.4018/ijehmc.349980
  • Liu, C., Wei, M., Qin, Y., Zhang, M., Jiang, H., Xu, J., ... & Zhou, J. (2024). Harnessing large language models for structured reporting in breast ultrasound: a comparative study of OpenAI (GPT-4.0) and Microsoft Bing (GPT-4). Ultrasound in Medicine & Biology, 50(11), 1697-1703.
  • Liu, Y., He, S., Chen, H., Qing, W., Su, H., & Deng, G. (2023). Investigation on response strategies for the impact of ChatGPT technology application on college Chinese language education. International Journal of Educational Innovation and Science, 4(1), 156–164. https://doi.org/10.38007/ijeis.2023.040114
  • Makridakis, S. (2017). Forecasting the impact of artificial intelligence (AI). Foresight: The International Journal of Applied Forecasting, 47, 7-13.
  • McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (1955). A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955. AI Magazine, 27(4), 12.
  • Mich, L., & Garigliano, R. (2023). ChatGPT for e-tourism: A technological perspective. Information Technology & Tourism, 25, 1-12. https://doi.org/10.1007/s40558-023-00248-x
  • Moor, J. (2006). The Dartmouth College artificial intelligence conference: The next fifty years. AI Magazine, 27(4), 87–87.
  • Murindanyi, S., Wycliff Mugalu, B., Nakatumba-Nabende, J., & Marvin, G. (2023). Interpretable machine learning for predicting customer churn in retail banking. 7th International Conference on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India. https://doi.org/10. 1109/ICOEI56765.2023.10125859
  • Okur, H., & Ekşi, A. (2024). Yapay zekâ (AI) teknolojilerinin islam eşya hukuku bilgisi üzerine bir değerlendirme: Chatgpt ve Google Gemini karşılaştırması. Dinbilimleri Akademik Araştırma Dergisi. 24(3), 29-54. https://doi.org/10.33415/daad.1582059.
  • Osama, M., & Afridi, S. (2023). ChatGPT: a new era in research writing assistance. Journal of the Pakistan Medical Association, 73(9), 1929-1930. https://doi.org/10.47391/jpma.9183
  • Pavlik, V. J. (2023). Collaborating with ChatGPT: considering the implications of generative artificial intelligence for journalism and media education. Journalism & Mass Communication Educator, 78(1), 84-93.
  • Precedence Research. (2025, May 22). Generative aı market. Generative aı market size to hit USD 1005.07 bn by 2034. Precedence Research. https://www.precedenceresearch.com/generative-ai-market
  • Rane, N., Choudhary, S., & Rane, J. (2024). Gemini or ChatGPT? Capability, Performance, and Selection of Cutting-edge Generative Artificial Intelligence (AI) in Business Management. Studies in Economics and Business Relations, 5(1), 40–50. https://doi.org/10.48185/sebr.v5i1.1051
  • Rossettini, G., Rodeghiero, L., Corradi, F., Cook, C., Pillastrini, P., Turolla, A., … & Palese, A. (2024). Comparative accuracy of ChatGPT-4, Microsoft CoPilot, and Google Gemini in the Italian entrance test for healthcare sciences degrees: a cross-sectional study. BMC Medical Education, 24(1), 1-9 https://doi.org/10.1186/s12909-024-05630-9
  • Sabir, A., Ali, H. A., & Aljabery, M. A. (2024). ChatGPT tweets sentiment analysis using machine learning and data classification. Informatica, 48(7), 103–112. https://doi.org/10.31449/inf.v48i7.5535
  • Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare, 11(6), 887. https://doi.org/10.3390/healthcare11060887
  • Salman, I. M., Ameer, O. Z., Khanfar, M. A., & Hsieh, Y-H. (2025). Artificial intelligence in healthcare education: evaluating the accuracy of ChatGPT, CoPilot, and Google Gemini in cardiovascular pharmacology. Front Med (Lausanne). 12:1495378. https://doi: 10.3389/fmed.2025.1495378
  • Shaheen, A., Afflitto, G. G., & Swaminathan, S. S. (2025). ChatGPT-Assisted classification of postoperative bleeding following microinvasive glaucoma surgery using electronic health record data. Ophthalmology Science, 5(1), 100602.
  • Shihab, R., Sultana, N., & Samad, A. (2023). Revisiting the use of ChatGPT in business and educational fields: possibilities and challenges. BULLET: Jurnal Multidisiplin Ilmu, 2(3), 534–545. https://journal.mediapublikasi.id/index.php/bullet/article/view/2761
  • Smales, L. A. (2023). Classification of RBA monetary policy announcements using ChatGPT. Finance Research Letters, 58, 104514. https://doi.org/10.1016/j.frl.2023.104514
  • Sudirjo, F., Diantoro, K., Al-Gasawneh, J. A., Khootimah Azzaakiyyah, H., & Almaududi Ausat, A. M. (2023). Application of ChatGPT in ımproving customer sentiment analysis for businesses. Jurnal Teknologi Dan Sistem Informasi Bisnis, 5(3), 283-288. https://doi.org/10.47233/jteksis.v5i3.871
  • Şensoy, E., & Çıtırık, M. (2025). Glokom konusunda yapay zeka sohbet botlarının performans analizi: ChatGPT-3.5, Gemini ve CoPilot’un İngilizce ve Türkçe sorulardaki başarı farklılıkları. MN Oftalmoloji, 32(1), 17-21.
  • Thormnundsson, B. (2023, June 20). ChatGPT-statistics & facts. Statista. https://www.statista.com/topics/10446/chatgpt/#topicOverview
  • Uyar, T. (2024). Chat gpt’nin serbest mantıksal safsata tespitinde kullanımı. Yeni Medya Elektronik Dergisi, 8(1), 144-179.
  • Ventura, M., & Filho, A. (2023). ChatGPT: limitations, challenges, and potential applications. Brazilian Journal of Science, 3(1), 65–68. https://doi.org/10.14295/bjs.v3i1.427
  • Waisberg, E., Ong, J., Masalkhi, M., Zaman, N., Sarker, P., Lee, A. G., & Tavakkoli, A. (2024). Google's AI chatbot "Bard": a side-by-side comparison with ChatGPT and its utilization in ophthalmology. Eye (London, England), 38(4), 642–645. https://doi.org/10.1038/s41433-023-02760-0
  • Wiles, J. (2023, January 26). Beyond Chatgpt: the future of generative AI for enterprises. Gartner. https://www.gartner.com/en/articles/beyond-chatgpt-the-future-of-generative-ai-for-enterprises
  • Wong, M., Guo, S., Hang, C., Ho, S., & Tan, C. (2023). Natural language generation and understanding big code for AI-assisted programming: a review. Entropy, 25(6), 888. https://doi.org/10.3390/e25060888
  • Wulcan, J. M., Jacques, K. L., Lee, M. A., Kovacs, S. L., Dausend, N., Prince, L. E., ... & Keller, S. M. (2025). Classification performance and reproducibility of GPT-4 Omni for information extraction from veterinary electronic health records. Frontiers in Veterinary Science, 11, 1490030.
  • Guo, Y., & Wan, Z. (2024). Performance Evaluation of Multimodal Large Language Models (LLaVA and GPT-4-based ChatGPT) in Medical Image Classification Tasks. 12th International Conference on Healthcare Informatics (ICHI), Orlando, FL, USA. Doi: 10.1109/ICHI61247.2024.00080.
  • Dökme Yağar, S. (2023). ChatGPT’nin sağlık alanındaki potansiyel kullanımına ilişkin çıkarımlar. Business & Management Studies: An International Journal, 11(3), 1226–1240. https://doi.org/10.15295/bmij.v11i3.2264.
  • Yalçın-Çelik, A., & Çoban, Ö. K. (2023). Kimya sorularının cevaplanmasında yapay zekâ tabanlı sohbet robotlarının performansının incelenmesi. Journal of Turkish Educational Sciences, 21(3), 1540-1561.
  • Yılmaz, M., & Çil, G. (2024). Yapay zekâ, erektil disfonksiyon ile ilgili sorulara nasıl yanıtlar veriyor: ChatGPT vs Google AI. Androloji Bülteni 26(2), 116−122. https://doi.org/10.24898/tandro.2024.12989
  • Zhang, B., Liang, P., Zhou, X., Ahmad, A., & Waseem, M. (2023). Demystifying practices, challenges, and expected features of using GitHub CoPilot. International Journal of Software Engineering and Knowledge Engineering, 33(11n12), 1653–1672. https://doi.org/10.1142/s0218194023410048
  • Zhu, J. J., Jiang, J., Yang, M., & Ren, Z. J. (2023). ChatGPT and environmental research. Environmental Science & Technology, 57. 1-4. https://doi.org/10.1021/acs.est.3c01818

Using Generative Artificial Intelligence as a Decision Support Tool in Purchasing Processes: Comparison of ChatGPT, CoPilot, and Gemini Tools

Year 2025, Volume: 5 Issue: 2, 101 - 112
https://doi.org/10.70101/ussmad.1758962

Abstract

Accurate prioritization of purchase requests in enterprises is critical for ensuring business continuity and effective resource management. Throughout the day, requests generated by different departments are usually ranked subjectively by the purchasing unit, which may cause some urgent requests to be deprioritized. Managing the process under human control leads to time loss and inaccurate prioritization. This study integrated three generative artificial intelligence tools—ChatGPT-4.5, Microsoft CoPilot, and Google Gemini—into a manufacturing company’s ERP system via an API. A total of 100 purchase requests were classified first into three categories (“Urgent,” “Normal,” and “Not Urgent”) and then into two categories (“Urgent” and “Normal”). The results produced by the AI models were compared with the classifications made by the purchasing staff and evaluated using accuracy, Cohen’s Kappa, precision, recall, and F1-score metrics. In addition, the correct response performance of generative artificial intelligence tools was analyzed using the Pearson Chi-square test; the results revealed a significant interdependence among the tools, with Copilot and Gemini showing an exceptionally high consistency across both triple and binary classifications. The findings revealed that all three models performed well in the binary classification, with CoPilot achieving higher accuracy than the others. The study demonstrates that generative AI tools can be practical decision-support systems in purchasing processes, offering significant advantages in preliminary classification, efficiency, and time savings.

References

  • Ahmadi, S. (2023). Open AI and its impact on fraud detection in the financial industry. Journal of Knowledge Learning and Science Technology ISSN: 2959–6386 (Online), 2(3), 263–281.
  • Alhur, A. (2024). Redefining healthcare with artificial intelligence (AI): The contributions of ChatGPT, Gemini, and Co-Pilot. Cureus, 16(4): e57795. https://doi.org/10.7759/cureus.57795
  • Al‐Worafi, Y., Hermansyah, A., Tan, C., Choo, C., Bouyahya, A., Paneerselvam, G., & Ming, L. (2023). Applications, benefits, and risks of ChatGPT in medical and health sciences research: An experimental study. Progress in Microbes & Molecular Biology, 6(1), 1–13. https://doi.org/10.36877/pmmb.a0000337
  • Aydın, F. O., Aksoy, B. K., & Ceylan, A. (2024). Refraktif cerrahide sık sorulan sorular için ChatGPT 3.5, ChatGPT 4.0, Gemini ve Microsoft CoPilot tarafından oluşturulan yanıtların okunabilirliği ve uygunluğu. Turkish Journal of Ophthalmology, 54(6), 313-317. https://doi:10.4274/tjo.galenos.2024.28234.
  • Ayhan, M., & Kılıç, Z. (2024). Yapay zekâ modellerinin hadis tarihi sorularına verdiği yanıtların karşılaştırmalı analizi: ChatGPT ve Gemini örneği. Dinbilimleri Akademik Araştırma Dergisi, 24(3), 137-159.
  • Banimelhem, O., & Amayreh, W. (2023). The performance of ChatGPT in emotion classification. The 14th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan.
  • Boduroğlu, E., Koç, O., & Yiğiter, M. S. (2023). Madde güçlüklerinin tahmin edilmesinde uzman görüşleri ve chatgpt performansının karşılaştırılması / comparison of expert opinions and ChatGPT performance in predicting item difficulties. Disiplinlerarası Eğitim Araştırmaları Dergisi, 7(15), 202-210. https://doi.org/10.57135/jier.1296255
  • Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1), 198-204. https://doi.org/10.5281/ zenodo.7716416
  • Brigo, F., Broggi, S., Strigaro, G., Olivo, S., Tommasini, V., Massar, M., ... & Zaboli, A. (2025). Artificial intelligence (ChatGPT 4.0) vs. Human expertise for epileptic seizure and epilepsy diagnosis and classification in Adults: An exploratory study. Epilepsy & Behavior, 166, 110364.
  • Brynjolfsson, E., Li, D., & Raymond, L. R. (2023). Generative AI at work (No. w31161). National Bureau of Economic Research.
  • Bulut, C. (2025). Sağlık kurumlarında yapay zekâ destekli karar destek sistemlerinin kullanımı. Uluslararası Sağlık Yönetimi ve Stratejileri Araştırma Dergisi, 11(1), 27-37.
  • Butean, O., Felek, T., Erkal, D., & Er, K. (2025). Comparison of responses from different large language models on dens invaginatus cases: a pilot study. Recep Tayyip Erdoğan Üniversitesi Diş Hekimliği Fakültesi 3. Ulusal Öğrenci Kongresi, Rize, Turkey.
  • Chatterjee, S., Liu, C., Rowland, G., & Hogarth, T. (2024). The impact of AI tools on engineering at ANZ Bank: An empirical study on GitHub CoPilot within a corporate environment. 10th International Conference on Software Engineering (SEC 2024), Melbourne, Australia. https://doi.org/10.5121/csit.2024.140702
  • Chu, M. N. (2023). Assessing the benefits of ChatGPT for business: an empirical study on organizational performance. IEEE Access, 11, 76427–76436.
  • Coşar, M. (2024). Kurumsal bilgi güvenliği yönetiminde yapay zekâ destekli risk analizi. Denetişim, 31(1), 144-155. https://doi.org/10.58348/denetisim.1519578
  • Crewson, P. E. (2005). Reader agreement studies. American Journal of Roentgenology, 184(5), 1391–1397.
  • Çeber, B. (2024). Reklam ajanslarında yapay zekâ kullanımı: sektör profesyonellerinin ChatGPT ve Midjourney deneyimlerine yönelik bir araştırma. Erciyes İletişim Dergisi, 11(2), 583-606. https://doi.org/10.17680/erciyesiletisim.1439479
  • Çelik, A., Balcıoğlu, Y., & Altındağ, E. (2025). Understanding user engagement with Gemini through sentiment analysis and concept mapping. International Journal of Business Analytics, 12(1), 1-14. https://doi.org/10.4018/ijban.375429
  • Dabbas, W. F., Odeibat, Y. M., Alhazaimeh, M., Hiasat, M. Y., Alomari, A. A., Marji, A., ... & Momani, G. M. (2024). Accuracy of ChatGPT in neurolocalization. Cureus 16(4): e59143. https://doi.org/10.7759/cureus.59143
  • Dalgıç, A. (2023). Restoran işletmelerine yapılan olumsuz yorumları ChatGPT değerlendirebilir mi? TripAdvisor’da bir uygulama. İşletme Araştırmaları Dergisi, 15(4), 3069-3080.
  • Darlan, D., Ajani, O. S., An, J. W., Bae, N. Y., Lee, B., Park, T., & Mallipeddi, R. (2025). SmartBerry for AI-based growth stage classification and precision nutrition management in strawberry cultivation. Scientific Reports, 15(1), 14019. https://doi.org/10.1038/s41598-025-97168-z
  • Das, D., Kumar, N., Longjam, L.A., Sinha, R., Deb Roy, A., Mondal, H., et al., (2023). Assessing the capability of ChatGPT in answering first- and second-order knowledge questions on microbiology as per competency-based medical education curriculum. Cureus, 15(3), e36034 (Mar).
  • Dave, T., Athaluri, S., & Singh, S. (2023). ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Frontiers in Artificial Intelligence, 6. https://doi.org/10.3389/frai.2023.1169595
  • Deloitte AI Institute. (2023). A new frontier in artificial intelligence: Implications of Generative AI for businesses. Deloitte. https://www.deloitte.com/content/dam/assets-zone3/us/en/docs/services/consulting/2024/us-ai-institute-generative-artificial-intelligence.pdf
  • Dis, E., Bollen, J., Zuidema, W., Rooij, R., & Bockting, C. (2023). Chatgpt: five priorities for research. Nature, 614(7947), 224-226. https://doi.org/10.1038/d41586-023-00288-7
  • Duarte, F. (2025). Number of ChatGPT users (July 2025). Exploding Topics. https://explodingtopics.com/blog/chatgpt-users
  • Emekli, E., Özel, B., & Emekli, E., (2025). Psikiyatri eğitiminde klinik akıl yürütme becerilerinin geliştirilmesi: chatgpt ile otomatik senaryo ve soru üretimi. 10. Psikiyatri Zirvesi ve 17. Anksiyete Kongresi, Antalya, Turkey.
  • En, L., Tha, N., & Thuan, T. (2024). Exploring the application of ChatGPT in learning English of students – a study on perception of sophomores at Nguyen Tat Thanh University. Journal of Science and Technology, 7(3), 41-47. https://doi.org/10.55401/fg15by35
  • Erdem, E. (2024a). Yapay zeka uygulamalarinin sosyal bilim alaninda yapilan çalişmalarda uygulanabilirliği: Chatgpt, Bing ve Youchat örneği. İletişim Bilimi Araştırmaları Dergisi, 4(3), 218-234.
  • Erdem, M. B. (2024b). Lojistik Firma Web Sitelerinin ChatGPT İle Analiz Edilmesi. Erciyes Akademi, 38(4), 1085-1099. https://doi.org/10.48070/erciyesakademi.1596551
  • Fabijan, A., Polis, B., Fabijan, R., Zakrzewski, K., Nowosławska, E., & Zawadzka-Fabijan, A. (2023). Artificial ıntelligence in scoliosis classification: an ınvestigation of language-based models. Journal of Personalized Medicine, 13(12), 1695. https://doi.org/10.3390/jpm13121695
  • Gelmiş, M., Ayten, A., Özsoy, Ç., Bulut, B., & Köse, M. G. (2025). Comparing ChatGPT and Google Gemini in urology: Which AI model provides superior patient education on penile prosthesis?. Androloji Bülteni, 27(1), 8-12.
  • Göktaş, L. S. (2023). ChatGPT uzaktan eğitim sınavlarında başarılı olabilir mi? Turizm alanında doğruluk ve doğrulama üzerine bir araştırma. Journal of Tourism and Gastronomy Studies, 11(2), 892-905. https://doi.org/10.21325/jotags.2023.1224
  • Gupta, R., Hamid, A., Jhaveri, M., Patel, N., & Suthar, P. (2024). Comparative evaluation of AI models such as ChatGPT 3.5, ChatGPT 4.0, and Google Gemini in neuroradiology diagnostics. Cureus.16(8): e67766 https://doi.org/10.7759/cureus.67766
  • Güney, A., & Özcan, E. (2025). Yapay zekanın muhasebede kullanımı üzerine bir araştırma. Journal of Accounting Institute, (72), 1-13. https://doi.org/10.26650/MED.1492095
  • Hacıbey, İ. & Halis, A. (2025). Assessment of artificial intelligence performance in answering questions on onabotulinum toxin and sacral neuromodulation. Investigative and Clinical Urology, 66(3), 188-193. https://doi.org/10.4111/icu.20250040
  • Harada, T., Sato, S. & Nishiura, M. (2025). Using Generative AI to Improve Library Book Classification Accuracy: The Role of Increased Training Data. Oliver, G., Frings-Hessami, V., Du, J.T., Tezuka, T. (eds) Sustainability and Empowerment in the Context of Digital Libraries. ICADL 2024. Lecture Notes in Computer Science, vol 15494. Springer, Singapore. https://doi.org/10.1007/978-981-96-0868-3_20
  • Huang, S., Grady, P. & GPT-3. (2022, September 19). Generative AI: A creative new world. Sequoia. https://www.sequoiacap.com/article/generative-ai-a-creative-new-world/
  • Hussain, A., & Ahmed, A. (2023). Auto-classification of harmonized tariff codes using ChatGPT. 4th International Conference on Distributed Sensing and Intelligent Systems (ICDSIS 2023), Dubai, UAE.
  • İbiş, S. (2025). Yapay zekâ teknolojilerinin gastronomi turizminde kullanımı: ChatGPT örneği. Güncel Turizm Araştırmaları Dergisi, 9(1), 109-131. https://doi.org/10.32572/guntad.1518594
  • Jalil, S., Rafi, S., LaToza, T. D., Moran, K., & Lam, W. (2023). ChatGPT and software testing education: Promises & perils. IEEE International Conference on Software Testing, Verification, and Validation Workshops (ICSTW), Dublin, Ireland.
  • Kaftan, A., Hussain, M., & Naser, F. (2024). Response accuracy of ChatGPT 3.5 CoPilot and Gemini in interpreting biochemical laboratory data: A pilot study. Scientific Reports, 14(1), https://doi.org/10.1038/s41598-024-58964-1
  • Kaiser, K., Hughes, A., Yang, A., Turk, A., Mohanty, S., Gonzalez, A., … & Ellis, R. (2024). Accuracy and consistency of publicly available large language models as clinical decision support tools for the management of colon cancer. Journal of Surgical Oncology, 130(5), 1104–1110. https://doi.org/10.1002/jso.27821
  • Kaplan, A., & Haenlein, M. (2019). Siri, Siri, in my hand: Who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business Horizons, 62(1), 15–25.
  • Koubaa, A. (2023). GPT-4 vs. GPT-3.5: A concise showdown. Prerprints. https://doi.org/10.20944/preprints202303.0422.v1
  • Kutlucan, E., & Seferoğlu, S. S. (2024). Eğitimde yapay zekâ kullanımı: ChatGPT’nin KEFE ve PEST analizi. Türk Eğitim Bilimleri Dergisi, 22(2), 1059-1083. https://doi.org/10.37217/tebd.1368821
  • Küçük, D., & Fazlı, C. (2024) Hukuki metinlerin otomatik işlenmesinde yapay zekâ teknolojilerinin kullanımı. Bilişim Hukuku Dergisi, 6(1):1-23. https://doi.org/10.55009/bilisimhukukudergisi.1450588.
  • Küçüker, M. (2023). Muhasebede yapay zekâ uygulamaları: ChatGPT’nin muhasebe sınavı. Firat University Journal of Social Sciences, 33(2), 875-888. https://doi.org/10.18069/firatsbed.1289885
  • Lawton, G. (2023, July 20). What is generative AI? Everything you. Techtarget. https://www.techtarget.com/searchenterpriseai/definition/generative-AI
  • Lee, K., Kessler, D., Caglič, I., Kuo, Y., Shaida, N., & Barrett, T. (2024). Assessing the performance of ChatGPT and Bard/Gemini against radiologists for prostate imaging-reporting and data system classification based on prostate multiparametric MRI text reports. British Journal of Radiology, 98(1167), 368–374. https://doi.org/10.1093/bjr/tqae236
  • Li, M., & Guenier, A. (2024). ChatGPT and health communication. International Journal of E-Health and Medical Communications, 15(1), 1–26. https://doi.org/10.4018/ijehmc.349980
  • Liu, C., Wei, M., Qin, Y., Zhang, M., Jiang, H., Xu, J., ... & Zhou, J. (2024). Harnessing large language models for structured reporting in breast ultrasound: a comparative study of OpenAI (GPT-4.0) and Microsoft Bing (GPT-4). Ultrasound in Medicine & Biology, 50(11), 1697-1703.
  • Liu, Y., He, S., Chen, H., Qing, W., Su, H., & Deng, G. (2023). Investigation on response strategies for the impact of ChatGPT technology application on college Chinese language education. International Journal of Educational Innovation and Science, 4(1), 156–164. https://doi.org/10.38007/ijeis.2023.040114
  • Makridakis, S. (2017). Forecasting the impact of artificial intelligence (AI). Foresight: The International Journal of Applied Forecasting, 47, 7-13.
  • McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (1955). A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955. AI Magazine, 27(4), 12.
  • Mich, L., & Garigliano, R. (2023). ChatGPT for e-tourism: A technological perspective. Information Technology & Tourism, 25, 1-12. https://doi.org/10.1007/s40558-023-00248-x
  • Moor, J. (2006). The Dartmouth College artificial intelligence conference: The next fifty years. AI Magazine, 27(4), 87–87.
  • Murindanyi, S., Wycliff Mugalu, B., Nakatumba-Nabende, J., & Marvin, G. (2023). Interpretable machine learning for predicting customer churn in retail banking. 7th International Conference on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India. https://doi.org/10. 1109/ICOEI56765.2023.10125859
  • Okur, H., & Ekşi, A. (2024). Yapay zekâ (AI) teknolojilerinin islam eşya hukuku bilgisi üzerine bir değerlendirme: Chatgpt ve Google Gemini karşılaştırması. Dinbilimleri Akademik Araştırma Dergisi. 24(3), 29-54. https://doi.org/10.33415/daad.1582059.
  • Osama, M., & Afridi, S. (2023). ChatGPT: a new era in research writing assistance. Journal of the Pakistan Medical Association, 73(9), 1929-1930. https://doi.org/10.47391/jpma.9183
  • Pavlik, V. J. (2023). Collaborating with ChatGPT: considering the implications of generative artificial intelligence for journalism and media education. Journalism & Mass Communication Educator, 78(1), 84-93.
  • Precedence Research. (2025, May 22). Generative aı market. Generative aı market size to hit USD 1005.07 bn by 2034. Precedence Research. https://www.precedenceresearch.com/generative-ai-market
  • Rane, N., Choudhary, S., & Rane, J. (2024). Gemini or ChatGPT? Capability, Performance, and Selection of Cutting-edge Generative Artificial Intelligence (AI) in Business Management. Studies in Economics and Business Relations, 5(1), 40–50. https://doi.org/10.48185/sebr.v5i1.1051
  • Rossettini, G., Rodeghiero, L., Corradi, F., Cook, C., Pillastrini, P., Turolla, A., … & Palese, A. (2024). Comparative accuracy of ChatGPT-4, Microsoft CoPilot, and Google Gemini in the Italian entrance test for healthcare sciences degrees: a cross-sectional study. BMC Medical Education, 24(1), 1-9 https://doi.org/10.1186/s12909-024-05630-9
  • Sabir, A., Ali, H. A., & Aljabery, M. A. (2024). ChatGPT tweets sentiment analysis using machine learning and data classification. Informatica, 48(7), 103–112. https://doi.org/10.31449/inf.v48i7.5535
  • Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare, 11(6), 887. https://doi.org/10.3390/healthcare11060887
  • Salman, I. M., Ameer, O. Z., Khanfar, M. A., & Hsieh, Y-H. (2025). Artificial intelligence in healthcare education: evaluating the accuracy of ChatGPT, CoPilot, and Google Gemini in cardiovascular pharmacology. Front Med (Lausanne). 12:1495378. https://doi: 10.3389/fmed.2025.1495378
  • Shaheen, A., Afflitto, G. G., & Swaminathan, S. S. (2025). ChatGPT-Assisted classification of postoperative bleeding following microinvasive glaucoma surgery using electronic health record data. Ophthalmology Science, 5(1), 100602.
  • Shihab, R., Sultana, N., & Samad, A. (2023). Revisiting the use of ChatGPT in business and educational fields: possibilities and challenges. BULLET: Jurnal Multidisiplin Ilmu, 2(3), 534–545. https://journal.mediapublikasi.id/index.php/bullet/article/view/2761
  • Smales, L. A. (2023). Classification of RBA monetary policy announcements using ChatGPT. Finance Research Letters, 58, 104514. https://doi.org/10.1016/j.frl.2023.104514
  • Sudirjo, F., Diantoro, K., Al-Gasawneh, J. A., Khootimah Azzaakiyyah, H., & Almaududi Ausat, A. M. (2023). Application of ChatGPT in ımproving customer sentiment analysis for businesses. Jurnal Teknologi Dan Sistem Informasi Bisnis, 5(3), 283-288. https://doi.org/10.47233/jteksis.v5i3.871
  • Şensoy, E., & Çıtırık, M. (2025). Glokom konusunda yapay zeka sohbet botlarının performans analizi: ChatGPT-3.5, Gemini ve CoPilot’un İngilizce ve Türkçe sorulardaki başarı farklılıkları. MN Oftalmoloji, 32(1), 17-21.
  • Thormnundsson, B. (2023, June 20). ChatGPT-statistics & facts. Statista. https://www.statista.com/topics/10446/chatgpt/#topicOverview
  • Uyar, T. (2024). Chat gpt’nin serbest mantıksal safsata tespitinde kullanımı. Yeni Medya Elektronik Dergisi, 8(1), 144-179.
  • Ventura, M., & Filho, A. (2023). ChatGPT: limitations, challenges, and potential applications. Brazilian Journal of Science, 3(1), 65–68. https://doi.org/10.14295/bjs.v3i1.427
  • Waisberg, E., Ong, J., Masalkhi, M., Zaman, N., Sarker, P., Lee, A. G., & Tavakkoli, A. (2024). Google's AI chatbot "Bard": a side-by-side comparison with ChatGPT and its utilization in ophthalmology. Eye (London, England), 38(4), 642–645. https://doi.org/10.1038/s41433-023-02760-0
  • Wiles, J. (2023, January 26). Beyond Chatgpt: the future of generative AI for enterprises. Gartner. https://www.gartner.com/en/articles/beyond-chatgpt-the-future-of-generative-ai-for-enterprises
  • Wong, M., Guo, S., Hang, C., Ho, S., & Tan, C. (2023). Natural language generation and understanding big code for AI-assisted programming: a review. Entropy, 25(6), 888. https://doi.org/10.3390/e25060888
  • Wulcan, J. M., Jacques, K. L., Lee, M. A., Kovacs, S. L., Dausend, N., Prince, L. E., ... & Keller, S. M. (2025). Classification performance and reproducibility of GPT-4 Omni for information extraction from veterinary electronic health records. Frontiers in Veterinary Science, 11, 1490030.
  • Guo, Y., & Wan, Z. (2024). Performance Evaluation of Multimodal Large Language Models (LLaVA and GPT-4-based ChatGPT) in Medical Image Classification Tasks. 12th International Conference on Healthcare Informatics (ICHI), Orlando, FL, USA. Doi: 10.1109/ICHI61247.2024.00080.
  • Dökme Yağar, S. (2023). ChatGPT’nin sağlık alanındaki potansiyel kullanımına ilişkin çıkarımlar. Business & Management Studies: An International Journal, 11(3), 1226–1240. https://doi.org/10.15295/bmij.v11i3.2264.
  • Yalçın-Çelik, A., & Çoban, Ö. K. (2023). Kimya sorularının cevaplanmasında yapay zekâ tabanlı sohbet robotlarının performansının incelenmesi. Journal of Turkish Educational Sciences, 21(3), 1540-1561.
  • Yılmaz, M., & Çil, G. (2024). Yapay zekâ, erektil disfonksiyon ile ilgili sorulara nasıl yanıtlar veriyor: ChatGPT vs Google AI. Androloji Bülteni 26(2), 116−122. https://doi.org/10.24898/tandro.2024.12989
  • Zhang, B., Liang, P., Zhou, X., Ahmad, A., & Waseem, M. (2023). Demystifying practices, challenges, and expected features of using GitHub CoPilot. International Journal of Software Engineering and Knowledge Engineering, 33(11n12), 1653–1672. https://doi.org/10.1142/s0218194023410048
  • Zhu, J. J., Jiang, J., Yang, M., & Ren, Z. J. (2023). ChatGPT and environmental research. Environmental Science & Technology, 57. 1-4. https://doi.org/10.1021/acs.est.3c01818
There are 85 citations in total.

Details

Primary Language English
Subjects Business Administration
Journal Section Research Articles
Authors

Hakan Aşan 0000-0001-9550-3345

Early Pub Date October 27, 2025
Publication Date October 29, 2025
Submission Date August 6, 2025
Acceptance Date October 6, 2025
Published in Issue Year 2025 Volume: 5 Issue: 2

Cite

APA Aşan, H. (2025). Using Generative Artificial Intelligence as a Decision Support Tool in Purchasing Processes: Comparison of ChatGPT, CoPilot, and Gemini Tools. Uluslararası Sosyal Siyasal Ve Mali Araştırmalar Dergisi, 5(2), 101-112. https://doi.org/10.70101/ussmad.1758962