TY  - JOUR
T1  - Performance of Generative Artificial Intelligence Models (GPT-4o, Gemini, Copilot) in YKS/TYT Exam: A Comparative Study
TT  - Üretken Yapay Zeka Modellerinin (GPT-4o, Gemini, Copilot) YKS/TYT Sınavındaki Performansı: Karşılaştırmalı Bir Çalışma
AU  - Bulut, Selma
PY  - 2025
DA  - October
Y2  - 2025
DO  - 10.17671/gazibtd.1575755
JF  - Bilişim Teknolojileri Dergisi
PB  - Gazi Üniversitesi
WT  - DergiPark
SN  - 1307-9697
SP  - 283
EP  - 296
VL  - 18
IS  - 4
LA  - en
AB  - This study attempts to evaluate the potential of generative artificial intelligence (GAI) models in the field of education. A simulated exam environment was planned and error analysis was performed based on the responses of GPT (Generative Pre-trained Transformer) family models to questions asked in university entrance exams. In the study, GPT, Gemini and Copilot models were tested with questions in the Higher Education Institutions Exam (YKS)/ Basic Proficiency Exam ( TYT) session in Turkey. The responses produced by these models were analyzed in terms of both accuracy rates and academic depth and consistency. While GPT-4o achieved 50.8% accuracy, Gemini 43.3% and Copilot achieved 42.5% accuracy. The findings provide important clues about the potential of GAI models to be used as a learning aid in education and whether they can be evaluated as a supportive tool in university exam preparation processes. The study sheds light on the possibility of using GAI models in the personalized education field that emerged with the rise of GAI models in the field of education. Further studies in this field may help develop new strategies for the integration of GAI models into the education system.
KW  - generative AI
KW  - chatbots
KW  - chatGPT
KW  - gemini
KW  - copilot
N2  - Bu çalışmada üretken yapay zekâ (GAI) modellerinin eğitim alanındaki potansiyeli değerlendirilmeye çalışılmıştır. Benzetimli bir sınav ortamı planlanmış ve GPT (Generative Pre-trained Transformer) ailesi modellerinin üniversite giriş sınavlarında sorulan sorulara verdiği yanıtlar baz alınarak hata analizi yapılmıştır. Çalışmada GPT, Gemini ve Copilot modelleri Türkiye’de Yükseköğretim Kurumları Sınavı (YKS)/ Temel Yeterlilik Sınavı ( TYT) oturumunda yer alan sorularla test edilmiştir. Bu modellerin ürettiği yanıtlar hem doğruluk oranları hem de akademik derinlik ve tutarlılık açısından analiz edilmiştir. GPT-4o %50.8 doğruluk elde ederken, Gemini %43.3 ve Copilot %42.5 doğruluk elde etmiştir. Bulgular, GAI modellerinin eğitimde bir öğrenme yardımcısı olarak kullanılabilme potansiyeli ve üniversite sınavlarına hazırlık süreçlerinde destekleyici bir araç olarak değerlendirilip değerlendirilemeyeceği konusunda önemli ipuçları sunmaktadır. Çalışma, GAI modellerinin eğitim alanında yükselişiyle ortaya çıkan kişiselleştirilmiş eğitim alanında kullanılabilme ihtimaline ışık tutmaktadır. Bu alanda yapılacak daha fazla çalışma, GAI modellerinin eğitim sistemine entegrasyonu için yeni stratejiler geliştirmeye yardımcı olabilir.
CR  - T., Eloundou, S., Manning, P., Mishkin, & D. Rock, “Gpts are gpts: An early look at the labor market impact potential of large language models”, arXiv preprint arXiv:2303.10130. 2023.
CR  - Y. Liu, H. Wang. “Who on Earth Is Using Generative AI?”, https://elements.visualcapitalist.com/wp-content/uploads/2024/09/1726222967151.pdf, 2024.
CR  - Internet: D. Gewirtz/ZDNET https://www.zdnet.com/article/the-most-popular-ai-tools-of-2024-and-what-that-even-means/. 23.12.2024.
CR  - S. Bulut, “Üretken Yapay Zeka Teknolojisi: GPT-4o”, 2024. International Journal of Advanced Natural Sciences And Engineering Researches (8)4: 380 – 387.
CR  - A. Bozkurt, ChatGPT, “Üretken yapay zeka ve algoritmik paradigma değişikliği”, Alanyazın, 4(1), 63-72. 2023.
CR  - Y., Cao, S., Li, Y., Liu, Z., Yan, Y., Dai, P. S., Yu, & L, “Sun, A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt”, arXiv preprint arXiv:2303.04226. 2023.
CR  - B. D., Lund, & T. Wang, “Chatting about ChatGPT: how may AI and GPT impact academia and libraries?”, Library hi tech news, 40(3), 26-29. 2023.
CR  - D. R., Cotton, P. A., Cotton, & J. R. Shipway, “Chatting and cheating: Ensuring academic integrity in the era of ChatGPT”, Innovations in education and teaching international, 61(2), 228-239. 2024.
CR  - M. Firat, “How ChatGPT can transform autodidactic experiences and open education?”, 2023.
CR  - A. Iskender, “Holy or unholy? Interview with OpenAI’s ChatGPT”, European Journal of Tourism Research, 34, 3414-3414. 2023.
CR  - P., Zhang, & G. Tur, “A systematic review of ChatGPT use in K‐12 education”. European Journal of Education, 59(2), e12599. 2024.
CR  - M., Liu, T., Okuhara, Z., Dai, W., Huang, H., Okada, F., Emi, & T. Kiuchi, “Performance of Advanced Large Language Models (GPT-4o, GPT-4, Gemini 1.5 Pro, Claude 3 Opus) on Japanese Medical Licensing Examination: A Comparative Study”, medRxiv, 2024-07. 2024.
CR  - L. Lian, Comparative Study of GPT-4.0, “ERNIE Bot 4.0, and GPT-4o in the 2023 Chinese Medical Licensing Examination”, 2024.
CR  - J., Savelka, A., Agarwal, C., Bogart, & M. Sakr, From GPT-3 to GPT-4: On the Evolving Efficacy of LLMs to Answer Multiple-choice Questions for Programming Classes in Higher Education, In International Conference on Computer Supported Education (pp. 160-182). Cham: Springer Nature Switzerland, 2023.
CR  - D. M., Katz, M. J., Bommarito, S., Gao, & P. Arredondo, “Gpt-4 passes the bar exam”, Philosophical Transactions of the Royal Society A, 382(2270), 20230254, 2024.
CR  - F., Farhat, B. M., Chaudhry, M., Nadeem, S. S., Sohail, & D. Ø. Madsen, “Evaluating large language models for the National Premedical Exam in India: comparative analysis of GPT-3.5, GPT-4, and Bard”, JMIR Medical Education, 10, e51523, 2024.
CR  - M. L., Elias, J., Burshtein, & V. R. Sharon, “OpenAI's GPT‐4 performs to a high degree on board‐style dermatology questions”, International Journal of Dermatology, 63(1), 73-78, 2024.
CR  - T. Talan, & Y. Kalınkara, “The role of artificial intelligence in higher education: ChatGPT assessment for anatomy course”, Uluslararası Yönetim Bilişim Sistemleri ve Bilgisayar Bilimleri Dergisi, 7(1), 33-40. 2023.
CR  - J. Doughty, Z. Wan  A. Bompelli, J. Qayum, T. Wang, J. Zhang, & M. Sakr, A comparative study of AI-generated (GPT-4) and human-crafted MCQs in programming education, In Proceedings of the 26th Australasian Computing Education Conference, pp. 114-123. 2024.
CR  - I. Azaiz, N. Kiesler, & S. Strickroth, Feedback-Generation for Programming Exercises With GPT-4 In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1, pp. 31-37. 2024.
CR  - Y. Sonoda, R. Kurokawa, Y. Nakamura, J. Kanzawa, M. Kurokawa, Y. Ohizumi,... & O. Abe, “Diagnostic performances of GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro in “Diagnosis Please” cases”, Japanese Journal of Radiology, pp. 1-5, 2024.
CR  - M. Ishida, W. Gonoi, K. Nyunoya, H. Abe, G. Shirota, N. Okimoto,... & O. Abe, “Diagnostic Performance of GPT-4o and Claude 3 Opus in Determining Causes of Death From Medical Histories and Postmortem CT Findings”, Cureus, 16(8), 2024.
CR  - M. F. Firdaus, J. N. Wibawa, & F. F. Rahman, “Utilization of GPT-4 to Improve Education Quality Through Personalized Learning for Generation Z in Indonesia”, IT for Society, 8(1), 2023.
CR  - Madrid-García, A., Rosales-Rosado, Z., Freites-Nuñez, D., Pérez-Sancristóbal, I., Pato-Cour, E., Plasencia-Rodríguez, C., ... & Rodríguez-Rodríguez, L. “Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training”, Scientific Reports, 13(1), 22129, 2023.
CR  - M. G Rizzo, N. Cai, & D. Constantinescu, “The performance of ChatGPT on orthopaedic in-service training exams: A comparative study of the GPT-3.5 turbo and GPT-4 models in orthopaedic education”, Journal of Orthopaedics, 50, 70-75, 2024.
CR  - F. B. Warwas, & N. Heim, “Performance of GPT-4 in Oral and Maxillofacial Surgery Board Exams”, Challenges in Specialized Question, 2024.
CR  - J. Doughty, Z. Wan, A. Bompelli, J. Qayum, T. Wang, J. Zhang, & M. Sakr, A comparative study of AI-generated (GPT-4) and human-crafted MCQs in programming education. In Proceedings of the 26th Australasian Computing Education Conference, pp. 114-123, 2024.
CR  - J. Samardžija, M. Žagar, & N. Drašković, “Work in progress: enhancing exam preparation with Chatgpt among university freshmen students.” In ICERI 2024 Proceedings, pp. 9687-9694, 2024.
CR  - V., Mavrych, P., Ganguly & O. Bolgova, “Using large language models (ChatGPT, Copilot, PaLM, Bard, and Gemini) in Gross Anatomy course: Comparative analysis.”,  Clinical Anatomy, 2024.
CR  - M. Abu-Haifa, B. A. Etawi, H. Alkhatatbeh, & A. Ababneh, “Comparative Analysis of ChatGPT, GPT-4, and Microsoft Copilot Chatbots for GRE Test.”, International Journal of Learning, Teaching and Educational Research, 23(6), 327-347, 2024.
CR  - A. Koç & A. B. Öztiryaki, “Comparison of the accuracy performances of the Gemini Advanced, the GPT-4, the Copilot, and the GPT-3.5 models in medical imaging systems: A Zero-shot prompting analysis.”, Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi, 13(4), 1216-1223, 2024.
CR  - Öter, A., Ersöz, B., Bülbül, H. İ., & Sağıroğlu, Ş. (2024). Using Generative Artificial Intelligence in Exams: A Research on KPSS with ChatGPT. International Journal of Educational Research Review, 9(4), 269-274.   
 
C. Leiter, R. Zhang, Y. Chen, J. Belouadi, D. Larionov, V. Fresen & S. Eger, “Chatgpt: A meta-analysis after 2.5 months.”, Machine Learning with Applications, 16, 100541, 2024.
CR  - D. Milmo,. "ChatGPT reaches 100 million users two months after launch". The Guardian. ISSN 0261-3077, https://www.theguardian.com/technology/2023/feb/02/chatgpt-100-million-users-open-ai-fastest-growing-app, 2024.
CR  - S. Bulut., “Üretken Yapay Zeka: ChatGPT, Bing ve Bard Karşılaştırmalı Bir İnceleme”, International Journal Of Advanced Natural Sciences And Engineering Researches (Ijanser) (7)9, 104 – 109. Doi: 10.59287/ijanser.1517. 2023.
CR  - Internet:	 OpenAI, Hello GPT 4o, 2024. https://OpenAI.com/index/hello-gpt-4o/, 29.08.2024.
CR  - Internet:Gerttalkative. ChatGPT-4o vs GPT-4 vs GPT-3.5: What’s the Difference?https://gettalkative.com/info/gpt-models-compared, 22.12.2024.
CR  - Internet: S. Shubham, ChatGPT Statistics (AUG 2024) – Users Growth Data, 2024, https://www.demandsage.com/chatgpt-statistics/, 26.08.2024.
CR  - Internet:  Community, Update data Chat GPT fed with data until 09.2021, https://community.openai.com/t/update-data-chat-gpt-fed-with-data-until-09-2021/451130 , 26.08.2024.
CR  - Internet: D. V. Meer , Number of ChatGPT Users and Key Stats (September 2024), https://www.namepepper.com/chatgpt-users, 29.08.2024.
CR  - Internet: Community, ChatGPT can now access the live Internet. Can the API? https://community.openai.com/t/chatgpt-can-now-access-the-live-internet-can-the-api/401928, 29.08.2024.
CR  - Internet: M. Diaz, R. Rajkumar, How to use Microsoft Copilot (formerly called Bing Chat), https://www.zdnet.com/article/how-to-use-Copilot/ 2024, 26.08.2024.
CR  - Internet: Microsoft, Microsoft Copilot Prohttps://www.microsoft.com/en-us/store/b/Copilotpro, 29.08.2024.
CR  - Internet: A. Subramanya, Gemini’s big upgrade: Faster responses with 1.5 Flash, expanded access and more. https://blog.google/products/gemini/google-gemini-new-features-july-2024/, 27.08.2024.
CR  - Internet: S. Pichai ve D. Hassabis, Introducing Gemini: our largest and most capable AI model,  https://blog.google/technology/ai/google-gemini-ai/#capabilities, 27.08.2024.
CR  - Internet: ÖSYM, Yükseköğretim Kurumları Sınavı, https://www.osym.gov.tr/TR,29434/2024-yuksekogretim-kurumlari-sinavi-2024-yks-temel-soru-kitapciklari-ve-cevap-anahtarlari-yayimlandi-09062024.html, 29.08.2024.
CR  - M. A. Cohen, “Some new evidence on the seriousness of crime.”, Criminology, 26(2), 343-353, 1988.
CR  - Internet: yks 2024’e ilk bakiş. https://www.linkedin.com/pulse/yks-2024e-ilk-bakiş-tedmem1-0mr5f/ , 29.08.2024.
CR  - Internet: M. Çakalp, Yapay zeka boğaziçini kazandı, https://www.hurriyet.com.tr/egitim/yapay-zeka-bogazicini-kazandi-42476894, 29.08.2024.
CR  - Internet: G. Uyar, GPT-4, Üniversite Sınavına Girdi: Türkiye'nin %99'undan Daha Başarılı Oldu! https://www.webtekno.com/gpt-4-universite-sinavi-turkiye-yuzde-99-gecti-h133365.html, 21.12.204.
CR  - G. Beutel, E. Geerits, & J. T. Kielstein, “Artificial hallucination: GPT on LSD?.” Critical Care, 27(1), 148, 2023.
CR  - T. B. Brown, “Language models are few-shot learners.”, arXiv preprint arXiv:2005.14165, 2020.
CR  - G. P. Reddy, Y. P. Kumar, & K. P. Prakash, Hallucinations in Large Language Models (LLMs)., In 2024 IEEE Open Conference of Electrical, Electronic and Information Sciences (eStream) (pp. 1-6). IEEE, 2024.
CR  - J. Waldo, & S. Boussard, “GPTs and Hallucination: Why do large language models hallucinate?.”, Queue, 22(4), 19-33, 2024.
UR  - https://doi.org/10.17671/gazibtd.1575755
L1  - https://dergipark.org.tr/tr/download/article-file/4324255
ER  -