Yapay Zekâ Destekli Veri Analizi: ChatGPT-4.0 ile Geçerlik ve Güvenirlik Kestirimleri

Year 2025, Volume: 12 Issue: 23, 83 - 99, 30.04.2025

Duygu Koçak

https://doi.org/10.29129/inujgse.1617323

Abstract

Amaç: Bu araştırmada yapay zekâ uygulamasının veri analizi yapmadaki performansı yapı geçerliği ve güvenirlik kanıtları üzerinden incelenmesi amaçlanmıştır. Bu amaç doğrultusunda farklı örneklem büyüklüğü ve madde sayısı koşullarında üretilen verilerde açımlayıcı faktör analizi Cronbach alfa ve madde toplam test korelasyonu hesaplamaları R programı ve ChatGPT ile yapılarak karşılaştırılmıştır.

Yöntem: Araştırma verilerin yapay olarak elde edilerek çeşitli istatistiksel tekniklerin test edilerek karşılaştırmaların yapıldığı simülatif bir araştırmadır. Araştırmada örneklem büyüklüğü 250, 500 ve 1000, madde sayısı 10 ve 20 olarak değişimlenerek 100 tekrar ile simülatif olarak üretilmiştir. Üretilen veri setlerine açımlayıcı faktör analizi yapılarak KMO değeri, açıklanan toplam varyans oranı ve faktör sayısı kestirimleri; iç tutarlılık anlamında güvenirlik için Cronbach alfa kestirimleri ve madde ayırt ediciliği için madde toplam test puanı korelasyon katsayısı hem R hem de ChatGPT ile ayrı ayrı yapılmıştır. Elde edilen bulgular betimsel olarak karşılaştırılmıştır.

Bulgular: Araştırmada ChatGPT ve R programından elde edilen değerlerin birbiri ile uyumlu olduğu bulgusuna erişilmiştir. Buna göre KMO değeri, açıklanan toplam varyans değer, faktör sayısı ve Cronbach alfa değeri benzerdir. Örneklem büyüklüğü arttıkça dolayısıyla veri matrisi büyüdükçe ChatGPT’nin elde ettiği kesitirimler ile R arasında fark oluştuğu görülmüştür.

Sonuçlar ve Öneriler: ChatGPT tarafından kestirilen KMO, açıklanan toplam varyans, faktör sayısı ve Cronbach alfa katsayısı değerleri R ile uyum göstermektedir. Niceliksel değerlere bağlı kararlarda yapay zekâ kullanılabilir ancak faktör isimlendirme gibi araştırmacı yargısı gerektiren durumlarda bütünüyle yapay zekaya güvenmemek gerekmektedir. Bu araştırmada yapay zekâ uygulamasında analizler tek zamanda yapılmıştır, farklı zamanlarda yapıldığında farklı sonuçlar verebileceği göz önünde bulundurulmalıdır. Yapay zekâ uygulamalarından yalnızca ChatGPT üzerinden denemeler yapılmıştır farklı uygulamalarda da benzer sonuçlar verip vermediği araştırılabilir.

Keywords

Yapay zeka , veri analizi , ChatGPT , Psikometrik özellik

AI-Assisted Data Analysis: Validity and Reliability Estimations with ChatGPT-4.0

Abstract

Purpose: This study aims to examine the performance of an artificial intelligence application in data analysis based on evidence of construct validity and reliability. For this purpose, exploratory factor analysis, Cronbach's alpha, and item-total test correlation calculations were performed using both R and ChatGPT on datasets generated under varying sample sizes and item numbers. The results obtained from these two methods were then compared.

Design & Methodology: This is a simulative study in which data were artificially generated and various statistical techniques were tested and compared. The sample sizes were set at 250, 500, and 1000, with item numbers set to 10 and 20, and repeated 100 times for simulation purposes. The generated datasets were subjected to exploratory factor analysis, and estimations for KMO value, total explained variance, and the number of factors were obtained. For internal consistency, Cronbach's alpha was calculated, and for item discrimination, item-total test score correlation coefficients were computed using both R and ChatGPT. The results were compared descriptively.

Findings: The study found that the values obtained from ChatGPT and R were consistent with each other. Specifically, the KMO value, total explained variance, number of factors, and Cronbach's alpha were found to be similar. However, as the sample size increased and thus the data matrix expanded, discrepancies were observed between the estimates obtained by ChatGPT and R.

Implications & Suggestions: The estimations for KMO, total explained variance, number of factors, and Cronbach's alpha values obtained by ChatGPT were in agreement with those obtained using R. Artificial intelligence can be used in decision-making processes that rely on quantitative values, but it is advised not to rely entirely on AI for tasks requiring researcher judgment, such as naming factors. In this study, analyses using the AI application were performed at a single point in time; it should be noted that results may vary if conducted at different times. Additionally, this study only tested ChatGPT; future research could explore whether similar results are obtained with other AI applications.

Keywords

Artificial Intelligence , Data Analysis , ChatGPT , Psychometric Properties

References

• Arafat, S. M. Y., Chowdhury, H. R., Qusar, M. S., ve Hafez, M. A. (2016). Cross-cultural adaptation and psychometric validation of research instruments: A methodological review. Journal of Behavioral Health, 5(3), 129–136. https://doi.org/10.5455/jbh.20160615121755
• Arbağ, S. S., ve Ertekin, E. (2020). Matematik eğitimi lisansüstü tezlerindeki geçerlik ve güvenirlik çalışmalarının incelenmesi. OPUS International Journal of Society Researches, 16(31), 3924-3957.
• Baidoo-Anu, D., ve Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. SSRN. Retrieved from https://ssrn.com/abstract=4337484
• Bandalos, D. L., ve Finney, S. J. (2018). Factor analysis: Exploratory and confirmatory. The Guilford Press.
• Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., ve Skolits, G. J. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18(18), 1–13.
• Braeken, J., ve Van Assen, M. A. (2017). An empirical Kaiser criterion. Psychological Methods, 22(3), 450–466. https://doi.org/10.1037/met0000074
• Comrey, A. L., ve Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Lawrence Erlbaum Associates.
• Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Sage.
• Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
• Çokluk, Ö., Şekercioğlu, G., ve Büyüköztürk, Ş. (2014). Sosyal bilimler için çok değişkenli istatistik: SPSS ve LISREL uygulamaları. Pegem Akademi Yayıncılık.
• Dehouche, N. (2021). Harnessing the power of AI in education: ChatGPT as an intelligent tutoring system. Journal of Artificial Intelligence in Education, 31(3), 456-469.
• DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). Sage Publications.
• Dooley, K. (2002). Simulation research methods. In J. Baum (Ed.), Companion to organizations (pp. 829-848). Blackwell.
• Erkuş, A. (2011). Psikolojide ölçme ve ölçek geliştirme-I: Temel kavramlar ve işlemler. Pegem Akademi.
• Field, A. (2009). Discovering statistics using SPSS. SAGE Publications Ltd.
• George, D., ve Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference (4th ed.). Allyn & Bacon.
• Gillani, N., Eynon, R., Chiabaut, C., ve Finkel, K. (2023). Unpacking the “black box” of AI in education. Educational Technology & Society, 26(1), 99–111.
• Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Saunders.
• Hair, J. F., Black, W. C., Babin, B. J., ve Anderson, R. E. (2014). Exploratory factor analysis. In J. F. Hair, W. C. Black, B. J. Babin, & R. E. Anderson (Eds.), Multivariate data analysis (7th ed., pp. 88–127). Prentice Hall.
• Howard, M. C. (2016). A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51–62. https://doi.org/10.1080/10447318.2015.1087664
• Izquierdo, I., Olea, J., ve Abad, F. J. (2014). Exploratory factor analysis in validation studies: Uses and recommendations. Psicothema, 26(4), 395–400. https://doi.org/10.7334/psicothema2013.349
• Kartal, S. K., ve Dirlik, E. M. (2016). Geçerlik kavramının tarihsel gelişimi ve güvenirlikte en çok tercih edilen yöntem: Cronbach alfa katsayısı. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 16(4), 1865-1879.
• Kasneci, E., Kasneci, G., Stede, M., ve Naumann, F. (2023). ChatGPT’s role in automated text analysis: Strengths, limitations, and future directions. Computational Linguistics, 49(1), 99-117.
• Liang, X., Xiao, T., Liu, H., Zhou, W., Wang, Y., & Li, J. (2023). Transformer-based models and their impact on AI research: A review. Artificial Intelligence Review, 56(1), 57–88. https://doi.org/10.1007/s10462-023-10277-0
• Lloret, S., Ferreres, A., Hernandez, A., ve Tomas, I. (2017). The exploratory factor analysis of items: Guided analysis based on empirical data and software. Anales de Psicología, 33(2), 417–432. https://doi.org/10.6018/analesps.33.2.270211
• Mhlanga, D. (2023). Open AI in education: The responsible and ethical use of ChatGPT towards lifelong learning. SSRN. https://ssrn.com/abstract=4354422 https://doi.org/10.2139/ssrn.4354422
• Nemorin, S., Gillani, N., ve Wood, S. (2023). AI and plagiarism: Challenges and solutions in academic integrity. Journal of Educational Technology, 32(4), 245-258.
• Nunnally, J. C., ve Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
• Pallant, J. (2010). SPSS survival manual: A step by step guide to data analysis using SPSS. Open University Press/McGraw-Hill.
• R Development Core Team. (2011). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Retrieved from https://www.R-project.org
• Revelle, W., ve Revelle, M. W. (2015). Package ‘psych’. The Comprehensive R Archive Network, 337(338), 161–165. Retrieved from https://CRAN.R-project.org/package=psych
• Rudolph, J., & Tan, S. C. (2023). ChatGPT and generative AI in academic settings: Friend or foe? Journal of Educational Technology, 24(3), 215–230. https://doi.org/10.1007/s11423-023-10211-7
• Shrestha, N. (2021). Factor analysis as a tool for survey analysis. American Journal of Applied Mathematics and Statistics, 9(1), 4–11. https://doi.org/10.12691/ajams-9-1-2
• Singh, V., Chen, S. S., Singhania, M., Nanavati, B., ve Gupta, A. (2022). How are reinforcement learning and deep learning algorithms used for big data based decision making in financial industries–A review and research agenda. International Journal of Information Management Data Insights, 2(2), 100094.
• Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99-103.
• Tabachnick, B. G., ve Fidell, L. S. (2019). Using multivariate statistics (7th ed.). Pearson Education.
• Tavakol, M., ve Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53-55.
• Terwiesch, C. (2023). GPT-4: The next generation of generative AI for business and academia. Harvard Business Review, 101(4), 45–50.
• Tlili, A., Huang, R., Gao, W., ve Kinshuk. (2023). AI and education: From theory to practice. Computers and Education: Artificial Intelligence, 4, 100150. https://doi.org/10.1016/j.caeai.2023.100150
• Vaswani, A. (2017). Attention is all you need. Advances in Neural Information Processing Systems.
• Watkins, M. W. (2018). Exploratory factor analysis: A guide to best practices. Journal of Black Psychology, 44(1), 36–65. https://doi.org/10.1177/0095798418771807