Modern AI Models for Text Analysis: A Comparison of Chatgpt and Rag

Year 2025, Volume: 1 Issue: 1, 1 - 13, 30.01.2025

Aslan Nurzhanov Altynbek Sharipbay

Abstract

This study presents a comparative analysis of two text-processing models: ChatGPT and Retrieval Augmented Generation (RAG).

ChatGPT, built on the Generative Pre-trained Transformer (GPT) architecture, excels at generating coherent and contextually appropriate texts, making it widely applicable in fields such as education, healthcare, and business. However, it has a significant limitation—it relies solely on pre-trained data, lacking the ability to access real-time information, which can affect the relevance of its responses in dynamic contexts.

In contrast, RAG integrates text generation with external data retrieval, offering a substantial advantage in terms of real-time data relevance. This feature enhances both the accuracy and completeness of the generated responses, especially for tasks that require up-to-date information. The study evaluates both models based on several key performance indicators, including accuracy, completeness, processing time, and scalability.

The conclusion highlights the strengths and weaknesses of each model and suggests potential improve ments for their future application across various domains. By offering a deeper understanding of the capabilities and limitations of these technologies, this research contributes to their optimal use and further development.

Keywords

Artificial intelligence (AI), machine learning, natural language processing (NLP), ChatGPT, RAG

References

• Yuntao Wang, Yanghe Pan, Miao Yan, Zhou Su1, and Tom H. Luan. 2023. A SurveY on ChatGPT: Al-Generated Contents, Challenges, and Solutions. arXiv:2305.18339. Retrieved from https://arxiv.org/abs/2305.18339 google scholar
• SiYun Zhao, Yuqing Yang, Zilong Wang, ZhiYuan He, Luna K. Qiu, Lili Qiu. 2024. Retrieval Augmented Generation (RAG) and BeYond: A Comprehensive SurveY on How to Make Your LLMs use External Data More WiselY. arXiv:2409.14924. Retrieved from https:// arxiv.org/abs/2409.14924 google scholar
• Walid Harir. 2024. Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing. arXiv:2304.02017. Retrieved from https://arxiv.org/abs/2304.02017 google scholar
• BiYang Guo, Xin Zhang, ZiYuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, Yupeng Wu. 2023. How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. arXiv:2301.07597. Retrieved from https://arxiv.org/abs/2301. 07597 google scholar
• ShaYekh Bin lslam, Md Asib Rahman, K S M Tozammel Hossain, Enamul Hoque, Shafiq JotY, Md Rizwan Parvez. 2024. Open-RAG: Enhanced Retrieval-Augmented Reasoning with Open-Source Large Language Models. arXiv:2410.01782. Retrieved from https:// arxiv.org/abs/2410.01782 google scholar
• Chengcheng Yu, Jinzhe Yan, Na Cai. 2024. ChatGPT in higher education: factors influencing ChatGPT user satisfaction and continued use intention. 2024. Frontiers in Education, 9, Article 2 (MaY 2024), 11 pages. https://doi.org/10.3389/feduc.2024.1354929 google scholar
• Tirth Dave, Sai Anirudh Athaluri, SatYam Singh. 2023. ChatGPT in medicine: an overview of its applications, advantages, limita-tions, future prospects, and ethical considerations. Frontiers in Artificial lntelligence, 6, Article 64 (MaY 2023), 5 pages. https:// doi.org/10.3389/frai.2023.1169595 google scholar
• Karen D. Wang, Eric Burkholder, Carl Wieman, Shima Salehi, Nick Haber. 2024. Examining the potential and pitfalls of ChatGPT in science and engineering problem-solving. Frontiers in Education, 8, Article 5 (JanuarY 2024), 11 pages. https://doi.org/10.3389/ feduc.2023.1330486 google scholar
• Sasuke Fujimoto, Kazuhiro Takemoto. 2023. Revisiting the political biases of ChatGPT. Frontiers in Artificial lntelligence, 6, Article 174 (October 2023), 6 pages. https://doi.org/10.3389/frai.2023.1232003 google scholar
• Anissa M. Bettayeb, Manar Abu Talib, Al Zahraa Sobhe Altayasinah, Fatima Dakalbab. 2024. Exploring the impact of ChatGPT: conversational Al in education. Frontiers in Education, 9, Article 1 (JulY 2024), 16 pages. https://doi.org/10.3389/feduc.2024.1379796 google scholar
• Maria Grazia Maggio, Gennaro Tartarisco, Davide Cardile, Mirjam Bonanno, Roberta Bruschetta, Loris Pignolo, Giovanni Pioggia, Rocco Salvatore Calabro, Antonio Cerasa. 2024. Exploring ChatGPT’s potential in the clinical stream of neurorehabilitation. Frontiers in Artificial lntelligence, 7, Article 1 (JanuarY 2024), 15 pages. https://doi.org/10.3389/frai.2024.1407905 google scholar
• Rex Bringula. 2024. ChatGPT in a programming course: benefits and limitations. Frontiers in Education, 9, Article 73 (FebruarY 2024), 6 pages. https://doi.org/10.3389/feduc.2024.1248705 google scholar
• ZeYan Liu, Zijun Yao, Fengjun Li, and Bo Luo. 2024. Check Me lf You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT. arXiv:2306.05524. Retrieved from https://arxiv.org/abs/2306.05524 google scholar
• Yunfan Gao, Yun Xiong, XinYu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A SurveY. arXiv:2312.10997v5. Retrieved from https://arxiv.org/abs/2312.10997 v5 google scholar
• Guangzhi Xiong, Qiao Jin, Xiao Wang, Minjia Zhang, ZhiYong Lu, Aidong Zhang. 2024. lmproving Retrieval-Augmented Generation in Medicine with lterative Follow-up Questions. arXiv:2408.00727. Retrieved from https://arxiv.org/abs/2408.00727 google scholar
• ZiYan Jiang, Xueguang Ma, Wenhu Chen. 2024. LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs. arXiv:2406.15319. Retrieved from https://arxiv.org/abs/2406.15319 google scholar
• Xun Xian, Ganghua Wang, Xuan Bi, JaYanth Srinivasa, Ashish Kundu, Charles Fleming, MingYi Hong, Jie Ding. 2024. On the Vulnera-bilitY of ApplYing Retrieval-Augmented Generation within Knowledge-lntensive Application Domains. arXiv:2409.17275. Retrieved from https://arxiv.org/abs/2409.17275 google scholar
• Zhuowan Li, Cheng Li, MingYang Zhang, Qiaozhu Mei, Michael BenderskY. 2024. Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive StudY and HYbrid Approach. arXiv:2407.16833. Retrieved from https://arxiv.org/abs/2407.16833 google scholar
• Ye Yuan, Chengwu Liu, JingYang Yuan, Gongbo Sun, Siqi Li, Ming Zhang. 2024. A HYbrid RAG SYstem with Comprehensive Enhancement on Complex Reasoning. arXiv:2408.05141. Retrieved from https://arxiv.org/abs/2408.05141 google scholar
• Fei Wang, Xingchen Wan, Ruoxi Sun, Jiefeng Chen, Sercan Ö. Arık. 2024. Astute RAG: Overcoming lmperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models. arXiv:2410.07176. Retrieved from https://arxiv.org/abs/2410.07176 google scholar
• SiYun Zhao, Yuqing Yang, Zilong Wang, ZhiYuan He, Luna K. Qiu, Lili Qiu. 2024. Retrieval Augmented Generation (RAG) and BeYond: A Comprehensive SurveY on How to Make Your LLMs use External Data More WiselY. arXiv:2409.14924. Retrieved from https:// arxiv.org/abs/2409.14924 google scholar
• Hossein Bahak, Farzaneh Taheri, Zahra Zojaji, Arefeh Kazemi. 2023. Evaluating ChatGPT as a Question Answering SYstem: A Comprehensive AnalYsis and Comparison with Existing Models. arXiv:2312.07592. Retrieved from https://arxiv.org/abs/2312.07592 google scholar
• Hao Yu, Aoran Gan, Kai Zhang, Shiwei Tong, Qi Liu, Zhaofeng Liu. 2024. Evaluation of Retrieval-Augmented Generation: A SurveY. arXiv:2405.07437. Retrieved from https://arxiv.org/abs/2405.07437 google scholar
• Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal ChoudharY, Rongze Daniel Gui, Ziran Will Jiang, ZiYu Jiang, Lingkun Kong, Brian Moran, Jiaqi Wang, Yifan Ethan Xu, An Yan, ChenYu Yang, Eting Yuan, Hanwen Zha, Nan Tang, Lei Chen, Nicolas Scheffer, Yue Liu, Nirav Shah, Rakesh Wanga, Anuj Kumar, Wen-tau Yih, Xin Luna Dong. 2024. CRAG - Comprehensive RAG Benchmark. arXiv:2406.04744. Retrieved from https://arxiv.org/abs/2406.04744 google scholar
• Kunal Sawarkar, Abhilasha Mangal, Shivam Raj Solanki. 2024. Blended RAG: lmproving RAG (Retriever-Augmented Generation) AccuracY with Semantic Search and HYbrid QuerY-Based Retrievers. arXiv:2404.07220v1. Retrieved from https://arxiv.org/abs/ 2404.07220v1 google scholar
• Yizheng Huang, JimmY Huang. 2024. Exploring ChatGPT for Next-generation lnformation Retrieval: Opportunities and Challenges. arXiv:2402.11203. Retrieved from https://arxiv.org/abs/2402.11203 google scholar
• Yihang Zheng, Bo Li, Zhenghao Lin, Yi Luo, Xuanhe Zhou, Chen Lin, Jinsong Su, Guoliang Li, Shifu Li. 2024. Revolutionizing Database Q&A with Large Language Models: Comprehensive Benchmark and Evaluation. arXiv:2409.04475. Retrieved from https://arxiv. org/abs/2409.04475 google scholar
• Yuntao Wang, Yanghe Pan, Miao Yan, Zhou Su, Tom H. Luan. 2023. A SurveY on ChatGPT: Al-Generated Contents, Challenges, and Solutions. arXiv:2305.18339. Retrieved from https://arxiv.org/abs/2305.18339 google scholar
• Weiwei Sun, LingYong Yan, XinYu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, Zhaochun Ren. 2023. ls ChatGPT Good at Search? lnvestigating Large Language Models as Re-Ranking Agents. arXiv:2304.09542. Retrieved from https://arxiv.org/ abs/2304.09542 google scholar
• Yu Bai, Yukai Miao, Li Chen, Dan Li, YanYu Ren, Hongtao Xie, Ce Yang, Xuhui Cai. 2024. Pistis-RAG: A Scalable Cascading Framework Towards TrustworthY Retrieval-Augmented Generation. arXiv:2407.00072. Retrieved from https://arxiv.org/abs/2407.00072 google scholar
• Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang. 2024. Searching for Best Practices in Retrieval-Augmented Generation. arXiv:2407.01219. Retrieved from https://arxiv.org/abs/2407.01219 google scholar
• Weizhi Fei, XueYan Niu, PingYi Zhou, Lu Hou, Bo Bai, Lei Deng, Wei Han. 2023. Extending Context Window of Large Language Models via Semantic Compression. arXiv:2312.09571. Retrieved from https://arxiv.org/abs/2312.09571 google scholar
• Yuetong Zhao, HongYu Cao, XianYu Zhao, Zhijian Ou. 2024. An Empirical StudY of Retrieval Augmented Generation with Chain-of-Thought. arXiv:2407.15569. Retrieved from https://arxiv.org/abs/2407.15569 google scholar
• Yizheng Huang, JimmY Huang. 2024. A SurveY on Retrieval-Augmented Text Generation for Large Language Models. arXiv: 2404.10981v1. Retrieved from https://ar5iv.labs.arxiv.org/html/2404.10981v1 google scholar
• HarrY Guinness. 2024. What is RAG (retrieval augmented generation)? (August 2024). Retrieved October 15, 2024 from https:// zapier.com/blog/retrieval-augmented-generation/. google scholar
• Frank Joublin, Antonello Ceravola, Joerg Deigmoeller, Michael Gienger, Mathias Franzius, Julian Eggert. 2023. A Glimpse in ChatGPT Capabilities and its impact for Al research. arXiv:2305.06087. Retrieved from https://arxiv.org/abs/2305.06087 google scholar
• Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman GoYal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim RocktâscheL, Sebastian RiedeL, Douwe KieLa. 2021. Retrieval-Augmented Generation for Knowledge-lntensive NLP Tasks. arXiv:2005.11401. Retrieved from https://arxiv.org/abs/2005.11401 google scholar
• Weiwei Sun, LingYong Yan, XinYu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, Zhaochun Ren. 2023. ls ChatGPT Good at Search? lnvestigating Large Language Models as Re-Ranking Agents. arXiv:2304.09542. Retrieved from https://arxiv.org/ abs/2304.09542 google scholar
• Christoph Leiter, Ran Zhang, Yanran Chen, Jonas Belouadi, Daniil Larionov, Vivian Fresen, Steffen Eger. 2023. ChatGPT: A Meta-AnalYsis after 2.5 Months. arXiv:2302.13795. Retrieved from https://arxiv.org/abs/2302.13795 google scholar
• RuiYang Qin, ZheYu Yan, Dewen Zeng, Zhenge Jia, Dancheng Liu, Jianbo Liu, Zhi Zheng, NingYuan Cao, Kai Ni, Jinjun Xiong, YiYu Shi. 2024. Robust lmplementation of Retrieval-Augmented Generation on Edge-based Computing-in-MemorY Architectures. arXiv:2405.04700. Retrieved from https://arxiv.org/abs/2405.04700 google scholar