UTILIZING NATURAL LANGUAGE PROCESSING, OBSERVATIONAL METRICS FOR PREDICTIVE ANALYSIS OF GENERATIVE ARTIFICIAL INTELLIGENCE MODELS

Abstract

In the era of massive language models, there is a growing need to determine the future of the enduring desire to use ChatGPT, an example of a generative artificial intelligence (GenAI) model. It's doubtful that consumers' present feelings and degree of interest will last over time. This work investigated the predictive analytics of observational metrics on GenAI models using online data and natural language processing techniques to forecast future sentiments and search interest. Time-bound web analytics data and Twitter metrics related to GenAI were collected using Google Trend and the Twitter API on Orange Data Mining Toolkit. Google trend data was forecasted using Autoregressive Integrated Moving Average (ARIMA), whereas sentiment polarities and search interest time series were predicted using Naive Bayes. The experiment's results indicated a limited correlation between tweet sentiment polarity scores and engagement metrics. Five subjects in all were returned by the topic modeling: doubts or skepticism about OpenAI and Microsoft, Microsoft and AI Use, French discussions on ChatGPT, ChatGPT arguments and usage, and making something funny in relation to intelligence and analysis. The findings revealed a predominantly positive sentiment tendency among the 50 anticipated sentiment instances, with 41 (82%), 4 (8%) and 5 (10%) denoting good, neutral, and negative sentiments, respectively. This implies a generally positive outlook. These findings showed how important it is to look at sentiment and interest trends to fully understand the evolution of GenAI models.

Keywords

• Autoregressive
• sentiment
• GenAI
• ChatGPT
• predictive analytics.

Ethical Statement

This study does not contain any studies with human or animal subjects performed by any of the authors.

Thanks

The authors acknowledge the efforts of the reviewers of this paper. We also appreciate their meaningful contribution, valuable suggestions, and comments to this paper which helped us in improving the quality of the manuscript.

References

1. M. Haenlein and A. Kaplan. A brief history of artificial intelligence: On the past, present, and future of artificial intelligence. California Management Review, 61(4):5–14, 2019.
2. A. Tlili, B. Shehata, M. A. Adarkwah, A. Bozkurt, D. T. Hickey, R. Huang, and B. Agyemang. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(15), 2023.
3. K. Ayoub and K. Payne. Strategy in the age of artificial intelligence. Journal of Strategic Studies, 39(5):793–819, 2016.
4. J. Gao, M. Galley, and L. Li. Neural approaches to conversational AI. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1371–1374. ACM, 2018.
5. M. Zaib, Q. Sheng, and W. E. Zhang. A short survey of pre-trained language models for conversational AI: A new age in NLP. In Proceedings of the Australasian Computer Science Week Multiconference, pages 1–4. ACM, 2020.
6. J. Qadir. Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education, 2022. Preprint or unpublished manuscript.
7. T. Lalwani, S. Bhalotia, A. Pal, V. Rathod, and S. Bisen. Implementation of a chatbot system using AI and NLP. International Journal of Innovative Research in Computer Science & Technology (IJIRCST), 6, 2018.
8. A. Vasilateanu and R. Ene. Call-center virtual assistant using natural language processing and speech recognition. Journal of ICT, Design, Engineering and Technological Science, pages 40–46, 2018.

9. D. M. Hussein. A survey on sentiment analysis challenges. Journal of King Saud University-Engineering Sciences, 30(4):330–338, 2018.
10. R. Boorugu and G. Ramesh. A survey on NLP based text summarization for summarizing product reviews. In 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA), pages 352–356. IEEE, 2020.
11. U. Kamath, J. Liu, and J. Whitaker. Deep learning for NLP and speech recognition. Cham, 2019.
12. E. E. Adam. Deep learning based NLP techniques in text to speech synthesis for communication recognition. Journal of Soft Computing Paradigm (JSCP), 2(4):209 215, 2020.
13. H. Shelar, G. Kaur, N. Heda, and P. Agrawal. Named entity recognition approaches and their comparison for custom ner model. Science & Technology Libraries, 39(3):324–337, 2020.
14. R. Alghamdi and K. Alfalqi. A survey of topic modeling in text mining. International Journal of Advanced Computer Science and Applications (IJACSA), 6(1), 2015.
15. S. Kumar, M. Yadava, and P. P. Roy. Fusion of EEG response and sentiment analysis of products review to predict customer satisfaction. Information Fusion, 52:41–52, 2019.
16. U. Bukar, M. S. Sayeed, S. F. Razak, S. Yogarayan, and O. A. Amodu. Text analysis of ChatGPT as a tool for academic progress or exploitation. SSRN, 2023. SSRN 4381394.
17. X. Fang and J. Zhan. Sentiment analysis using product review data. Journal of Big Data, 2(1):1–14, 2018.
18. A. M. Alkalbani, A. M. Ghamry, F. K. Hussain, and O. K. Hussain. Sentiment analysis and classification for software as a service reviews. In 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA), pages 1–10. IEEE, 2016.
19. M. Usama, B. Ahmad, E. Song, M. S. Hossain, M. Alrashoud, and G. Muhammad. Attention-based sentiment analysis using convolutional and recurrent neural network. Future Generation Computer Systems, 113:571–578, 2020.
20. A. M. Rahat, A. Kahir, and A. K. Masum. Comparison of naive bayes and SVM algorithm based on sentiment analysis using review dataset. In 2019 8th International Conference System Modeling and Advancement in Research Trends (SMART), pages 266–270. IEEE, 2019