Combining Sentiment Analysis Models Using Stacking Ensemble Learning Techniques on BIST30 Stocks

Year 2024, Volume: 1 Issue: 2, 91 - 97, 27.10.2024

Mahmut Sami Sivri

https://doi.org/10.5281/zenodo.13996517

Abstract

In recent years, sentiment analysis has become a crucial task in the field of natural language processing (NLP). Despite significant advancements in individual sentiment analysis models, combining multiple models can further enhance performance and robustness. This paper proposes an ensemble model using stacking to integrate the outputs of different sentiment analysis models applied to news articles related to BIST30 stocks traded on Borsa Istanbul. The base models include Long Short-Term Memory (LSTM), Bidirectional Encoder Representations from Transformers (BERT), Naive Bayes, and Support Vector Machines (SVM). The meta-learner is a logistic regression model that aggregates the predictions of the base models. This ensemble approach demonstrates improved accuracy and generalization capabilities over single-model approaches in analyzing the sentiment of financial news.

Keywords

Sentiment Analysis, Ensemble Learning, Stacking, Machine Learning, Stock Markets

BIST30 Hisseleri Üzerinde Duygu Analizi Modellerini Yığınlama Topluluk Öğrenmesi Teknikleriyle Birleştirme

Abstract

Son yıllarda, duygu analizi, doğal dil işleme (NLP) alanında önemli bir görev haline gelmiştir. Bireysel duygu analizi modellerinde önemli ilerlemeler kaydedilmiş olmasına rağmen, birden fazla modelin birleştirilmesi performansı ve dayanıklılığı daha da artırabilir. Bu makale, Borsa İstanbul'da işlem gören BIST30 hisseleriyle ilgili haber makalelerine uygulanan farklı duygu analizi modellerinin çıktılarının entegrasyonu için stacking kullanan bir ensemble modeli önermektedir. Temel modeller arasında Uzun Kısa Süreli Bellek (LSTM), İki Yönlü Kodlayıcı Temsillerinden Transformatorler (BERT), Naive Bayes ve Destek Vektör Makineleri (SVM) bulunmaktadır. Meta-öğrenici, temel modellerin tahminlerini birleştiren lojistik regresyon modelidir. Bu ensemble yaklaşımı, finansal haberlerin duygu analizinde tek model yaklaşımlarına göre geliştirilmiş doğruluk ve genelleme yetenekleri sergilemektedir.

Keywords

Duygu Analizi, Topluluk Öğrenmesi, Yığınlama, Makine Öğrenmesi, Hisse Senedi Piyasaları

References

• [1] Li, X., “News sentiment and the stock market”, Journal of Financial Markets, Vol.13, Issue 1, Pages 1-30, 2010.
• [2] Tetlock, P. C., “Giving content to investor sentiment: The role of media in the stock market”, The Journal of Finance, Vol. 62, Issue 3, Pages 1139-1168, 2007.
• [3] Liu, Y., Zhang, J., “Enhancing financial sentiment analysis with ensemble deep learning”, Expert Systems with Applications, Vol. 205, Article No. 117455, 2022.
• [4] Bhatia, A., Kumar, S., “A hybrid approach for sentiment analysis of financial news articles using ensemble learning”, Journal of Finance and Data Science, Vol. 7, No. 1, Pages 45-58, 2021.
• [5] Breiman, L., “Bagging predictors”, Machine Learning, Vol. 24, Issue 2, Pages 123-140, 1996.
• [6] Friedman, J. H., “Greedy function approximation: A gradient boosting machine”, The Annals of Statistics, Vol. 29, Issue 5, Pages 1189-1232, 2001.
• [7] Wolpert, D. H., “Stacked generalization”, Neural Networks, Vol. 5, Issue 2, Pages 241-259, 1992.
• [8] Zhang, X., Xu, Y., “Sentiment analysis in finance: A comprehensive review”, International Journal of Financial Studies, Vol. 8, No. 4, Article No. 66, 2020.
• [9] Deng, Y., Wei, Z., “The impact of news sentiment on stock price movement: An empirical study”, Journal of Economic Behavior & Organization, Vol. 193, Pages 356-370, 2022.
• [10] Chen, Y., Wang, Y., “Multi-modal sentiment analysis for financial news: An ensemble learning approach”, IEEE Transactions on Knowledge and Data Engineering, Vol. 35, Issue 1, Pages 234-246, 2023.
• [11] Pang, B., Lee, L., “Opinion mining and sentiment analysis”, Foundations and Trends® in Information Retrieval, Vol. 2, Issue 1-2, Pages 1-135, 2008.
• [12] Hochreiter, S., Schmidhuber, J., “Long short-term memory”, Neural Computation, Vol. 9, Issue 8, Pages 1735-1780, 1997.
• [13] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K., “BERT: Pre-training of deep bidirectional transformers for language understanding”, Proceedings of NAACL-HLT, Pages 4171-4186, 2019.
• [14] Bollen, J., Mao, H., Zeng, X., “Twitter mood predicts the stock market”, Journal of Computational Science, Vol. 2, Issued 1, Pages 1-8, 2011.
• [15] Ghosh, S., Sarkar, S., Sarkar, R., “Combining convolutional and recurrent neural networks for text classification”. arXiv preprint arXiv:1611.02324, 2017.
• [16] Wang, W., Yin, F., Schütze, H., “Sentiment analysis in code-mixed languages with BERT and traditional machine learning techniques”, Proceedings of the 28th International Conference on Computational Linguistics, 2020.
• [17] Kumar, A., Gupta, V., “Sentiment analysis of financial news articles using hybrid ensemble methods”, Journal of Financial Technology, Vol. 3, Issue 2, Pages 111-129, 2023.
• [18] Zhao, Y., Liu, T., “Exploring the role of news sentiment in stock market behavior: Evidence from the BIST30 index”, Finance Research Letters, Vol. 38, Article No. 101493, 2021.
• [19] Akhtar, P., Khan, S., “A review of sentiment analysis techniques and their applications in financial markets”, Finance Research Letters, Vol. 46, Article No. 102160, 2023.
• [20] Begum, S., Gupta, V., “Sentiment Analysis: A Recent Survey with Applications and a Proposed Ensemble Model”, Springer International Publishing, 2022.
• [21] Kaur, A., Kumar, R., “Deep Learning and Ensemble Learning Models for Sentiment Analysis: An Experimental Study”, IEEE Access, 2023.
• [22] Patel, S., Shah, M., “A Hybrid Ensemble Model for Sentiment Analysis in Financial Markets”, Elsevier Journal of Financial Markets, 2022.
• [23] Dipa, N., Begum, S., “An Enhanced Approach for Sentiment Analysis Based on Meta-Ensemble Deep Learning”, Springer Advances in Intelligent Systems, 2022.
• [24] Cortes, C., Vapnik, V., “Support-vector networks”, Machine Learning, Vol. 20, Issue 3, Pages 273-297, 1995.
• [25] Hossain, M. S., Hossain, A., “Sentiment analysis and stock market prediction: A survey of methods and trends”, Journal of Economic Interaction and Coordination, Vol. 17, Issue 1, Pages 45-67, 2022.
• [26] Feng, Y., Qiao, Z., “Ensemble learning for stock market prediction: A case study of financial sentiment analysis”, Journal of Computational Science, Vol. 48, Article No. 101215, 2021.