LSTM Network based Sentiment Analysis for Customer Reviews

Abstract

Continuously increasing data bring new problems and problems usually reveal new research areas. One of the new areas is Sentiment Analysis. This field has some difficulties. The fact that people have complex sentiments is the main cause of the difficulty, but this has not prevented the progress of the studies in this field. Sentiment analysis is generally used to obtain information about persons by collecting their texts or expressions. Sentiment analysis can sometimes bring serious benefits. In this study, with singular tag-plural class approach, a binary classification was performed. An LSTM network and several machine learning models were tested. The dataset collected in Turkish, and Stanford Large Movie Reviews datasets were used in this study. Due to the noise in the dataset, the Zemberek NLP Library for Turkic Languages and Regular Expression techniques were used to normalize and clean texts, later, the data were transformed into vector sequences. The preprocessing process made 2% increase to the model performance on the Turkish Customer Reviews dataset. The model was established using an LSTM network. Our model showed better performance than Machine Learning techniques and achieved an accuracy of 90.59% on the Turkish dataset and an accuracy of 89.02% on the IMDB dataset.

Keywords

• Deep Learning
• machine learning
• sentiment analysis
• sequence embedding

References

1. [1] Pang B., Lee L. and Vaithyanathan S., “Thumbs up? Sentiment Classification Using Machine Learning Techniques”, Proceedings of EMNLP, 10: 79-86, (2002).
2. [2] Yıldırım S., Salman Y. B. and Ayvaz S., “Türkçe Duygu Kütüphanesi Geliştirme: Sosyal Medya Verileriyle Duygu Analizi Çalışması”, Avrupa Bilim ve Teknoloji Dergisi, 16: 51-60, (2019).
3. [3] Ayvaz S. and Shiha, M. O., “A Scalable Streaming Big Data Architecture for Real-Time Sentiment Analysis”, ICCBDC'18, 47–51, (2018).
4. [4] Brownlee J., “What is Deep Learning?”, Retrieved From: https://machinelearningmastery.com/what-is-deep-learning/, (2019).
5. [5] Amidi A. and Amidi S., “Recurrent Neural Networks cheatsheet”, Retrieved From: https://stanford.edu/~shervine/l/tr/teaching/cs-230/cheatsheet-recurrent-neural-networks, (2019).
6. [6] “Duygu Analizi için Türkçe Veri Seti”, From: https://www.kaggle.com/burhanbilenn/turkish-customer-reviews-for-binary-classification, (2020).
7. [7] Nalçakan Y., Bayramoğlu Ş. and Tuna S., “Sosyal Medya Verileri Üzerinde Yapay Öğrenme ile Duygu Analizi Çalışması”, (2015).
8. [8] Wilson T., Wiebe J. and Hoffmann P., “Recognizing contextual polarity in phrase-level sentiment analysis”, HLT/EMNLP, 05: 347-354, (2005).

9. [9] Dos Santos C. and Gatti de Bayser M., “Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts”, Proceedings of COLING, 25th International Conference on Computational Linguistics: Technical, 69-78, (2014).
10. [10] Ruder S., Ghaffari P. and Breslin J., “A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis”, EMNLP, 999-1005, (2016).
11. [11] Dey L., Chakraborty S., Biswas A., Bose B. and Tiwari S., “Sentiment Analysis of Review Datasets Using Naïve Bayes and K-NN Classifier”, International Journal of Information Engineering and Electronic Business, 8(4): 54-62, (2016).
12. [12] Baid P., Gupta A. and Chaplot N., “Sentiment Analysis of Movie Reviews using Machine Learning Techniques”, International Journal of Computer Applications, 179(7): 45-49, (2017).
13. [13] Singla Z., Randhawa S. and Jain S., “Sentiment analysis of customer product reviews using machine learning”, International Conference on Intelligent Computing and Control (I2C2), 1-5, (2017).
14. [14] Dehkharghani R., Saygin Y., Yanikoglu, B. and Oflazer, Kemal., “SentiTurkNet: a Turkish polarity lexicon for sentiment analysis”, Language Resources and Evaluation, 50: 667-685, (2015).
15. [15] Sağlam F., Genç B. and Sever H., “Extending a sentiment lexicon with synonym–antonym datasets: SWNetTR++”, Turkish Journal of Electrical Engineering and Computer Sciences, 27: 1806-1820, (2019).
16. [16] Kamisli Ozturk Z., Erzurum Cicek Z. and Ergul Aydin Z., “Sentiment Analysis: an Application to Anadolu University”, Acta Physica Polonica A, 132: 753-755, (2017).
17. [17] Wang Y., Huang M., Zhu X. and Zhao L., “Attention-based LSTM for Aspect-level Sentiment Classification”, Proceedings of the Conference on Empirical Methods in Natural Language Processing, 606-615, (2016).
18. [18] Arras L., Montavon G., Müller K., and Samek W., “Explaining Recurrent Neural Network Predictions in Sentiment Analysis”, Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, 159-168, (2017).
19. [19] Pant D., Neupane P., Poudel A., Pokhrel A. and Lama B., “Recurrent Neural Network Based Bitcoin Price Prediction by Twitter Sentiment Analysis”, IEEE 3rd International Conference on Computing, Communication and Security (ICCCS), 128-132, (2018).
20. [20] Oğuzlar A., “Veri Ön İşleme”, Erciyes Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 21: 67-76, (2003).
21. [21] Akın A. A. and Akın M. D., “Zemberek, an open source nlp framework for Turkic languages”, Structure, 10:1-5, (2007).
22. [22] Go A., Bhayani R. and Huang L., “Twitter sentiment classification using distant supervision”, Processing, 1-6, (2009).
23. [23] Dwarampudi M. and Reddy N. V., “Effects of padding on LSTMs and CNNs”, arXiv, (2019).
24. [24] Tripathy A., Agrawal A. and Rath S., “Classification of Sentimental Reviews Using Machine Learning Techniques”, Procedia Computer Science, 57:821-829, (2015).
25. [25] Maas A., Daly R., Pham P., Huang D., Ng Andrew and Potts C., “Learning Word Vectors for Sentiment Analysis”, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 1:142-150, (2011).
26. [26] CountVectorizer, Scikit-learn, Retrieved From: https://www.scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html
27. [27] Text Tokenizer, Keras, Retrieved From: https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer
28. [28] Embedding Layer, Keras, Retrieved From: https://keras.io/api/layers/core_layers/embedding/