Analysis of Turkish Sentiment Expressions About Touristic Sites Using Machine Learning

Öz

Analyzing data by inferring from unstructured data about customers is one of the main purposes of the tourism and many other industries as well. However, performing unstructured data analysis using traditional methods is quite inconvenient and costly. This can be overcome by using sentiment analysis, an area of application of text mining. Since there is no proven methodology for sentiment analysis, researchers often perform their studies by trial and error. Many studies on sentiment analysis have focused on comparing the preprocessing or the performance of various machine learning algorithms. Both for these reasons and since research on sentiment analysis with Turkish content is limited, this study aimed to determine the effects of labeling, stemming, and negation on the success of sentiment analysis using Turkish touristic site analysis. From the data set prepared for this study, 12 different variations were created according to labeling, number of classes, stemming, and negation. These data sets were classified using the algorithms Naive Bayes (NB), Multinominal Naive Bayes (MNB), k-Nearest Neighbor, and Support Vector Machines (SVM), often used in sentiment analyses, and the findings were compared.

Anahtar Kelimeler

• Text mining
• Sentiment analysis
• machine learning
• unstructured data analysisc
• naïve bayes
• classification
• multinominal naïve bayes
• support vector machines

Turistik Mekanlar Hakkındaki Türkçe Duygu İfadelerinin Makine Öğrenmesi Yöntemleri ile İncelenmesi

Öz

Müşteriler ile ilgili yapılandırılmamış verilerden çıkarımlar yaparak bu verileri analiz etmek birçok sektör için olduğu gibi turizm sektörü için de temel amaçlardandır. Yapılandırılmamış veri analizinin geleneksel yöntemlerle gerçekleştirilmesi oldukça zahmetli ve maliyetli olmaktadır. Metin analizi uygulama alanlarından biri olan duygu analizi kullanılarak bu sorunun üstesinden gelinebilmektedir. Duygu analizi çalışmalarında henüz kanıtlanmış bir metodoloji bulunmadığı için araştırmacılar genellikle deneme yanılma yoluyla çalışmalarını yürütmektedirler. Duygu analizi alanında yapılan birçok çalışma duygu analizi ön işlemlerinin ya da farklı makine öğrenimi algoritmalarının performanslarının karşılaştırılması üzerinedir. Hem bu nedenlerden dolayı hem de Türkçe içeriklerle gerçekleştirilmiş duygu analizi çalışmalarının kısıtlı olmasından dolayısıyla bu çalışmada Türkçe turistik mekân incelemeleri kullanılarak duygu analizi ön işlemlerinden etiketleme, köklerine ayırma ve olumsuzlaştırma işlemlerinin duygu analizinin başarısına olan etkileri tespit edilmeye çalışılmıştır. Bu nedenle bu çalışma için hazırlanan veri setinden etiketlenme şekline, sınıf sayısına, köklerine ayırma ve olumsuzlaştırma durumlarına göre 12 farklı varyasyon oluşturulmuştur. Oluşturulan bu veri setleri duygu analizi çalışmalarında sıklıkla kullanılan Naive Bayes (NB), Multinominal Naive Bayes (MNB), k-Nearest Neighbor ve Support Vector Machines (SVM) algoritmalarıyla sınıflandırılarak elde edilen sonuçlar karşılaştırılmıştır.

Anahtar Kelimeler

• Metin madenciliği
• duygu analizi
• makine öğrenmesi
• yapılandırılmamış veri analizi
• sınıflandırma
• naive bayes
• multinominal naive bayes
• destek vektör makineleri

Kaynakça

1. Akın, A. A., & Akın, M. D. (2018). Zemberek-NLP. Retrieved from https://github.com/ahmetaa/ zemberek-nlp
2. Altunkaynak, B., 2017. Veri Madenciliği Yöntemleri ve R Uygulamaları [Data Mining Methods and R Applications]. Seçkin Yayıncılık, Ankara. ISBN: 9789750253478, pp:256.
3. Aydoğan, E., & Akcayol, M. A., 2016. A comprehensive survey for sentiment analysis tasks using machine learning techniques. Proceedings of the Proceedings of International Symposium on Innovations in Intelligent Systems and Applications, August 2-5, IEEE Xplore, Romania, pp: 1-7. DOI:10.1109/INISTA.2016.7571856.
4. Baccianella, S., Esuli, A., & Sebastiani, F., 2010. Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. Proceedings of the Proceedings of the Seventh International Conference on Language Resources and Evaluation, May 17-23, Malta, pp: 2200-2204. Retrieved from http://www.lrec-conf.org/proceedings/lrec2010/pdf/769_Paper.pdf
5. Bag-of-Words model. 2007. Wikipedia. https://en.wikipedia.org/wiki/Bag-of-words_model (Accesed on July 20, 2020)
6. Bayes, T., 1763. An essay towards solving a problem in the doctrine of chances. By the late Rev. Mr. Bayes, F. R. S. communicated by Mr. Price, in a letter to John Canton, A. M. F. R. S. Royal Society, 53: 370-418. DOI:https://doi.org/10.1098/rstl.1763.0053
7. Beal, V., n.d. Unstructured data. https://www.webopedia.com/TERM/U/unstructured_data.html (Accesed on November 29, 2019)
8. Bilgin, M., & Şentürk, İ. F., 2017. Sentiment analysis on Twitter data with semi-supervised Doc2Vec. Proceedings of the International Conference on Computer Science and Engineering, October 5-8, IEEE Xplore, Turkey, pp: 661-666. DOI:10.1109/UBMK.2017.8093492.

9. Blumberg, R., & Atre, S., 2003. The problem with unstructured data. Dm Review. Retrieved from http://soquelgroup.com/wp-content/uploads/2010/01/dmreview_0203_problem.pdf
10. Can, Ü., & Alataş, B., 2017. Duygu analizi ve fikir madenciliği algoritmalarının incelenmesi [Review of sentiment analysis and opinion mining algorithms]. International Journal of Pure and Applied Sciences, 3 (1): 75-111. Retrieved from https://dergipark.org.tr/tr/pub/ijpas/issue/29969/304149
11. Chang, Y.-C., Ku, C.-H., & Chen, C.-H., 2019. Social media analytics: Extracting and visualizing Hilton hotel ratings and reviews from TripAdvisor. International Journal of Information Management, 48: 263-279. DOI:10.1016j.ijinfomgt.2017.11.001
12. Çoban, Ö., 2016. Metin sınıflandırma teknikleri ile türkçe twitter duygu analizi [Turkish twitter sentiment analysis using text classification techniques]. Master's Thesis, Atatürk University, Erzurum, Turkey. Retrieved from http://earsiv.atauni.edu.tr/xmlui/handle/123456789/4640
13. Çoban, Ö., Özyer, B., & Özyer, G. T., 2015. Sentiment analysis for Turkish Twitter feeds. Proceedings of the 23nd Signal Processing and Communications Applications Conference, May 16-19, IEEE Xplore, Turkey, pp: 2388-2391. DOI:10.1109/SIU.2015.7130362.
14. Cortes, C., & Vapnik, V., 1995. Support-vector networks. Machine Learning, 20 (3): 273-297. DOI:10.1007/BF00994018
15. Data Scraping. (2021). Retrieved from https://en.wikipedia.org/wiki/Data_scraping
16. Datashake. (2021). Retrieved from https://www.datashake.com/
17. Esen, M. F., & Türkay, B., 2017. Turizm endüstrilerinde büyük veri kullanımı [Big data applications in tourism]. Journal of Tourism and Gastronomy Studies, 4 (4): 92-115. DOI:10.21325/jotags.2017.140
18. Esuli, A., & Sebastiani, F., 2006. Sentiwordnet: A publicly available lexical resource for opinion mining. Proceedings of the Fifth International Conference on Language Resources and Evaluation, May 22-28, European Language Resources Association, Italy, pp: 417-422. Retrieved from http://www.lrec-conf.org/proceedings/lrec2006/pdf/384_pdf.pdf
19. F1 score. 2006. Wikipedia. https://en.wikipedia.org/wiki/F1_score (Accesed on July 8, 2020)
20. Gezici, G., & Yanıkoğlu, B., 2018. Sentiment analysis in Turkish. In: Turkish Natural Language Processing, K. Oflazer & M. Saraçlar (Eds.), Springer International Publishing, Cham, Switzerland, pp. 255-271. ISBN: 978-3-319-90165-7
21. Google. (2019). Google Translation API. Retrieved from https://cloud.google.com/translate/
22. Güran, A., Uysal, M., & Doğrusöz, Ö., 2014. Destek vektör makineleri parametre optimizasyonunun duygu analizi üzerindeki etkisi [Effects of support vector machines parameter optimization on sentiment anaylsis]. Dokuz Eylul University Faculty of Engineering Journal of Science and Engineering, 16 (48): 86-93. Retrieved from https://dergipark.org.tr/tr/pub/deumffmd/issue/40797/492168
23. Harris, Z. S., 1954. Distributional structure. Word, 10 (2-3): 146-162. DOI:10.1080/00437956.1954.11659520
24. Hearst, M., 2003. What is text mining. https://people.ischool.berkeley.edu/~hearst/text-mining.html (Accesed on November 29, 2019)
25. Kaya, M., Fidan, G., & Toroslu, I. H., 2012. Sentiment analysis of turkish political news. Proceedings of the International Joint Conferences on Web Intelligence and Intelligent Agent Technology, December 4-7, IEEE, China, pp: 174-180. DOI:10.1109/WI-IAT.2012.115.
26. Kaynar, O., Görmez, Y., Yıldız, M., & Albayrak, A., 2016. Makine Öğrenmesi Yöntemleri ile Duygu Analizi [Sentiment Analysis With Machine Learning Techniques]. Proceedings of the International Artificial Intelligence and Data Processing Symposium, September 17-18, Inonu University, Turkey, pp: 234-241. Retrieved from http://idapold.inonu.edu.tr/panel/uploads/1/1/09.2016/IDAP16_SemposiumProceeding.pdf
27. Khan, M., Ding, Q., & Perrizo, W., 2002. K-nearest neighbor classification on spatial data streams using p-trees. Proceedings of the 6th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, May 6-8, Springer-Verlag, Berlin, pp: 517-518. DOI:10.1007/3-540-47887-6_51.
28. Kızılkaya, Y. M., 2018. Duygu Analizi ve Sosyal Medya Alanında Uygulama [Sentiment Analysis and Social Media Application]. Doctoral Disertation, Bursa Uludağ University, Bursa, Turkey. Retrieved from https://acikerisim.uludag.edu.tr/bitstream/11452/1058/1/516866.pdf
29. Köse, İ., 2018. Veri Madenciliği Teori Uygulama ve Felsefesi [Data Mining Theory Practice and Philosophy]. Papatya Yayıncılık Eğitim, İstanbul. ISBN: 978-605-9594-34-9.
30. Ku, L.-W., Liang, Y.-T., & Chen, H.-H., 2006. Opinion extraction, summarization and tracking in news and blog corpora. Proceedings of the AAAI Spring Symposium, March 27-29, AAAI, CA, USA, pp: 100-107. Retrieved from https://www.aaai.org/Papers/Symposia/Spring/2006/SS-06-03/SS06-03-020.pdf
31. Kulcu, S., & Dogdu, E., 2016. A scalable approach for sentiment analysis of Turkish tweets and linking tweets to news. Proceedings of the International Conference on Semantic Computing, February 4-6, IEEE, Laguna Hills, CA, USA, pp: 471-476. DOI:10.1109/ICSC.2016.66.
32. Liu, B., 2012. Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers, Williston. ISBN: 9781608458844, pp:180.
33. Liu, K.-L., Li, W.-J., & Guo, M., 2012. Emoticon smoothed language models for twitter sentiment analysis. Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, July 22-26, AAAI Press, Ontario, Canada, pp: 1678-1684. Retrieved from https://dl.acm.org/doi/10.5555/2900929.2900966
34. Lohr, S., 2012. The age of big data. New York Times. https://www.nytimes.com/2012/02/12/sunday-review/big-datas-impact-in-the-world.html (Accesed on November 29, 2019)
35. Maron, M. E., 1961. Automatic Indexing: An Experimental Inquiry. J. ACM, 8 (3): 404-417. DOI:10.1145/321075.321084
36. Meral, M., & Diri, B., 2014. Sentiment analysis on Twitter. Proceedings of the 22nd Signal Processing and Communications Applications Conference, April 23-25, IEEE Xplore Press, Trabzon, Turkey, pp: 690-693. DOI:10.1109/SIU.2014.6830323.
37. Miner, G., Elder IV, J., Fast, A., Hill, T., Nisbet, R., & Delen, D., 2012. Practical text mining and statistical analysis for non-structured text data applications. Academic Press, Waltham, MA. ISBN: 978-0-12-386979-1, pp:1053.
38. Özyurt, B., & Akçayol, M. A., 2018. Fikir madenciliği ve duygu analizi, yaklaşımlar, yöntemler üzerine bir araştırma [A survey on sentiment analysis and opinion mining, methods and approaches]. Selcuk University Journal of Engineering, Science and Technology, 6 (4): 668-693. Retrieved from http://sujest.selcuk.edu.tr/sumbtd/article/download/584/628
39. Pang, B., Lee, L., & Vaithyanathan, S., 2002. Thumbs up? Sentiment classification using machine learning techniques. Proceedings of the Conference on Empirical Methods in Natural Language Processing, July 6-7, Association for Computational Linguistics, Philadelphia, USA, pp: 79-86. DOI:10.3115/1118693.1118704.
40. Parlar, T., & Özel, S. A., 2016. A new feature selection method for sentiment analysis of Turkish reviews. Proceedings of the International Symposium on Innovations in Intelligent Systems and Applications, August 2-5, IEEE Xplore Press, Sinaia, Romania, pp: 1-6. DOI:10.1109/INISTA.2016.7571833.
41. Provost, F., & Fawcett, T., 2013. Data science and its relationship to big data and data-driven decision making. Big data, 1 (1): 51-59. DOI:10.1089/big.2013.1508
42. Rennie, J. D. M., Shih, L., Teevan, J., & Karger, D. R., 2003. Tackling the poor assumptions of naive bayes text classifiers. Proceedings of the International Conference on Machine Learning, Agust 21-24, AAAI Press, Washington, DC, USA, pp: 616-623. Retrieved from https://dl.acm.org/doi/10.5555/3041838.3041916
43. Rijsbergen, C. J. V., 1979. Information Retrieval. Butterworth-Heinemann. ISBN: 0408709294.
44. Saad, M. K., 2010. The impact of text preprocessing and term weighting on arabic text classification. Master’s Thesis, The Islamic University, Gaza,Palestine. Retrieved from http://hdl.handle.net/20.500.12358/18770
45. Salur, M. U., Aydın, İ., & Alghrsi, S. A., 2019. SmartSenti: A twitter-based sentiment analysis system for the smart tourism in Turkey. Proceedings of the International Artificial Intelligence and Data Processing Symposium, September 21-22, IEEE Xplore Press, Malatya, Turkey, pp: 1-5. DOI:10.1109/IDAP.2019.8875922.
46. Şeker, S. E., 2016. Duygu analizi [Sentiment analysis]. Management Information Systems Encyclopedia, 3 (3): 21-36. Retrieved from http://ybsansiklopedi.com/wp-content/uploads/2016/09/duygu_analizi.pdf
47. Sevindi, B. İ., 2013. Comparison of supervised and dictionary based sentiment analysis approaches on Turkish text. Master’s Thesis, Gazi University, Ankara, Turkey. Retrieved from https://tez.yok.gov.tr/UlusalTezMerkezi/tezDetay.jsp?id=-hunBqU7X8Xef7VsYSve5g&no=pshUKfJWw6uE0H0aViWTEg
48. Shung, K. P., 2020. Accuracy, Precision, Recall or F1? Medium. https://towardsdatascience.com/accuracy-precision-recall-or-f1-331fb37c5cb9 (Accesed on July 20, 2020)
49. Silahtaroğlu, G., 2013. Veri madenciliği: Kavram ve algoritmaları [Data mining: Concept and algorithms]. Papatya Yayıncılık Eğitim, İstanbul. ISBN: 9756797819, pp:304.
50. Spärck Jones, K., 2004. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 60 (5): 493-502. DOI:10.1108/00220410410560573
51. Taecharungroj, V., & Mathayomchan, B., 2019. Analysing TripAdvisor reviews of tourist attractions in Phuket, Thailand. Tourism Management, 75: 550-568. DOI:https://doi.org/10.1016/j.tourman.2019.06.020
52. Toçoğlu, M. A., 2018. Lexicon-based emotion analysis in Turkish. Doctoral Dissertation, Dokuz Eylül University, İzmir, Turkey. Retrieved from https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=fS4sqEZr79C_n60Rk6MjFRsa3aAwCXsuupy0qzym8RJZBM0BloOaPHJbEezM612h
53. Türkmenoğlu, C., 2015. Türkçe Metinlerde Duygu Analizi [Sentiment Analysis In Turkish Texts]. Master’s Thesis, Istanbul Technical University, İstanbul, Turkey. Retrieved from http://hdl.handle.net/11527/12950
54. Türkmenoglu, C., & Tantug, A. C., 2014. Sentiment analysis in Turkish media. Proceedings of the International Conference on International Conference on Machine Learning, June 21-26, Beijing, China.
55. Velioğlu, R., Yıldız, T., & Yıldırım, S., 2018. Sentiment analysis using learning approaches over emojis for Turkish tweets. Proceedings of the 3rd International Conference on Computer Science and Application Engineering, September 20-23, IEEE Xplore Press, Sanya, China, pp: 303-307. DOI:10.1109/UBMK.2018.8566260.
56. Waykole, R. N., & Thakare, A., 2018. A Review of Feature Extraction Methods for Text Classification. IJAERD, 5 (04): 351-254. DOI:10.21090/IJAERD.89982
57. Witten, I. H., Frank, E., Hall, M. A., & Pal, C. J., 2016. The WEKA Workbench. Online Appendix for "Data Mining: Practical Machine Learning Tools and Techniques". Morgan Kaufmann, Burlington, MA. ISBN: 9780128042915, pp:654.
58. Yıldırım, E., Çetin, F. S., Eryiğit, G., & Temel, T., 2015. The impact of NLP on Turkish sentiment analysis. TBV Journal of Computer Science and Engineering, 7 (1): 41-51. Retrieved from https://dergipark.org.tr/tr/pub/tbbmd/issue/22247/238817
59. Yıldız, O., 2016. Metin madenciliğinde anahtar kelime seçimi bir üniversite örneği [Keyword selection in text mining: A university sample]. Journal of Management Information Systems, 2 (1): 29-50. Retrieved from https://dergipark.org.tr/tr/pub/ybs/issue/31331/341834