Metin Tabanlı Transformer Modelleri Kullanılarak Otel Yorumlarından Müşteri Deneyiminin Değerlendirilmesi

Öz

Konaklama sektörünün hızla gelişen ortamında, dijital geri bildirimler aracılığıyla müşteri deneyimini anlamak ve iyileştirmek stratejik bir zorunluluk haline gelmiştir. Bu çalışma, otel yorumlarındaki müşteri duygularını değerlendirmek için gelişmiş transformer tabanlı mimarilerle desteklenen yüksek hassasiyetli bir çerçeve sunmaktadır. Bu alandaki önemli bir zorluk olan sınıf dengesizliği, azınlık sınıfları için anlamsal olarak tutarlı yeniden ifade edilmiş (paraphrased) örnekler üretmek amacıyla ChatGPT'den yararlanan yenilikçi bir veri artırma stratejisi ile ele alınmıştır. Çalışma kapsamında beş güncel modelin (BERT, DistilBERT, RoBERTa, ELECTRA ve GPT-2) kapsamlı bir karşılaştırmalı analizi gerçekleştirilmiştir. Metodoloji, özel bir veri seti olan Bali Otel Yorum Derlemi (BHRC) ile birlikte küresel çapta tanınan dört kıyaslama veri seti (IMDB, SST-1, SST-2 ve MR) kullanılarak doğrulanmıştır. Deneysel sonuçlar, önerilen çerçevenin olağanüstü doğruluk seviyelerine ulaştığını ve ELECTRA modelinin dengelenmiş koşullar altında %99,96'lık bir zirve doğruluk değerine ulaşarak diğer modelleri geride bıraktığını göstermektedir. Bu performans, ELECTRA'nın alana özgü dizi sınıflandırmasında standart maskelenmiş veya üretken yaklaşımlardan daha üstün olduğu kanıtlanan verimli ayrıştırıcı (discriminative) ön eğitim hedefine bağlanmaktadır. Bulgularımız, gerçek zamanlı müşteri deneyimi değerlendirmesi için sağlam ve kaynak açısından verimli bir temel sunarak, otel yöneticilerine modern dijital turizm ekonomisinde hizmet kalitesini artıracak uygulanabilir içgörüler sağlamaktadır.

Anahtar Kelimeler

• Müşteri Deneyimi Değerlendirmesi
• Duygu Analizi
• Transformer Mimarileri
• ELECTRA
• Veri Artırımı

Appraising Customer Experience from Hotel Reviews using Text-Based Transformers

Öz

In the rapidly evolving landscape of the hospitality industry, comprehending and enhancing customer experience through digital feedback has become a strategic imperative. This study introduces a high-precision framework powered by advanced transformer-based architectures to evaluate customer sentiments in hotel reviews. A significant challenge in this domain, class imbalance, was addressed through an innovative data augmentation strategy utilizing ChatGPT to generate semantically consistent paraphrased samples for minority classes. We conducted a comprehensive comparative analysis of five state-of-the-art models: BERT, DistilBERT, RoBERTa, ELECTRA, and GPT-2. The methodology was validated using a specialized dataset, the Bali Hotel Review Corpus (BHRC), alongside four globally recognized benchmarks (IMDb, SST-1, SST-2, and MR). Experimental results demonstrate that the proposed framework achieves exceptional accuracy levels, with the ELECTRA model outperforming others by reaching a peak accuracy of 99.96% under balanced conditions. This performance is attributed to ELECTRA’s efficient discriminative pre-training objective, which proves superior to standard masked or generative approaches for domain-specific sequence classification. Our findings provide a robust and resource-efficient benchmark for real-time customer experience appraisal, offering hotel managers actionable insights to drive service quality in the modern digital tourism economy.

Anahtar Kelimeler

• Customer experience appraisal
• Sentiment Analysis
• Transformer architectures
• Electra
• Data Augmentation

Kaynakça

1. [1] Fu JR, Lu IW, Chen JHF, Farn CK. Investigating consumers' online social shopping intention: an information processing perspective. International Journal of Information Management. 2020;54:102189. Available from: https://doi.org/10.1016/j.ijinfomgt.2020.102189
2. [2] Tang L, Wang X, Kim E. Predicting conversion rates in online hotel bookings with customer reviews. Journal of Theoretical and Applied Electronic Commerce Research. 2022;17(4):1264-1278. Available from: https://doi.org/10.3390/jtaer17040064
3. [3] Güler YC, Huseynov F. The impact of online consumer reviews on online hotel booking intention. Journal of Business Research-Turk. 2021;13(3):2634-2652. Available from: https://doi.org/10.20491/isarder.2021.1282
4. [4] Touni R, Kim WG, Haldorai K, Rady A. Customer engagement and hotel booking intention: the mediating and moderating roles of customer-perceived value and brand reputation. International Journal of Hospitality Management. 2022;104:103246. Available from: https://doi.org/10.1016/j.ijhm.2022.103246
5. [5] Wankhade M, Rao ACS, Kulkarni C. A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review. 2022;55(7):5731-5780. Available from: https://doi.org/10.1007/s10462-022-10144-1
6. [6] Aakash A, Gupta Aggarwal A. Assessment of hotel performance and guest satisfaction through eWOM: big data for better insights. International Journal of Hospitality & Tourism Administration. 2022;23(2):317-346. Available from: https://doi.org/10.1080/15256480.2020.1746218
7. [7] Sann R, Lai PC. Understanding homophily of service failure within the hotel guest cycle: applying NLP-aspect-based sentiment analysis to the hospitality industry. International Journal of Hospitality Management. 2020;91:102678. Available from: https://doi.org/10.1016/j.ijhm.2020.102678
8. [8] Obiedat R, Qaddoura R, Al-Zoubi AM, Al-Qaisi L, Harfoushi O, Alrefai M, Faris H. Sentiment analysis of customers' reviews using a hybrid evolutionary SVM-based approach in an imbalanced data distribution. IEEE Access. 2022;10:22260-22273. Available from: https://doi.org/10.1109/ACCESS.2022.3149482

9. [9] Anis S, Saad S, Aref M. Sentiment analysis of hotel reviews using machine learning techniques. In: Hassanien AE, Slowik A, Snášel V, El-Deeb H, Tolba FM, editors. Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2020. Cham: Springer; 2021. p. 227-234. Available from: https://doi.org/10.1007/978-3-030-58669-0_21
10. [10] Noori B. Classification of customer reviews using machine learning algorithms. Applied Artificial Intelligence. 2021;35(8):567-588. Available from: https://doi.org/10.1080/08839514.2021.1922843
11. [11] Tiffani IE. Optimization of naïve Bayes classifier by implemented unigram, bigram, trigram for sentiment analysis of hotel review. Journal of Soft Computing Exploration. 2020;1(1):1-7. Available from: https://doi.org/10.52465/joscex.v1i1.4
12. [12] Sayed AA, Elgeldawi E, Zaki AM, Galal AR. Sentiment analysis for Arabic reviews using machine learning classification algorithms. 2020 International Conference on Innovative Trends in Communication and Computer Engineering; 2020 Feb 8-9; Aswan, Egypt. p. 56-63. Available from: https://doi.org/10.1109/ITCE48509.2020.9047822
13. [13] Gunathilaka S, De Silva N. Aspect-based sentiment analysis on mobile application reviews. 22nd International Conference on Advances in ICT for Emerging Regions (ICTer); 2022 Nov 2-3; Colombo, Sri Lanka. p. 183-188. Available from: https://doi.org/10.1109/ICTer58063.2022.10024070
14. [14] Zhou G, Liao C. Dynamic measurement and evaluation of hotel customer satisfaction through sentiment analysis on online reviews. Journal of Organizational and End User Computing. 2021;33(6):1-27. Available from: https://doi: 10.4018/JOEUC.20211101.oa8
15. [15] Ishaq A, Umer M, Mushtaq MF, Medaglia C, Siddiqui HUR, Mehmood A, Choi GS. Extensive hotel reviews classification using long short-term memory. Journal of Ambient Intelligence and Humanized Computing. 2021;12(10):9375-9385. Available from: https://doi.org/10.1007/s12652-020-02654-z
16. [16] Hassan J, Shoaib U. Multi-class review rating classification using deep recurrent neural network. Neural Processing Letters. 2020;51(2):1031-1048. Available from: https://doi.org/10.1007/s11063-019-10125-6
17. [17] Puh K, Bagić Babac M. Predicting sentiment and rating of tourist reviews using machine learning. Journal of Hospitality and Tourism Insights. 2023;6(3):1188-1204. Available from: https://doi.org/10.1108/JHTI-02-2022-0078
18. [18] Khedkar, S., Shinde, S. Deep learning and ensemble approach for praise or complaint classification. Procedia Computer Science 2020, 167, 449-458.
19. [19] Ray B, Garain A, Sarkar R. An ensemble-based hotel recommender system using sentiment analysis and aspect categorization of hotel reviews. Applied Soft Computing. 2021;98:106935. Available from: https://doi.org/10.1016/j.asoc.2020.106935
20. [20] Kotei E, Thirunavukarasu R. A systematic review of transformer-based pre-trained language models through self-supervised learning. Information. 2023;14(3):187. Available from: https://doi.org/10.3390/info14030187
21. [21] Acheampong FA, Nunoo-Mensah H, Chen W. Transformer models for text-based emotion detection: a review of BERT-based approaches. Artificial Intelligence Review. 2021;54(8):5789-5829. Available from: https://doi.org/10.1007/s10462-021-09958-2
22. [22] Greco CM, Simeri A, Tagarelli A, Zumpano E. Transformer-based language models for mental health issues: a survey. Pattern Recognition Letters. 2023;167:204-211. Available from: https://doi.org/10.1016/j.patrec.2023.02.016
23. [23] Vakili YZ, Fallah A, Sajedi H. Distilled BERT model in natural language processing. 14th International Conference on Computer and Knowledge Engineering (ICCKE); 2024 Nov 19-20; Mashhad, Iran. p. 243-250. Available from: https://doi.org/10.1109/ICCKE65377.2024.10874673
24. [24] Areshey AM, Mathkour H. Exploring transformer models for sentiment classification: a comparison of BERT, RoBERTa, ALBERT, DistilBERT, and XLNet. Expert Systems. 2024;41(11):e13701. Available from: https://doi.org/10.1111/exsy.13701
25. [25] Dong C, Liu L, Cheng H, Shang J, Gao J, Liu X. Fast-ELECTRA for efficient pre-training. 12th International Conference on Learning Representations (ICLR); 2024 May 7-11; Vienna, Austria.
26. [26] Yenduri G, Ramalingam M, Selvi GC, Supriya Y, Srivastava G, Maddikunta PKR, Raj DG, Jhaveri RH, Prabadevi B, Wang W, Vasilakos AV, Gadekallu TR. GPT (Generative Pre-Trained Transformer): a comprehensive review on enabling technologies, potential applications, emerging challenges, and future directions. IEEE Access. 2024;12:54608-54649. Available from: https://doi.org/10.1109/ACCESS.2024.3389497
27. [27] Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf. Published 2019. Accessed 2026 Jul 17
28. [28] Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences. 52nd Annual Meeting of the Association for Computational Linguistics; 2014 Jun 22-27; Baltimore, Maryland, USA. p. 655-665. Available from: https://doi.org/10.3115/v1/P14-1062
29. [29] Kim Y. Convolutional neural networks for sentence classification. 2014 Conference on Empirical Methods in Natural Language Processing; 2014 Oct 25-29; Doha, Qatar. p. 1746-1751. Available from: https://doi.org/10.3115/v1/D14-1181
30. [30] Tai KS, Socher R, Manning CD. Improved semantic representations from tree-structured long short-term memory networks. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing; 2015 Jul 26-31; Beijing, China. p. 1556-1566. Available from: https://doi.org/10.3115/v1/P15-1150
31. [31] Wang X, Jiang W, Luo Z. Combination of convolutional and recurrent neural network for sentiment analysis of short texts. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics; 2016 Dec 11-16; Osaka, Japan. p. 2428-2437. Available from: https://aclanthology.org/C16-1229/
32. [32] Zhang Y, Wallace B. A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification. 8th International Joint Conference on Natural Language Processing; 2017 Nov 27-Dec 1; Taipei, Taiwan. p. 253-263. Available from: https://aclanthology.org/I17-1026/
33. [33] Hassan A, Mahmood A. Deep learning approach for sentiment analysis of short texts. 3rd International Conference on Control, Automation and Robotics (ICCAR); 2017 Apr 24-26; Nagoya, Japan. p. 705-710. Available from: https://doi.org/10.1109/ICCAR.2017.7942788
34. [34] Chen T, Xu R, He Y, Wang X. Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN. Expert Systems with Applications. 2017;72:221-230. Available from: https://doi.org/10.1016/j.eswa.2016.10.065
35. [35] Hassan A, Mahmood A. Convolutional recurrent deep learning model for sentence classification. IEEE Access. 2018;6:13949-13957. Available from: https://doi.org/10.1109/ACCESS.2018.2814818
36. [36] Liu Y, Ji L, Huang R, Ming T, Gao C, Zhang J. An attention-gated convolutional neural network for sentence classification. Intelligent Data Analysis. 2019;23(5):1091-1107. Available from: https://doi.org/10.3233/IDA-184311
37. [37] Wen S, Wei H, Yang Y, Guo Z, Zeng Z, Huang T, Chen Y. Memristive LSTM network for sentiment analysis. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2021;51(3):1794-1804. Available from: https://doi.org/10.1109/TSMC.2019.2906098
38. [38] Usama M, Ahmad B, Song E, Hossain MS, Alrashoud M, Muhammad G. Attention-based sentiment analysis using convolutional and recurrent neural network. Future Generation Computer Systems. 2020;113:571-578. Available from: https://doi.org/10.1016/j.future.2020.07.022
39. [39] Tammina S, Annareddy S. Sentiment analysis on customer reviews using convolutional neural network. International Conference on Computer Communication and Informatics (ICCCI); 2020 Jan 22-24; Coimbatore, India. p. 1-6. Available from: https://doi.org/10.1109/ICCCI48352.2020.9104086
40. [40] Dang NC, Moreno-García MN, De la Prieta F. Sentiment analysis based on deep learning: a comparative study. Electronics. 2020;9(3):483. Available from: https://doi.org/10.3390/electronics9030483
41. [41] Cortiz D. Exploring transformer models for emotion recognition: a comparison of BERT, DistilBERT, RoBERTa, XLNet and ELECTRA. In: Proceedings of the 2022 3rd International Conference on Control, Robotics and Intelligent System (CCRIS ’22); 2022 Aug 26-28; Virtual Event, China. New York (NY): Association for Computing Machinery; 2022. p. 1-7. Available from: https://doi.org/10.1145/3562007.3562051
42. [42] Ameur A, Hamdi S, Ben Yahia S. Sentiment analysis for hotel reviews: a systematic literature review. ACM Computing Surveys. 2023;56(2):Article 51, 1-38. Available from: https://doi.org/10.1145/3605152