Deception Detection in Turkish Hotel Reviews: A Comparative Study of Machine Learning and Deep Learning Approaches

Abstract

This study presents a comparative analysis of machine learning and deep learning models for detecting deception in Turkish hotel reviews. For this purpose, fake, real and artificial intelligence-generated Turkish language hotel reviews are utilized. The real reviews dataset was created by filtering the data obtained from the Tripadvisor platform according to certain criteria, while two separate classes of deceptive reviews were established: one consisting of reviews written by human volunteers and another generated by artificial intelligence. The performances of machine learning and deep learning algorithms were tested for the detection of fake and AI-generated reviews. The results show that the BERTurk model achieved the highest performance with an F1-score of 0.93, followed by Artificial Neural Network (ANN), while Long Short Term Memory (LSTM), Support Vector Machine (SVM), and Random Forest (RF) also demonstrated strong classification capabilities. This study represents one of the first comprehensive deception detection studies in the Turkish language and contributes to the literature by demonstrating the effectiveness of transformer-based models for this task.

Keywords

• Deception Detection
• Fake Reviews
• Turkish Texts
• Machine Learning
• Deep Learning
• Natural Language Processing

References

1. Akkol, E., & Gökşen, Y. (2024). Creating A Comprehensive Data Set For Deception Detection Studies In Turkish Texts. Research Journal of Business and Management, 11(2), 138-145. https://doi.org/10.17261/Pressacademia.2024.1960 google scholar
2. Alsubari, S. N., Deshmukh, S. N., Alqarni, A. A., Alsharif, N., Aldhyani, T. H., Alsaade, F. W., & Khalaf, O. I. (2022). Data analytics for the identification of fake reviews using supervised learning. Computers, Materials & Continua, 70(2), 3189-3204. google scholar
3. Arzu, M., & Aydoğan, M. (2023). Türkçe Duygu Sınıflandırma İçin Transformers Tabanlı Mimarilerin Karşılaştırılmalı Analizi. Computer Science, (IDAP-2023), 1-6. google scholar
4. Barbado, R., Araque, O., & Iglesias, C. A. (2019). A framework for fake review detection in online consumer electronics retailers. Information Processing & Management, 56(4), 1234-1244. google scholar
5. Biau, G. (2012). Analysis of a random forests model. The Journal of Machine Learning Research, 13(1), 1063-1095. google scholar
6. Bond, C. F., & Depaulo, B. M. (2006). Accuracy of Deception Judgments. In Personality and Social Psychology Review (Vol. 3). google scholar
7. Bozuyla, M. (2022). Advanced turkish fake news prediction with bidirectional encoder representations from transformers. Konya Journal of Engineering Sciences, 10(3), 750-761. google scholar
8. Breiman, L. (2001). Random forests. Machine learning, 45, 5-32. google scholar

9. Canbay, P., & Ekinci, E. (2023). Derin ve Sığ Makine Öğrenmesi Yöntemleri ile Türkçe Tweetlerden Saldırgan Dil Tespiti. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, 16(1), 1-10. google scholar
10. Colwell, K., Hiscock, C. K., & Memon, A. (2002). Interviewing techniques and the assessment of statement credibility. Applied Cognitive Psychology, 16(3), 287–300. https://doi.org/10.1002/acp.788 google scholar
11. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273-297. google scholar
12. DePaulo, B. M., & Pfeifer, R. L. (1986). On‐the‐Job Experience and Skill at Detecting Deception. Journal of Applied Social Psychology, 16(3), 249–267. https://doi.org/10.1111/J.1559-1816.1986.TB01138.X google scholar
13. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) (pp. 4171-4186). google scholar
14. Dev, D. G., & Bhatnagar, V. (2024). Hybrid RFSVM: Hybridization of SVM and Random Forest Models for Detection of Fake News. Algorithms, 17(10), 459. google scholar
15. Gers, F. A., Schmidhuber, J., & Cummins, F. (1999). Continual prediction using LSTM with forget gates. In Neural Nets WIRN Vietri-99: Proceedings of the 11th Italian Workshop on Neural Nets, Vietri Sul Mare, Salerno, Italy, 20–22 May 1999 (pp. 133-138). Springer London. google scholar
16. Gupta, R., Jindal, V., & Kashyap, I. (2024). Recent state-of-the-art of fake review detection: a comprehensive review. The Knowledge Engineering Review, 39, e8. doi:10.1017/S0269888924000067 google scholar
17. Güler, G., & Gündüz, S. (2023). Deep learning based fake news detection on social media. International Journal of Information Security Science, 12(2), 1-21. google scholar
18. Koru, G. K., & Uluyol, Ç. (2024). Detection of Turkish fake news from tweets with BERT models. IEEE Access, 12, 14918-14931. google scholar
19. Köhnken, G. (1987). Training police officers to detect deceptive eyewitness statements: Does it work? Social Behaviour. google scholar
20. Kumar, A., & Saroj, K. (2020). Impact of Customer Review on Social Media Marketing Strategies. International Journal of Research in Business Studies, 5(2), 105-114. google scholar
21. Lu, J., Zhan, X., Liu, G., Zhan, X., & Deng, X. (2023). BSTC: A Fake Review Detection Model Based on a Pre-Trained Language Model and Convolutional Neural Network. Electronics, 12(10), 2165. https://doi.org/10.3390/electronics12102165 google scholar
22. Mathur, L. (2021). Affect-Aware Machine Learning Models for Deception Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), 15968–15969. https://doi.org/10.1609/aaai.v35i18.17980 google scholar
23. Mathur, L., & Matarić, M. J. (2020). Introducing Representations of Facial Affect in Automated Multimodal Deception Detection. ICMI 2020 - Proceedings of the 2020 International Conference on Multimodal Interaction, 305–314. https://doi.org/10.1145/3382507.3418864 google scholar
24. Mothukuri, R., Aasritha, A., Maremalla, K. C., Pokala, K. N., & Perumalla, G. K. (2022, April). Fake review detection using unsupervised learning. In 2022 International Conference on Sustainable Computing and Data Communication Systems (ICSCDS) (pp. 119-125). IEEE. google scholar
25. Oh, Y. W., & Park, C. H. (2021). Machine cleaning of online opinion spam: Developing a machine-learning algorithm for detecting deceptive comments. American behavioral scientist, 65(2), 389-403. google scholar
26. Ott, M., Cardie, C., & Hancock, J. T. (2013). Negative deceptive opinion spam. In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: human language technologies (pp. 497-501). google scholar
27. Ott, M., Choi, Y., Cardie, C., & Hancock, J. T. (2011). Finding deceptive opinion spam by any stretch of the imagination. arXiv preprint arXiv:1107.4557. google scholar
28. Paget, S. (2023). Local consumer review survey 2023. Online verfügbar unter https://www. brightlocal. com/research/local-consumer-review-survey/, zuletzt geprüft am, 19, 2023. google scholar
29. Ren, Y., & Zhang, Y. (2016, December). Deceptive opinion spam detection using neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (pp. 140-150). google scholar
30. Salminen, J., Kandpal, C., Kamel, A. M., Jung, S. G., & Jansen, B. J. (2022). Creating and detecting fake reviews of online products. Journal of Retailing and Consumer Services, 64, 102771. google scholar
31. Schweter, S. (2020). BERTurk-BERT models for Turkish. Zenodo, 2020, 3770924. google scholar
32. Shinde, S. A., Pawar, R. R., Jagtap, A. A., Tambewagh, P. A., Rajput, P. U., Mali, M. K., Kale, S. D. & Mulik, S. V. (2024). Deceptive opinion spam detection using bidirectional long short-term memory with capsule neural network. Multimedia Tools and Applications, 83(15), 45111-45140. google scholar
33. Taskin, S. G., Kucuksille, E. U., & Topal, K. (2022). Detection of Turkish fake news in Twitter with machine learning algorithms. Arabian Journal for Science and Engineering, 47(2), 2359-2379. google scholar
34. Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24-54. google scholar
35. Venkatesh, B., & Yadav, B. R. (2024). HACNN: hierarchical attention convolutional neural network for fake review detection. Social Network Analysis and Mining, 14(1), 223. google scholar
36. Vrij, A. (2019). Deception and truth detection when analyzing nonverbal and verbal cues. Applied Cognitive Psychology, 33(2), 160–167. https://doi.org/10.1002/acp.3457 google scholar
37. Wang, J., Kan, H., Meng, F., Mu, Q., Shi, G., & Xiao, X. (2020). Fake review detection based on multiple feature fusion and rolling collaborative training. IEEE Access, 8, 182625-182639. google scholar
38. Weng, C. H., Lin, K. C., & Ying, J. C. (2022). Detection of chinese deceptive reviews based on pre-trained language model. Applied Sciences, 12(7), 3338. google scholar
39. Wiener, E., Pedersen, J. O. & Weigend, A. S. (1995). A neural network approach to topic spotting. Proceedings of SDAIR-95, 4th Annual Symposium on Document Analysis and Information Retrieval, 317-332. google scholar