Deception Detection in Turkish Hotel Reviews: A Comparative Study of Machine Learning and Deep Learning Approaches

Year 2025, Volume: 9 Issue: 2, 402 - 418, 31.12.2025

Ekin Akkol , Yılmaz Gökşen

https://doi.org/10.26650/acin.1606539

https://izlik.org/JA92EP59AR

Abstract

This study presents a comparative analysis of machine learning and deep learning models for detecting deception in Turkish hotel reviews. For this purpose, fake, real and artificial intelligence-generated Turkish language hotel reviews are utilized. The real reviews dataset was created by filtering the data obtained from the Tripadvisor platform according to certain criteria, while two separate classes of deceptive reviews were established: one consisting of reviews written by human volunteers and another generated by artificial intelligence. The performances of machine learning and deep learning algorithms were tested for the detection of fake and AI-generated reviews. The results show that the BERTurk model achieved the highest performance with an F1-score of 0.93, followed by Artificial Neural Network (ANN), while Long Short Term Memory (LSTM), Support Vector Machine (SVM), and Random Forest (RF) also demonstrated strong classification capabilities. This study represents one of the first comprehensive deception detection studies in the Turkish language and contributes to the literature by demonstrating the effectiveness of transformer-based models for this task.

Keywords

Deception Detection , Fake Reviews , Turkish Texts , Machine Learning , Deep Learning , Natural Language Processing

References

• Akkol, E., & Gökşen, Y. (2024). Creating A Comprehensive Data Set For Deception Detection Studies In Turkish Texts. Research Journal of Business and Management, 11(2), 138-145. https://doi.org/10.17261/Pressacademia.2024.1960 google scholar
• Alsubari, S. N., Deshmukh, S. N., Alqarni, A. A., Alsharif, N., Aldhyani, T. H., Alsaade, F. W., & Khalaf, O. I. (2022). Data analytics for the identification of fake reviews using supervised learning. Computers, Materials & Continua, 70(2), 3189-3204. google scholar
• Arzu, M., & Aydoğan, M. (2023). Türkçe Duygu Sınıflandırma İçin Transformers Tabanlı Mimarilerin Karşılaştırılmalı Analizi. Computer Science, (IDAP-2023), 1-6. google scholar
• Barbado, R., Araque, O., & Iglesias, C. A. (2019). A framework for fake review detection in online consumer electronics retailers. Information Processing & Management, 56(4), 1234-1244. google scholar
• Biau, G. (2012). Analysis of a random forests model. The Journal of Machine Learning Research, 13(1), 1063-1095. google scholar
• Bond, C. F., & Depaulo, B. M. (2006). Accuracy of Deception Judgments. In Personality and Social Psychology Review (Vol. 3). google scholar
• Bozuyla, M. (2022). Advanced turkish fake news prediction with bidirectional encoder representations from transformers. Konya Journal of Engineering Sciences, 10(3), 750-761. google scholar
• Breiman, L. (2001). Random forests. Machine learning, 45, 5-32. google scholar
• Canbay, P., & Ekinci, E. (2023). Derin ve Sığ Makine Öğrenmesi Yöntemleri ile Türkçe Tweetlerden Saldırgan Dil Tespiti. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, 16(1), 1-10. google scholar
• Colwell, K., Hiscock, C. K., & Memon, A. (2002). Interviewing techniques and the assessment of statement credibility. Applied Cognitive Psychology, 16(3), 287–300. https://doi.org/10.1002/acp.788 google scholar
• Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273-297. google scholar
• DePaulo, B. M., & Pfeifer, R. L. (1986). On‐the‐Job Experience and Skill at Detecting Deception. Journal of Applied Social Psychology, 16(3), 249–267. https://doi.org/10.1111/J.1559-1816.1986.TB01138.X google scholar
• Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) (pp. 4171-4186). google scholar
• Dev, D. G., & Bhatnagar, V. (2024). Hybrid RFSVM: Hybridization of SVM and Random Forest Models for Detection of Fake News. Algorithms, 17(10), 459. google scholar
• Gers, F. A., Schmidhuber, J., & Cummins, F. (1999). Continual prediction using LSTM with forget gates. In Neural Nets WIRN Vietri-99: Proceedings of the 11th Italian Workshop on Neural Nets, Vietri Sul Mare, Salerno, Italy, 20–22 May 1999 (pp. 133-138). Springer London. google scholar
• Gupta, R., Jindal, V., & Kashyap, I. (2024). Recent state-of-the-art of fake review detection: a comprehensive review. The Knowledge Engineering Review, 39, e8. doi:10.1017/S0269888924000067 google scholar
• Güler, G., & Gündüz, S. (2023). Deep learning based fake news detection on social media. International Journal of Information Security Science, 12(2), 1-21. google scholar
• Koru, G. K., & Uluyol, Ç. (2024). Detection of Turkish fake news from tweets with BERT models. IEEE Access, 12, 14918-14931. google scholar
• Köhnken, G. (1987). Training police officers to detect deceptive eyewitness statements: Does it work? Social Behaviour. google scholar
• Kumar, A., & Saroj, K. (2020). Impact of Customer Review on Social Media Marketing Strategies. International Journal of Research in Business Studies, 5(2), 105-114. google scholar
• Lu, J., Zhan, X., Liu, G., Zhan, X., & Deng, X. (2023). BSTC: A Fake Review Detection Model Based on a Pre-Trained Language Model and Convolutional Neural Network. Electronics, 12(10), 2165. https://doi.org/10.3390/electronics12102165 google scholar
• Mathur, L. (2021). Affect-Aware Machine Learning Models for Deception Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), 15968–15969. https://doi.org/10.1609/aaai.v35i18.17980 google scholar
• Mathur, L., & Matarić, M. J. (2020). Introducing Representations of Facial Affect in Automated Multimodal Deception Detection. ICMI 2020 - Proceedings of the 2020 International Conference on Multimodal Interaction, 305–314. https://doi.org/10.1145/3382507.3418864 google scholar
• Mothukuri, R., Aasritha, A., Maremalla, K. C., Pokala, K. N., & Perumalla, G. K. (2022, April). Fake review detection using unsupervised learning. In 2022 International Conference on Sustainable Computing and Data Communication Systems (ICSCDS) (pp. 119-125). IEEE. google scholar
• Oh, Y. W., & Park, C. H. (2021). Machine cleaning of online opinion spam: Developing a machine-learning algorithm for detecting deceptive comments. American behavioral scientist, 65(2), 389-403. google scholar
• Ott, M., Cardie, C., & Hancock, J. T. (2013). Negative deceptive opinion spam. In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: human language technologies (pp. 497-501). google scholar
• Ott, M., Choi, Y., Cardie, C., & Hancock, J. T. (2011). Finding deceptive opinion spam by any stretch of the imagination. arXiv preprint arXiv:1107.4557. google scholar
• Paget, S. (2023). Local consumer review survey 2023. Online verfügbar unter https://www. brightlocal. com/research/local-consumer-review-survey/, zuletzt geprüft am, 19, 2023. google scholar
• Ren, Y., & Zhang, Y. (2016, December). Deceptive opinion spam detection using neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (pp. 140-150). google scholar
• Salminen, J., Kandpal, C., Kamel, A. M., Jung, S. G., & Jansen, B. J. (2022). Creating and detecting fake reviews of online products. Journal of Retailing and Consumer Services, 64, 102771. google scholar
• Schweter, S. (2020). BERTurk-BERT models for Turkish. Zenodo, 2020, 3770924. google scholar
• Shinde, S. A., Pawar, R. R., Jagtap, A. A., Tambewagh, P. A., Rajput, P. U., Mali, M. K., Kale, S. D. & Mulik, S. V. (2024). Deceptive opinion spam detection using bidirectional long short-term memory with capsule neural network. Multimedia Tools and Applications, 83(15), 45111-45140. google scholar
• Taskin, S. G., Kucuksille, E. U., & Topal, K. (2022). Detection of Turkish fake news in Twitter with machine learning algorithms. Arabian Journal for Science and Engineering, 47(2), 2359-2379. google scholar
• Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24-54. google scholar
• Venkatesh, B., & Yadav, B. R. (2024). HACNN: hierarchical attention convolutional neural network for fake review detection. Social Network Analysis and Mining, 14(1), 223. google scholar
• Vrij, A. (2019). Deception and truth detection when analyzing nonverbal and verbal cues. Applied Cognitive Psychology, 33(2), 160–167. https://doi.org/10.1002/acp.3457 google scholar
• Wang, J., Kan, H., Meng, F., Mu, Q., Shi, G., & Xiao, X. (2020). Fake review detection based on multiple feature fusion and rolling collaborative training. IEEE Access, 8, 182625-182639. google scholar
• Weng, C. H., Lin, K. C., & Ying, J. C. (2022). Detection of chinese deceptive reviews based on pre-trained language model. Applied Sciences, 12(7), 3338. google scholar
• Wiener, E., Pedersen, J. O. & Weigend, A. S. (1995). A neural network approach to topic spotting. Proceedings of SDAIR-95, 4th Annual Symposium on Document Analysis and Information Retrieval, 317-332. google scholar