Supervised Machine Learning Based Fake Profile Detection Using User Ratings and Reviews in Recommender Systems

Abstract

Recommendation systems produce content based on user's interests and aim to increase user satisfaction. In this way, the system keeps the user constantly active. Therefore, the reliability and robustness of these systems are essential. However, in recent years, with the influence of popular culture, recommendation systems have been struggling with fake users to highlight a particular product more or, conversely, to reduce the popularity of the product. Fake accounts mimic real user data and provide misleading information to the systems. This affects the accuracy of recommendation algorithms. This paper proposes a novel approach to detect fake user profiles by combining two different data sources: rating data and product reviews by using machine learning techniques, such as Decision Trees, Logistic Regression, Support Vector Machines, k-Nearest Neighbors and Naive Bayes algorithms. We also test the impact of integrating ensemble learning techniques on classification success. The research results show that the ensemble learning method Stack Classifier model has the highest detection success with an F1-score of 81.11%. This highlights that the innovative approach using multiple data sources together provides a more robust and reliable solution for detecting fake profiles, thus improving the accuracy and efficiency of recommender systems.

Keywords

• Recommender system
• fake profile detection
• machine learning
• robustness

References

1. C. Lin, S. Chen, M. Zeng, S. Zhang, M. Gao, H. Li, Shilling black-box recommender systems by learning to generate fake user profiles, IEEE Transactions on Neural Networks and Learning Systems 35 (1) (2022) 1305-1319.
2. T. T. Nguyen, N. Quoc Viet Hung, T. T. Nguyen, T. T. Huynh, T. T. Nguyen, M. Weidlich, H. Yin, Manipulating recommender systems: a survey of poisoning attacks and countermeasures, ACM Computing Surveys 57 (1) (2024) 1-39.
3. J. Suryawanshi, S. M. Abdul, R. P. Lal, A. Aramanda, N. Hoque, N. Yusoff, Enhanced recommender systems with the removal of fake user profiles, Procedia Computer Science 235 (2024) 347–360.
4. R. A. Zayed, L. F. Ibrahim, H. A. Hefny, H. A. Salman, A. Almohimeed, Using ensemble method to detect attacks in the recommender system, IEEE Access 11 (2023) 111315-111323.
5. Q. Zhou, J. Wu, L. Duan, Recommendation attack detection based on deep learning, Journal of Information Security and Applications 52 (2020) 102493.
6. B. Mobasher, R. Burke, R. Bhaumik, C. Williams, Effective attack models for shilling item-based collaborative filtering system, In Proceedings of the 2005 WebKDD Workshop, held in conjuction with ACM SIGKDD, Chicago, Illinois, 2005.
7. S. Shaw, A. Singh, Using machine learning algorithms to alleviate the shilling attack in a recommendation system, 11 (5) (2023) 1879-1883.
8. S. Rani, M. Kaur, M. Kumar, V. Ravi, U. Ghosh, J. R. Mohanty, Detection of shilling attack in recommender system for YouTube video statistics using machine learning techniques, Soft Computing 27 (1) (2023) 377-389.

9. J. Cao, Z. Wu, B. Mao, Y. Zhang, Shilling attack detection utilizing semi-supervised learning method for collaborative recommender system, World Wide Web 16 (5–6) (2013) 729–748.
10. A. B. Chopra, V. S. Dixit, An adaptive RNN algorithm to detect shilling attacks for online products in hybrid recommender system, Journal of Intelligent Systems 31 (1) (2022) 1133–1149.
11. R. A. Duma, Z. Niu, A. S. Nyamawe, J. Tchaye-Kondi, A. A. Yusuf, A Deep Hybrid Model for fake review detection by jointly leveraging review text, overall ratings, and aspect ratings, Soft Computing 27 (10) (2023) 6281–6296.
12. Y. Cheng, J. Guo, S. Long, Y. Wu, M. Sun, R. Zhang, Advanced Financial Fraud Detection Using GNN-CL Model, 2024 International Conference on Computers, Information Processing and Advanced Education (CIPAE), Ottowa, ON, Canada, 2024, pp. 453–460.
13. Z. Han, T. Zhou, G. Chen, J. Chen, C. Fu, A Robust Rating Prediction Model for Recommendation Systems Based on Fake User Detection and Multi-Layer Feature Fusion, Big Data Mining and Analytics 8 (2) (2025) 292–309.
14. S. Rayana, L. Akoglu, Collective opinion spam detection: Bridging review networks and metadata, in: L. Cao, C. Zhang (Eds.) KDD '15: The 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW Australia, 2015, pp. 985–994.
15. A. Sihombing, A. C. M. Fong, Fake Review Detection on Yelp Dataset Using Classification Techniques in Machine Learning, Proceedings of the 4th International Conference on Contemporary Computing and Informatics, IC3I, Singapore, 2019, pp. 64–68.
16. R. Barbado, O. Araque, C. A. Iglesias. A framework for fake review detection in online consumer electronics retailers, Information Processing and Management 56 (4) (2019) 1234-1244.
17. Y. Jian, X. Chen, X. Wang, Y. Liu, X. Chen, X. Lan, W. Wang, H. Wang, A metadata-aware detection model for fake restaurant reviews based on multimodal fusion, Neural Computing and Applications 37 (1) (2024) 475–498.
18. A. Mukherjee, V. Venkataraman, B. Liu, N. Glance, Fake Review Detection: Classification and Analysis of Real and Pseudo Reviews, Techical Report, Department of Computer Science (UIC-CS-2013-03) University of Illinois (2013) Chicago.
19. W. Zhou, J. Wen, Q. Xiong, M. Gao, J. Zeng, SVM-TIA a shilling attack detection method based on SVM and target item analysis in recommender systems, Neurocomputing 210 (2016) 197–205.
20. H. İ. Ayaz, Z. Kamışlı Öztürk, Shilling attack detection with one class support vector machines, Necmettin Erbakan University Journal of Science and Engineering 5 (2) (2023) 246-256.
21. P. K. Singh, P. K. D. Pramanik, N. Sinhababu, P. Choudhury, Detecting unknown shilling attacks in recommendation systems, Wireless Personal Communications 137 (1) (2024) 259–286.
22. A. M. Elmogy, U. Tariq, A. Ibrahim, A. Mohammed, Fake Reviews Detection using Supervised Machine Learning, IJACSA) International Journal of Advanced Computer Science and Applications 12 (1) 2021 601-606.
23. A. Dheyaa, YelpReviwsDB, https://www.kaggle.com/datasets, Accessed 2 April 2025.
24. T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space, (2013), https://arxiv.org/pdf/1301.3781, Accessed 2 April 2025.
25. B. Mahesh, Machine learning algorithms - a review, International Journal of Science and Research 9 (1) (2020) 381-386.
26. I. H. Sarker, Machine learning: algorithms, real-world applications and research directions, SN Computer Science 2 (3) (2021) 160.
27. X. Dong, Z. Yu, W. Cao, Y. Shi, Q. Ma, A survey on ensemble learning, Frontiers of Computer Science 14 (2) (2020) 241-258.
28. Scikit-learn, scikit-learn: Machine Learning in Python, https://scikit-learn.org/stable, Accessed 28 June 2025.
29. T. Chen, C. Guestrin, XGBoost: A scalable tree boosting system, Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California, USA, 2016, pp. 785-794.
30. W. J. Youden, Index for rating diagnostic tests, Cancer 3 (1) (1950) 32-35.