Malsmsdetector: malicious text message detector with hybrid feature vector and stacked ensemble model: a comparative study

Year 2025, Volume: 14 Issue: 2, 1 - 1

Recep Sinan Arslan

Abstract

In recent years, the emergence of telecommunication systems has led to an increase in global electronic messaging traffic. Most of this traffic contains unwanted content for the user. In this study, an approach is proposed in which feature vectors generated using DBOW and PV-DM techniques are used for classification as a hybrid for spam SMS detection. In the training and testing of the proposed method, four different datasets (UCI, BEC, Big NUS and DITNUS) that are widely used are combined and used. This dataset is tested with 10 different machine learning algorithms and then a unique stacked ensemble model is proposed to increase the performance. In the tests using the model, accuracy, precision, recall, F-score and AUC values are 98.38%, 98.39%, 98.39%, 98.37% and 96.81%, respectively. When 10-fold cross validation is applied to the obtained results, the standard deviation value is 0.004. The analysis time per sample is 0.087 milliseconds.

Keywords

Mobile spam filtering, Short message service (SMS), Bag of Words (BOW), Doc2Vec, hybrid feature vector (HFV)

Project Number

FKB-2022-1092

Thanks

This work has been supported by Kayseri University Scientific Research Projects Coordination Unit under grant number #FKB-2022-1092.

Malsmsdetector: Hibrit özellik vektörü ve istiflenmiş topluluk modeli kullanan kötücül mesaj tespit aracı: karşılaştırmalı bir çalışma

Abstract

Son yıllarda telekomünikasyon sistemlerinin ortaya çıkması, küresel elektronik mesajlaşma trafiğinde (SMS veya e-posta) artışa yol açmıştır. Bu trafiğin çoğu, kullanıcı için istenmeyen içerikler içermektedir. Bu çalışmada, spam SMS tespiti için DBOW ve PV-DM teknikleri kullanılarak üretilen öznitelik vektörlerinin hibrit olarak sınıflandırma için kullanıldığı bir yaklaşım önerilmiştir. Önerilen yöntemin eğitim ve testlerinde yaygın olarak kullanılan dört farklı veri kümesi (UCI, BEC, Big NUS ve DIT NUS) birleştirilerek kullanılmıştır. Bu veriseti 10 farklı makine öğrenmesi algoritması ile test edilmiş daha sonra başarımı artırmak için özgün bir yığılmış topluluk modeli önerilmiştir. Model kullanılarak yapılan testlerde doğruluk, kesinlik, geri çağırma, F-puanı ve AUC değerleri sırasıyla %98.38, %98.39, %98.39, %98.37 ve %96.81 olmuştur. Elde edilen sonuçlara, 10 katlı cross validation yapıldığında elde edilen standart sapma değeri 0,004'tür. Örnek başına analiz süresi 0.087 milisaniyedir. Testler sonucunda hibrit özellik vektörünün kullanımının SMS spam tespiti için başarılı sonuçlar sağladığı ve sistem performansının iyileştirilmesine katkıda bulunduğu gösterilmiştir.

Keywords

Mobil spam filtreleme, Kısa mesaj servisi (SMS), Kelime Torbası (BOW), Doc2Vec, hibrit özellik vektörü (HFV)

Supporting Institution

Kayseri Üniversitesi

Project Number

FKB-2022-1092

Thanks

This work has been supported by Kayseri University Scientific Research Projects Coordination Unit under grant number #FKB-2022-1092.

References

• M. Fallgren, T. Abbas, S. Allio, J. Alonso, G. Fodor, L. Gallo, A. Kousaridas, Y. Li, Z. Li, Z. Li, J. Luo, T. Mahmoodi, T. Svensson and G. Vivier, Multicast and broadcast enablers for high-performing cellular V2X systems. IEEE Transactions on Broadcasting, 65 (2), 454-463, 2019. https://doi.org/10.1109/tbc.2019.2912619.
•    M. A. Abid, M. F. Mushtaq, U. Akram, B. Mughal, M. Ahmad and M. Imran, Recommending domain specific keywords for Twitter. Advances in Intelligent Systems and Computing, 253-263, 2019. https://doi.org/10.1007/978-3-030-36056-6_25.
•    G. M. Duc, L. Manh and D. H. Tuan, A novel method to improve the speed and the accuracy of location prediction algorithm of mobile users for Cellular Networks. Journal of Research and Development on Information and Communication Technology, 2 (36), 113, 2017. https://doi.org/10.32913/mic-ict-research-vn.v2.n36.357.
•    M. A. Abid, S. Ullah, M. A. Siddique, M. F. Mushtaq, W. Aljedaani and F. Rustam, Spam SMS filtering based on text features and supervised machine learning techniques. Multimedia Tools and Applications, 81(28), 39853–39871, 2022. https://doi.org/10.1007/s11042-022-12991-0.
•    Digital around the world, Global digital insights. https://datareportal.com/global-digital-overview, Accessed 6 May 2023.
•    R. S. Arslan, E. Ölmez and O. Er, AFWDroid: Deep Feature Extraction and Weighting for Android Malware Detection. Dicle University Journal of Engineering, 12 (2), 237-245, 2021. https://doi.org/10.24012/dumf.875036.
•    M. Tasyurek and R. S. Arslan, RT-Droid: A novel approach for real-time Android application analysis with Transfer Learning-based CNN Models. Journal of Real-Time Image Processing, 20(3), 2023. https://doi.org/10.1007/s11554-023-01311-w.
•    R. S. Arslan, FG-Droid: Grouping Based Feature Size Reduction for Android malware detection. PeerJ Computer Science, 8:e1043, 2022. https://doi.org/10.7717/peerj-cs.1043.
•    E. M. El-Alfy and A.A. AlHasan, Spam Filtering Framework for multimodal mobile communication based on dendritic cell algorithm. Future Generation Computer Systems, 64, 98-107, 2016. https://doi.org/10.1016/j.future.2016.02.018.
• S. Ballı and O. Karasoy, Development of content‐based SMS classification application by using word2vec‐based feature extraction. IET Software, 13(4), 295–304, 2019. https://doi.org/10.1049/iet-sen.2018.5046.
• R. S. Arslan, AndroAnalyzer: Android Malicious Software Detection based on Deep Learning. PeerJ Computer Science, 7:e533, 2021. https://doi.org/10.7717/peerj-cs.533.
• G. Waja, G. Patil, C. Mehta and S. Patil, How AI can be used for governance of messaging services: A study on spam classification leveraging multi-channel convolutional Neural Network. International Journal of Information Management Data Insights, 3(1), 100147, 2023. https://doi.org/10.1016/j.jjimei.2022.100147.
• Nikolina Cveticanin, What's on the other side of your inbox - 20 spam statistics for 2023. https://dataprot.net/statistics/spam-statistics, Accessed 6 May 2023.
• O. Karasoy and S. Ballı, Spam SMS detection for Turkish language with deep text analysis and deep learning methods. Arabian Journal for Science and Engineering, 47(8), 9361–9377, 2021. https://doi.org/10.1007/s13369-021-06187-1.
• S. Mannheimer, USA text message statistics updated for 2023, SMS Comparison. https://www.smscomparison.com/mass-text-messaging/2021-statistics/, Accessed 6 May 2023.
• T. Xia and X. Chen, Category-learning attention mechanism for short text filtering. Neurocomputing, 510, 15–23, 2022. https://doi.org/10.1016/j.neucom.2022.08.076.
• S. Rao, A. K. Verma and T. Bhatia, Hybrid ensemble framework with self-attention mechanism for social spam detection on Imbalanced Data. Expert Systems with Applications, 217, 119594, 2023. https://doi.org/10.1016/j.eswa.2023.119594.
• W. H. Park, I. F. Siddiqui, C. Chakraborty, N. M. Qureshi and D. R. Shin, Scarcity-aware spam detection technique for Big Data Ecosystem. Pattern Recognition Letters, 157, 67–75, 2022. https://doi.org/10.1016/j.patrec.2022.03.021.
• R. Kiran, P. Kumar and B. Bhasker, OSLCFIT (organic simultaneous lstm and cnn fit): A novel deep learning based solution for sentiment polarity classification of reviews. Expert Systems with Applications, 157, 113488, 2020. https://doi.org/10.1016/j.eswa.2020.113488.
• S. Mishra and D. Soni, SMISHING detector: A security model to detect smishing through SMS content analysis and URL behavior analysis. Future Generation Computer Systems, 108, 803–815, 2020. https://doi.org/10.1016/j.future.2020.03.021.
• S. J. Delany, M. Buckley and D. Greene, SMS spam filtering: Methods and Data. Expert Systems with Applications, 39(10), 9899–9908, 2012. https://doi.org/10.1016/j.eswa.2012.02.053.
• T. Xia and X. Chen, A weighted feature enhanced Hidden Markov model for spam SMS filtering. Neurocomputing, 444, 48–58, 2021. https://doi.org/10.1016/j.neucom.2021.02.075.
• S. Abiramasundari, V. Ramaswamy and J. Sangeetha, Spam filtering using Semantic and Rule Based model via supervised learning. Annals of the Romanian Society for Cell Biology, 25(2), 3975-3992, 2021.
• S. Bhatnagar and A. Kumar, A rule-based classification of Short Message Service Type. 2018 2nd International Conference on Inventive Systems and Control (ICISC), pp. 1-4, Coimbatore, India, 2018. https://doi.org/10.1109/icisc.2018.8398982.
• T. Xia, A constant time complexity spam detection algorithm for boosting throughput on rule-based filtering systems. IEEE Access, 8, 82653–82661, 2020. https://doi.org/10.1109/access.2020.2991328.
• M. Novo-Lourés, D. Ruano-Ordás, R. Pavón, R. Laza, S. Gómez-Meire and J. R. Méndez, Enhancing representation in the context of multiple-channel spam filtering. Information Processing & Management, 59(2), 102812, 2022. https://doi.org/10.1016/j.ipm.2021.102812.
• S. D. Gupta, S. Saha and S. K. Das, SMS SPAM detection using machine learning. Journal of Physics: Conference Series, 1797(1), 012017, 2021. https://doi.org/10.1088/1742-6596/1797/1/012017.
• V. Gupta, A. Mehta, A. Goel, U. Dixit and A. C. Pandey, Spam Detection Using Ensemble Learning. Harmony Search and Nature Inspired Optimization Algorithms, 661–668, 2018. https://doi.org/10.1007/978-981-13-0761-4_63.
• O. Abayomi-Alli, S. Misra, A. Abayomi-Alli and M. Odusami, A review of soft techniques for SMS SPAM classification: Methods, approaches and applications. Engineering Applications of Artificial Intelligence, 86, 197–212, 2019. https://doi.org/10.1016/j.engappai.2019.08.024.
• T. Xia and X. Chen, A discrete hidden Markov model for SMS spam detection. Applied Sciences, 10(14), 5011, 2020. https://doi.org/10.3390/app10145011.
• P. K. Roy, J. P. Singh and S. Banerjee, Deep learning to filter sms spam. Future Generation Computer Systems, 102, 524–533, 2020. https://doi.org/10.1016/j.future.2019.09.001.
• N. N. A. Sjarif, N. F. M Azmi, S. Chuprat, H. Sarkan, Y. Yahya and S.M. Sam, SMS SPAM message detection using term frequency-inverse document frequency and random forest algorithm. Procedia Computer Science, 161, 509–515, 2019. https://doi.org/10.1016/j.procs.2019.11.150.
• X. Liu, H. Lu and A. Nayak, A Spam Transformer Model for SMS Spam Detection. IEEE Access, 9, 80253-80263, 2021. https://doi.org/10.1109/ACCESS.2021.3081479.
• O. M. Ebadati and F. Ahmadzadeh, Classification spam email with elimination of unsuitable features with hybrid of ga-naive Bayes. Journal of Information and Knowledge Management, 18(01), 1950008, 2019. https://doi.org/10.1142/s0219649219500084.
• C. Zhao, Y. Xin, X. Li, Y. Yang and Y. Chen, A heterogeneous ensemble learning framework for SPAM detection in social networks with Imbalanced Data. Applied Sciences, 10(3), 936, 2020. https://doi.org/10.3390/app10030936.
• Y. Hong, Q. Liu, S. Zhou and Y. Luo, A spam filtering method based on multi-modal fusion. Applied Sciences, 9(6), 1152, 2019. https://doi.org/10.3390/app9061152.
• G. Jain, M. Sharma and B. Agarwal, Optimizing Semantic LSTM for spam detection. International Journal of Information Technology, 11(2), 239–250, 2018. https://doi.org/10.1007/s41870-018-0157-5.
• U. Srinivasarao and A. Sharaff, Machine intelligence based hybrid classifier for spam detection and sentiment analysis of SMS messages. Multimedia Tools and Applications, 82, 31069-31099, 2023. https://doi.org/10.1007/s11042-023-14641-5.
• G. Jain, M. Sharma and B. Agarwal, SPAM detection on social media using semantic convolutional neural network. International Journal of Knowledge Discovery in Bioinformatics, 8(1), 12–26, 2018. https://doi.org/10.4018/ijkdb.2018010102.
• A. Sharaff, C. Kamal, S. Porwal, S. Bhatia, K. Kaur and M. M. Hassan, SPAM message detection using danger theory and krill herd optimization. Computer Networks, 199, 108453, 2021. https://doi.org/10.1016/j.comnet.2021.108453.
• Gazal and K. Juneja, Two-phase fuzzy feature-filter based hybrid model for Spam Classification. Journal of King Saud University - Computer and Information Sciences, 34(10), 10339–10355, 2022. https://doi.org/10.1016/j.jksuci.2022.10.025.
• SpamAssassin, Apache Software Foundation. https://spamassassin.apache.org/, Accessed 01 January 2025.
• R. S. Arslan, Kötücül web sayfalarının tespitinde doc2vec modeli ve makine öğrenmesi yaklaşımı. European Journal of Science and Technology, 27, 792-801, 2021. https://doi.org/10.31590/ejosat.981450.
• R. S. Arslan, Kötücül URL filtreleme için derin öğrenme modeli tasarımı. European Journal of Science and Technology, 29, 122-128, 2021. https://doi.org/10.31590/ejosat.1011961.
• T. A. Almeida, J. M. G. Hidalgo and A. Yamakami, Contributions to the study of SMS Spam Filtering. Proceedings of the 11th ACM Symposium on Document Engineering, pp. 259-262, California, USA, 2011. https://doi.org/10.1145/2034691.2034742.
• G. V. Cormack, J. M. G. Hidalgo and E. P. Sánz, Spam filtering for short messages. Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management, pp. 313-320, New York, USA, 2007 . https://doi.org/10.1145/1321440.1321486.
• M. T. Nuruzzaman, C. Lee and D. Choi, Independent and personal SMS spam filtering. 2011 IEEE 11th International Conference on Computer and Information Technology, pp. 1-7, Paphos, Cyprus, 2011. https://doi.org/10.1109/cit.2011.23.
• S. Rao, A. K. Verma and T. Bhatia, A review on Social Spam Detection: Challenges, open issues, and Future Directions. Expert Systems with Applications, 186, 115742, 2021. https://doi.org/10.1016/j.eswa.2021.115742.
• O. N. Akande, O. Gbenle, O. C. Abikoye, R. G. Jimoh, H. B. Akande, A. O. Balogun and A. Fatokun, SMSPROTECT: An automatic smishing detection mobile application. ICT Express, 9(2), 168–176, 2023. https://doi.org/10.1016/j.icte.2022.05.009.
• Q. Le and T. Mikolov, Distributed Representaions of Sentences and documents, International conference on machine learning, pp. 32(2):1188-1196, Beijing, China, 2014.
• M. V. C. Aragão, E. P. Frigieri, C. A. Ynoguti and A. P. Paiva, Factorial design analysis applied to the performance of SMS anti-spam filtering systems. Expert Systems with Applications, 64, 589–604, 2016. https://doi.org/10.1016/j.eswa.2016.08.038.
• O. Abayomi-Alli, S. Misra, A. Abayomi-Alli and M. Odusami, A deep learning method for automatic sms spam classification: Performance of Learning Algorithms on Indigenous Dataset. Concurrency and Computation: Practice and Experience, 34(17), 1-15, 2022. https://doi.org/10.1002/cpe.6989.
• W. Pan, J. Li, L. Gao, L. Yue, Y. Yang, L. Deng and C. Deng, Semantic Graph Neural Network: A conversion from spam email classification to Graph Classification. Scientific Programming, 2022:6737080, 1–8, 2022. https://doi.org/10.1155/2022/6737080.
• H. H. Mansoor and A. P. Shaker, Using classification techniques to SMS spam filter. International Journal of Innovative Technology and Exploring Engineering, 8(12), 1734–1739, 2019. https://doi.org/10.35940/ijitee.l3206.1081219.