E-posta Kimlik Avı Sınıflandırması için Derin Öğrenme ve Açıklanabilir Yapay Zekâ: TabNet, NODE ve FT-Transformer Modellerinin Karşılaştırmalı Bir İncelemesi

Yıl 2025, ERKEN GÖRÜNÜM, 1 - 1

Burçak Asal , Saadin Oyucu , Ferdi Doğan , Onur Polat , Ahmet Aksöz

https://doi.org/10.2339/politeknik.1745083

Öz

Siber güvenlik tehditlerinin değişen yapısında, kimlik avı e-postaları sürekli ve yıkıcı bir saldırı vektörü olmaya devam etmektedir. Bu çalışma, tablosal veriler kullanılarak gerçekleştirilen bir kimlik avı e-posta sınıflandırma görevinde, TabNet, NODE (Neural Oblivious Decision Ensembles) ve FT-Transformer mimarilerine odaklanarak derin öğrenme modellerinin etkinliğini araştırmaktadır. Kullanılan veri kümesi, e-postaların dilsel ve yapısal özelliklerini yansıtan sekiz giriş özelliği ile birlikte kimlik avı ya da normal sınıflandırmasını belirten ikili bir etiketi içermektedir. Ayrıca, veri kümesinde gözlenen ciddi sınıf dengesizliği problemiyle başa çıkmak amacıyla NearMiss alt örnekleme yaklaşımı uygulanmıştır. Deneysel sonuçlar, her üç modelin de güçlü performanslar sergilediğini ancak FT-Transformer modelinin en yüksek doğruluk (accuracy) ve dengeli hassasiyet-duyarlılık (precision-recall) skorlarıyla TabNet ve NODE modellerinden daha iyi performans gösterdiğini ortaya koymaktadır. Ayrıca, FT-Transformer modelinin karar verme sürecini yorumlamak için SHAP ve LIME gibi açıklanabilir yapay zekâ (XAI) yöntemleri kullanılmıştır ki bu karar verme süreci yazım hataları, eşsiz kelime sayıları ve aciliyetle ilişkili anahtar kelimelerin kimlik avı tespitinde kritik bir rol oynadığı vurgulamaktadır. Elde edilen bulgular, siber güvenlik alanındaki sekmeli veriler üzerinde transformer tabanlı yaklaşımların potansiyelini vurgulamakta ve kimlik avı tespit sistemlerinde güven ve şeffaflığı artırmak adına açıklanabilir yapay zekânın önemini ortaya koymaktadır.

Anahtar Kelimeler

Açıklanabilir Yapay Zeka , TabNet , NODE , FT-Transformer , E-posta Kimlik Avı Sınıflandırması , SHAP , LIME

Deep Learning and Explainable AI for Email Phishing Classification: A Comparative Study of TabNet, NODE and FT-Transformer Models

Öz

In the changing landscape of cybersecurity threats, phishing emails indicate a persistent and damaging attack vector. This study investigates the effectiveness of deep learning models on a phishing email classification task using tabular data and focusing on TabNet, NODE (Neural Oblivious Decision Ensembles), and FT-Transformer architectures. The utilized dataset includes eight input features capturing linguistic and structural characteristics of emails, with a binary label indicating phishing or normal classification. Additionally, the NearMiss under-sampling approach is applied to address the significant class imbalance. Experimental results demonstrate that while all three models achieve strong performance, the FT-Transformer model outperforms TabNet and NODE by achieving the highest classification accuracy and balanced precision-recall scores. Additionally, explainable artificial intelligence (XAI) methods, SHAP and LIME, are employed to interpret the FT-Transformer model’s decision-making process, which highlights the critical role of spelling errors, unique word counts, and urgency-related keywords in phishing detection. The findings emphasize the potential of transformer-based approaches for tabular cybersecurity applications and indicate the importance of interpretable AI in enhancing trust and transparency in phishing detection systems.

Anahtar Kelimeler

Explainable Artificial Intelligence (XAI) , TabNet , NODE , FT-Transformer , Email Phishing Classification , SHAP , LIME

Kaynakça

• [1] Apwg, “Phishing Activity Trends Report”, 4th Quarter 2023. 2024, Anti-Phishing Working Group, (2024).
• [2] Proofpoint, “2024 State of the Phish – Today’s Cyber Threats and Phishing Protection”, Proofpoint, (2024).
• [3] Ünal, C. and Şahin, İ., “İstenmeyen Elektronik Postaların (SPAM) Filtrelenmesi için Bir Uzman Sistem Tasarımı ve Gerçekleştirilmesi.”, Politeknik Dergisi, 20(2), 267-274, (2017).
• [4] Çıtlak, O., Dörterler, M. and Dogru, İ., “A hybrid spam detection framework for social networks.”, Politeknik Dergisi, 26(2), 823-837, (2022).
• [5] Fan, Z., Li, W., Laskey, K. B. and Chang, K. C., “Investigation of phishing susceptibility with explainable artificial intelligence.”, Future Internet, 16(1), 31, (2024).
• [6] Divakaran, D.M. and A. Oest, “Phishing detection leveraging machine learning and deep learning: A review.”, IEEE Security & Privacy, 20(5): p. 86-95, (2022).
• [7] Zuraiq, A.A. and M. Alkasassbeh. “Phishing detection approaches.”, In 2019 2nd International Conference on new Trends in Computing Sciences (ICTCS), IEEE, (2019).
• [8] Mohammad, R.M., F. Thabtah, and L. McCluskey, “Intelligent rule‐based phishing websites classification.”, IET Information Security, 8(3): p. 153-160, (2014).
• [9] Pentapalli, L.S., et al., “A Gradient-Optimized TSK Fuzzy Framework for Explainable Phishing Detection.”, arXiv preprint arXiv:2504.18636, (2025).
• [10] Gautam, S., K. Rani, and B. Joshi. “Detecting phishing websites using rule-based classification algorithm: a comparison.”, In Information and Communication Technology for Sustainable Development: Proceedings of ICT4SD 2016, Volume 1. Springer, (2018).
• [11] Najjar-Ghabel, S., S. Yousefi, and P. Habibi., “Comparative Analysis and Practical Implementation of Machine Learning Algorithms for Phishing Website Detection.”, in 2024 9th International Conference on Computer Science and Engineering (UBMK), IEEE, (2024).
• [12] Raj, D., R. Kumar, and S. Joshi. “Automated AI System for Online Phishing Detection and Mitigation.”, in 2024 International Conference on Electrical Electronics and Computing Technologies (ICEECT), IEEE, (2024).
• [13] G, G., T. Vidhyabharathi, and V. Sangeetha, “Outsmarting Phishers: A Comparative Analysis of Machine Learning Techniques.”, Journal of Information Technology and Digital World, 6(4): p. 347-361, (2024).
• [14] Singh, P., T. Hasija, and K. Ramkumar. “Integrated Machine Learning Approach to Phishing Detection: Comparing SVM, Random Forest, and XGBoost Models.”, in 2024 4th International Conference on Technological Advancements in Computational Sciences (ICTACS), IEEE (2024).
• [15] Saraswathi, P., et al. “Evaluating the Efficacy of Machine Learning Methods in Phishing Detection: A Comparative Analysis.”, in 2024 IEEE International Conference on Blockchain and Distributed Systems Security (ICBDS), IEEE, (2024).
• [16] El-Metwaly, A.E.-S., et al. “Detection of Phishing URLs Based on Machine Learning and Cybersecurity.”, in 2024 International Telecommunications Conference (ITC-Egypt), IEEE, (2024).
• [17] Sarma, D., et al. “Comparative analysis of machine learning algorithms for phishing website detection.”, in Inventive Computation and Information Technologies: Proceedings of ICICIT 2020, Springer, (2021).
• [18] Lone, A.N., et al. “Performance Evaluation on Detection of Phishing Websites Using Machine Learning Techniques.” in 2024 International Conference on Electrical Electronics and Computing Technologies (ICEECT), IEEE, (2024).
• [19] Dwivedi, D., et al., “Machine learning-powered defense against phishing websites.”, in 2024 IEEE International Students' Conference on Electrical, Electronics and Computer Science (SCEECS), IEEE, (2024).
• [20] Mousavi, S., M. Bahaghighat, and F. Ozen., “Advancements in Phishing Website Detection: A Comprehensive Analysis of Machine Learning and Deep Learning Models.”, in 2024 32nd Signal Processing and Communications Applications Conference (SIU), IEEE, (2024).
• [21] Ahmad, K.I., et al., “A Data-Driven Approach for Online Phishing Activity Detection.”, in 2024 International Conference on Advances in Computing, Communication, Electrical, and Smart Systems (iCACCESS), IEEE, (2024).
• [22] Altwaijry, N., et al., “Advancing phishing email detection: A comparative study of deep learning models.”, Sensors, 24(7): p. 2077, (2024).
• [23] Lobo, R., M.N. Abbas, and M.N. Asghar., “Email Phishing Attack Detection using Recurrent and Feed-forward Neural Networks.”, in 2023 Cyber Research Conference-Ireland (Cyber-RCI), IEEE, (2023).
• [24] Truong, C.K., P. Hao Do, and T. Duc Le, “A comparative analysis of email phishing detection methods: a deep learning perspective.”, IET, (2023).
• [25] MohamedAli, R.S. and R.A. Abduhameed., “Phishing Email Detection: Survey.”, in International Conference on Advanced Engineering, Technology and Applications, Springer, (2024).
• [26] Atawneh, S. and H. Aljehani, “Phishing email detection model using deep learning.”, Electronics, 12(20): p. 4261, (2023).
• [27] Yu, S., et al., “Phishing detection based on multi-feature neural network.”, in 2022 IEEE International Performance, Computing, and Communications Conference (IPCCC), IEEE, (2022).
• [28] Alshingiti, Z., et al., “A deep learning-based phishing detection system using CNN, LSTM, and LSTM-CNN.”, Electronics, 12(1): p. 232, (2023).
• [29] Do, N.Q., et al., “Deep learning for phishing detection: Taxonomy, current challenges and future directions.”, Ieee Access, 10: p. 36429-36463, (2022).
• [30] Camenisch, J., et al., “JCS special issue on EU-funded ICT research on Trust and Security.”, SAGE Publications Sage UK: London, England. p. 1-5, (2010).
• [31] AbuAlghanam, O., et al., “A new hierarchical architecture and protocol for key distribution in the context of IoT-based smart cities.”, Journal of Information Security and Applications, 67: p. 103173, (2022).
• [32] Gorishniy, Y., et al., “Revisiting deep learning models for tabular data.”, Advances in neural information processing systems, 34: p. 18932-18943, (2021).
• [33] Ribeiro, M.T., S. Singh, and C. Guestrin., “" Why should i trust you?" Explaining the predictions of any classifier.”, in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, (2016).
• [34] El Aassal, A., et al., “An in-depth benchmarking and evaluation of phishing detection research for security needs.”, Ieee Access, 8: p. 22170-22192, (2020).
• [35] Zeng, V., et al., “Diverse datasets and a customizable benchmarking framework for phishing.”, in Proceedings of the Sixth International Workshop on Security and Privacy Analytics, (2020).
• [36] Champa, A.I., F. Rabbi, and M.F. Zibran., “Why phishing emails escape detection: A closer look at the failure points.”, in 2024 12th International Symposium on Digital Forensics and Security (ISDFS), IEEE, (2024).
• [37] Sánchez-Paniagua, M., et al., “Phishing websites detection using a novel multipurpose dataset and web technologies features.”, Expert Systems with Applications, 207: p. 118010, (2022).
• [38] Bountakas, P. and C. Xenakis, “Helphed: Hybrid ensemble learning phishing email detection.”, Journal of network and computer applications, 210: p. 103545, (2023).
• [39] KnowBe, “Phishing By Industry Benchmarking Report.” (2024).
• [40] SlashNext, “The 2024 Phishing Intelligence Report.” (2024).
• [41] Elsharief, A.F., “Comparative Evaluation of Machine Learning Models for Phishing Email Detection.”, (2025).
• [42] Aleroud, A., Abu-Shanab, E., Al-Aiad, A., and Alshboul, Y. “An examination of susceptibility to spear phishing cyber attacks in non-English speaking communities.”, Journal of Information Security and Applications, 55, 102614, (2020).
• [43] Pantziou, G., F. Makedon, and P. Belsis, “Special issue on privacy and security on pervasive e-health and assistive environments.”, Security & Communication Networks, 4(11), (2011).
• [44] Arik, S.Ö. and T. Pfister. “Tabnet: Attentive interpretable tabular learning.”, in Proceedings of the AAAI conference on artificial intelligence, (2021).
• [45] Popov, S., S. Morozov, and A. Babenko, “Neural oblivious decision ensembles for deep learning on tabular data.”, arXiv preprint arXiv:1909.06312, (2019).
• [46] Cratchley, E., “Email Phishing Dataset.” https://www.kaggle.com/datasets/ethancratchley/email-phishing-dataset, (2025).
• [47] Mani, I. and I. Zhang., “kNN approach to unbalanced data distributions: a case study involving information extraction.” in Proceedings of workshop on learning from imbalanced datasets, ICML United States, (2003).