TF-IDF ve Yapısal Özelliklere Dayalı URL Tabanlı Kimlik Avı Tespitinde Makine Öğrenimi ve Ansambl Yöntemlerinin Karşılaştırmalı Analizi

Abstract

Kimlik avı (phishing) saldırıları, kullanıcıların kimlik ve finansal bilgilerini hedef alan, hızla yaygınlaşan bir siber güvenlik tehdididir; bu nedenle saldırıların erken ve düşük maliyetle tespiti kritik önem taşımaktadır. Web sayfası içeriğinin analizine dayalı geleneksel yöntemler yüksek hesaplama maliyeti ve gecikme oluştururken, statik kara liste tabanlı yaklaşımlar yeni oluşturulan veya kısa ömürlü URL'ler karşısında yetersiz kalmaktadır. Bu çalışma, yalnızca URL bilgisinden yararlanan hibrit bir tespit hattı önermektedir: karakter düzeyindeki TF-IDF n-gram temsilleri; URL uzunluğu, sembol yoğunluğu, alt alan adı derinliği ve alan adı/TLD özellikleri gibi 30 yapısal göstergeyle birleştirilmektedir. Deneyler, 549.346 URL içeren Phishing Site URLs veri seti üzerinde yürütülmüş; tekrar eden ve geçersiz etiketli kayıtların temizlenmesiyle 507.191 benzersiz URL elde edilmiş, sınıf oranı korunarak %30'luk katmanlı bir alt küme (152.157 URL) %80 eğitim / %20 test biçiminde ayrılmıştır. Logistic Regression, SGDClassifier, RidgeClassifier, Complement Naive Bayes, Random Forest, LightGBM ve XGBoost modelleri ile bunların olasılık tabanlı soft-voting birleşimi değerlendirilmiştir. Sınıf dengesizliği doğrusal modellerde RandomOverSampler, ağaç tabanlı modellerde sınıf ağırlıklandırmasıyla ele alınmış; hiperparametreler katmanlı çapraz doğrulamayla (GridSearchCV ve RandomizedSearchCV) ayarlanmış ve karar eşikleri out-of-fold tahminler üzerinden F1 skorunu maksimize edecek biçimde optimize edilmiştir. Sonuçlar, Logistic Regression'ın %97,33 doğruluk ve 0,94 F1 ile en iyi sonucu verdiğini; Ridge ve soft-voting modellerinin de Bad sınıfı için 0,93–0,94 F1 aralığında, 0,99'a ulaşan ROC-AUC değerleriyle güçlü performans sergilediğini göstermektedir. Bulgular, yüksek boyutlu URL özellik uzayında doğrusal modellerin etkili bir ayrım sağladığını ve optimize edilmiş, yalnızca URL tabanlı bir hattın güvenilir ve düşük maliyetli kimlik avı tespiti sunabileceğini ortaya koymaktadır.

Keywords

• Kimlik avı
• URL
• TF-IDF
• Makine öğrenmesi
• Ansambl öğrenme
• Sınıf dengesizliği

Supporting Institution

BAP

Project Number

FBA-2025-15092

Ethical Statement

Bu çalışma insan veya hayvan denekler üzerinde herhangi bir deney içermediğinden etik kurul onayı gerektirmemektedir.

A Comparative Analysis of Machine Learning and Ensemble Methods for URL-Based Phishing Detection Using TF-IDF and Structural Features

Abstract

Phishing attacks, which target users' credentials and financial data, are a rapidly growing cybersecurity threat, making their early and low-cost detection critical. Traditional approaches that analyze web page content introduce high computational cost and latency, while static blacklist-based methods fail against newly created or short-lived URLs. This study proposes a hybrid detection pipeline that relies solely on URL information: character-level TF-IDF n-gram representations are combined with 30 structural indicators such as URL length, symbol density, subdomain depth, and domain/TLD features. Experiments were conducted on the Phishing Site URLs dataset of 549,346 URLs; after removing duplicate and invalid-label records, 507,191 unique URLs were obtained, and a 30% stratified subset (152,157 URLs) was split into 80% training and 20% testing while preserving the class ratio. Logistic Regression, SGDClassifier, RidgeClassifier, Complement Naive Bayes, Random Forest, LightGBM, and XGBoost, together with a probability-based soft-voting ensemble, were evaluated. Class imbalance was addressed using RandomOverSampler for the linear models and class weighting for the tree-based models; hyperparameters were tuned via stratified cross-validation (GridSearchCV and RandomizedSearchCV), and decision thresholds were optimized on out-of-fold predictions to maximize the F1 score. The results show that Logistic Regression achieved the best performance with 97.33% accuracy and a 0.94 F1 score, while the Ridge and soft-voting models also performed strongly, with F1 scores in the 0.93–0.94 range for the Bad class and ROC-AUC values reaching 0.99. The findings demonstrate that linear models provide effective discrimination in the high-dimensional URL feature space and that an optimized, URL-only pipeline can deliver reliable and low-cost phishing detection.

Keywords

• Phishing
• URL
• TF-IDF
• Machine learning
• Ensemble learning
• Class imbalance

Project Number

FBA-2025-15092

References

1. Al Tawil, A., Almazaydeh, L., Qawasmeh, D., Qawasmeh, B., Al Shinwan, M., & Elleithy, K. (2024). Comparative analysis of machine learning algorithms for email phishing detection using TF-IDF, Word2Vec, and BERT. Computers, Materials & Continua, 81(2), 3395–3412. https://doi.org/10.32604/cmc.2024.057279
2. Alam, M. N., Saha, I., Sarma, D., Ulfath, R. E., Lima, F. F., & Hossain, S. (2020). Phishing attacks detection using machine learning approach. In 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT) (pp. 1173–1179). IEEE. https://doi.org/10.1109/ICSSIT48917.2020.9214225
3. Aljofey, A., Jiang, Q., Qu, Q., Huang, M., & Niyigena, J. P. (2022). An effective detection approach for phishing websites using URL and HTML features. Scientific Reports, 12(1), Article 8842. https://doi.org/10.1038/s41598-022-10841-5
4. Anandaraj, A. P. S., Ramesh, P. S., Arifulla, S., Madhuri, C. G., & Malepati, N. Y. (2024). Phishing detection system through hybrid machine learning based on URL. In 2024 Third International Conference on Electrical, Electronics, Information and Communication Technologies (ICEEICT) (pp. 1–7). IEEE. https://doi.org/10.1109/ICEEICT61591.2024.10718484
5. Asha Shajilal, B. (2024). A hybrid approach for detecting phishing mails using textual, content, and URL analysis with ensemble learning [Master's thesis, National College of Ireland]. NORMA@NCI Library.
6. BTK. (2025). Oltalama saldırıları ve zararlı bağlantı istatistikleri [Phishing attacks and malicious link statistics]. Bilgi Teknolojileri ve İletişim Kurumu. https://www.btk.gov.tr
7. Dalvi, S., Gressel, G., & Achuthan, K. (2019). Tuning the false positive rate / false negative rate with phishing detection models. International Journal of Engineering and Advanced Technology, 9(1S5), 7–13. https://doi.org/10.35940/ijeat.A1002.1291S519
8. Das Guptta, S., Shahriar, K. T., Alqahtani, H., Alsalman, D., & Sarker, I. H. (2024). Modeling hybrid feature-based phishing websites detection using machine learning techniques. Annals of Data Science, 11(1), 217–242. https://doi.org/10.1007/s40745-022-00379-8

9. Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55–67. https://doi.org/10.1080/00401706.1970.10488634
10. Hosmer, D. W., Jr., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed.). John Wiley & Sons. https://doi.org/10.1002/9781118548387
11. Jalil, S., Usman, M., & Fong, A. (2023). Highly accurate phishing URL detection based on machine learning. Journal of Ambient Intelligence and Humanized Computing, 14(7), 9233–9251. https://doi.org/10.1007/s12652-022-04426-3
12. Kaggle. (n.d.). Phishing site URLs dataset. https://www.kaggle.com/datasets/taruntiwarihp/phishing-site-urls (erişim tarihi: 08 Kasım 2025).
13. Karim, A., Shahroz, M., Mustofa, K., Belhaouari, S. B., & Joga, S. R. K. (2023). Phishing detection system through hybrid machine learning based on URL. IEEE Access, 11, 36805–36822. https://doi.org/10.1109/ACCESS.2023.3252366
14. Kumar, J. (2023). Hybrid feature-based machine learning method for phishing URL detection. In 2023 Third International Conference on Secure Cyber Computing and Communication (ICSCCC) (pp. 222–227). IEEE. https://doi.org/10.1109/ICSCCC58608.2023.10176901
15. Lakshmanarao, A., Babu, M. R., & Bala Krishna, M. M. (2021a). Malicious URL detection using NLP, machine learning and FLASK. In 2021 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES) (pp. 1–4). IEEE. https://doi.org/10.1109/ICSES52305.2021.9633889
16. Lakshmanarao, A., Rao, P. S. P., & Krishna, M. M. B. (2021b). Phishing website detection using novel machine learning fusion approach. In 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS) (pp. 1164–1169). IEEE. https://doi.org/10.1109/ICAIS50930.2021.9395810
17. Maneriker, P., Stokes, J. W., Lazo, E. G., Carutasu, D., Tajaddodianfar, F., & Gururajan, A. (2021). URLTran: Improving phishing URL detection using transformers. In MILCOM 2021 – 2021 IEEE Military Communications Conference (MILCOM) (pp. 197–204). IEEE. https://doi.org/10.1109/MILCOM52596.2021.9653028
18. Mathew, J. T., Nair, L. S., Menon, H. P., & A. P., A. (2025). PhishGuard: A hybrid ensemble framework for detecting malicious URLs. In 2025 8th International Conference on Computing Methodologies and Communication (ICCMC) (pp. 1901–1908). IEEE. https://doi.org/10.1109/ICCMC65190.2025.11140768
19. Ogunleye, G. O., Ayogu, B. A., Tolulope, O., & Ogunrekun, P. O. (2024). Comparative analysis of some machine learning techniques for uniform resource locator-based phishing detection. Journal of Engineering, Technology and Innovation, 3(2), 1–12.
20. Ok, M., & Kara, İ. (2021). İnternet sayfası verileri kullanarak kimlik avı web sitelerinin makine öğrenme tekniğiyle tespiti [Detection of phishing web sites using web page data with machine learning technique]. Avrupa Bilim ve Teknoloji Dergisi, (31), 182–187. https://doi.org/10.31590/ejosat.1008335
21. Palavali, D. R., & Pothireddy, S. (2025). Explainable ensemble learning for detecting phishing URLs using lightweight cyber threat intelligence. In 2025 9th International Conference on Electronics, Communication and Aerospace Technology (ICECA) (pp. 207–214). IEEE. https://doi.org/10.1109/ICECA66444.2025.11382884
22. Rashid, J., Mahmood, T., Nisar, M. W., & Nazir, T. (2020). Phishing detection using machine learning technique. In 2020 First International Conference of Smart Systems and Emerging Technologies (SMARTTECH) (pp. 43–46). IEEE. https://doi.org/10.1109/SMART-TECH49988.2020.00026
23. Sangeetha, T. S., & Nair, P. P. (2025). Phishing URL detection with deep learning: Trends, challenges, and future directions. In 2025 International Conference on Power, Instrumentation, Control, and Computing (PICC) (pp. 1–8). IEEE. https://doi.org/10.1109/PICC67314.2025.11291322
24. Siber Güvenlik Kanunu [Cyber Security Law]. (2025). Kanun No. 7545. T.C. Resmî Gazete (No. 32846, 19 March 2025). https://www.resmigazete.gov.tr (erişim tarihi: 01 Kasım 2025).
25. USOM. (2024). Ulusal siber güvenlik istatistikleri [National cyber security statistics]. Ulusal Siber Olaylara Müdahale Merkezi. https://www.usom.gov.tr (erişim tarihi: 11 Kasım 2025).