Machine Learning-Based Hybrid Approach to Increase Success in Detecting XSS Attacks

Öz

Cross-Site Scripting (XSS) attacks are a common type of attack that threatens the security of web applications. Today, many studies are being conducted to detect and prevent XSS attacks. One of the effective ways to prevent XSS attacks at the application layer is to use a Web Application Firewall (WAF). WAFs can generate excessive False Positives (FP) by applying strict rules at high security levels to block all potential attacks. Another disadvantage of using a WAF for web security is that attackers develop new attack vectors outside of these strict rules, which results in the WAF failing to provide the expected response. Machine learning techniques can detect XSS attacks by being trained on large datasets and recognizing patterns. In this study, 14312 benign and 12923 malicious XSS payloads were sent as HTTP requests to the WAF and six different machine learning models. The hybrid approach in this study makes decisions by combining machine learning techniques (including both individual models and ensemble methods) with the WAF's traditional attack signatures. The results of the WAF, machine learning, ensemble methods, and the hybrid approach were evaluated using accuracy, precision, sensitivity, specificity, and F1 score metrics. According to the findings, the most effective machine learning algorithm was the decision tree. According to the hybrid approach, where the WAF and decision tree were applied separately to obtain the joint result, the WAF accuracy value was improved by 30%, sensitivity by 1.27%, precision by 48%, specificity by 76%, and F1 score by 25%.

Anahtar Kelimeler

• XSS attacks
• web application firewall
• machine learning
• ensemble learning
• hybrid method.

XSS Saldırılarını Tespit Etmede Başarıyı Artırmak için Makine Öğrenme Tabanlı Hibrit Yaklaşım

Öz

Cross-Site Scripting (XSS, Siteler Arası Betik Çalıştırma) saldırıları, web uygulamalarının güvenliğini tehdit eden yaygın bir saldırı türüdür. Günümüzde XSS saldırılarının tespit edilmesi ve engellenmesi amacıyla birçok çalışma yapılmaktadır. XSS saldırılarını uygulama katmanında engellemenin etkili yollarından birisi Web Uygulama Güvenlik Duvarı (WAF) kullanmaktır. WAF, olası saldırıların tamamını engellemek amacıyla yüksek güvenlik seviyelerinde katı kurallar uygulayarak çok fazla Yanlış Pozitif (YP) oluşturabilmektedir. Web güvenliği için WAF kullanmanın diğer dezavantajı da saldırganlar bu katı kuralların dışında yeni saldırı vektörleri geliştirmekte ve bunun sonucunda WAF beklenen tepkiyi sağlamada başarısız olmaktadır. Makine öğrenme teknikleri, büyük veri kümeleri üzerinde eğitilerek ve desenleri tanıyarak XSS saldırılarını tespit edebilir. Çalışmada 14312 zararsız, 12923 zararlı XSS yükü HTTP isteği olarak WAF’a ve altı farklı makine öğrenme modeline gönderilmiştir. Çalışmadaki hibrit yaklaşım, makine öğrenme tekniklerini (tekil modeller ve topluluk yöntemler) WAF’ın geleneksel saldırı imzalarıyla birleştirerek karar vermektedir. WAF, makine öğrenmesi, topluluk yöntemi ve hibrit yaklaşımın sonuçları; doğruluk, kesinlik, duyarlılık, özgüllük ve F1 skoru metrikleriyle değerlendirilmiştir. Bulgulara göre, en efektif sonucu veren makine öğrenme algoritması karar ağacı olmuştur. WAF ve karar ağacının ayrı ayrı uygulanarak ortak sonucun elde edildiği hibrit yaklaşıma göre WAF’ın doğruluk değeri %30 oranında, duyarlılık %1,27 oranında, kesinlik %48 oranında, özgüllük %76 oranında ve F1 skoru da %25 oranında iyileştirilmiştir.

Anahtar Kelimeler

• XSS saldırıları
• web uygulama güvenlik duvarı
• makine öğrenmesi
• topluluk öğrenmesi
• hibrit yöntem.

Kaynakça

1. E. Saygılı, “OWASP, WAF ve Bazı Web Güvenlik Zafiyetleri”, içinde Web Uygulama Güvenliği ve Hacking Yöntemleri , 1. Baskı., Dikeyeksen Yayıncılık, 2018, ss. 293-302.
2. OWASP, “What’s changed in the Top 10 for 2021”, 2021. [Çevrimiçi]. Erişim adresi: https://owasp.org/Top10/A00_2021_Introduction/
3. “OWASP Top Ten 2017 | A7:2017-Cross-Site Scripting (XSS) | OWASP Foundation”. Erişim: 23 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://owasp.org/www-project-top-ten/2017/A7_2017-Cross-Site_Scripting_%28XSS%29
4. U. Sarmah, D. K. Bhattacharyya, ve J. K. Kalita, “A survey of detection methods for XSS attacks”, Journal of Network and Computer Applications, c. 118, ss. 113-143, Tem. 2018, doi: 10.1016/J.JNCA.2018.06.004.
5. W3Techs, “Usage statistics of JS as client-side programming language on websites”. Erişim: 08 Temmuz 2025. [Çevrimiçi]. Erişim adresi: https://w3techs.com/technologies/overview/client_side_language
6. G. Code, “Intrusion detection system for .NET based on phpids”, 2007. [Çevrimiçi]. Erişim adresi: https://code.google.com/archive/p/dotnetids/
7. M. Baykara ve R. Daş, “Saldırı tespit ve engelleme araçlarının incelenmesi”, 2019, Dicle Üniversitesi. doi: 10.24012/dumf.449059.
8. R. Barnett, “WAF Virtual Patching Challenge: Securing WebGoat with ModSecurity”, Tem. 2009.

9. S. Applebaum, T. Gaber, ve A. Ahmed, “Signature-based and Machine-Learning-based Web Application Firewalls: A Short Survey”, Procedia Comput Sci, c. 189, ss. 359-367, Oca. 2021, doi: 10.1016/J.PROCS.2021.05.105.
10. Folini, “Including OWASP ModSecurity Core Rule Set”, 2022. [Çevrimiçi]. Erişim adresi: https://www.netnea.com/cms/apache-tutorial-7_including-modsecurity-core-rules/
11. J. S. & Research, “False-Positive Card Declines Push Consumers to Abandon Issuers and Merchants”, 2015. [Çevrimiçi]. Erişim adresi: https://javelinstrategy.com/press-release/false-positive-card-declines-push-consumers-abandon-issuers-and-merchants
12. Aite ve Clearsale, “Balancing False Declines and Fraud Prevention”, 2019. [Çevrimiçi]. Erişim adresi: https://offer.clear.sale/false-declines-ecommerce-fraud-prevention-report-download
13. M. Choraś, R. Kozik, A. Flizikowski, W. Hołubowicz, ve R. Renk, “Cyber threats impacting critical infrastructures”, Studies in Systems, Decision and Control, c. 90, ss. 139-161, 2016, doi: 10.1007/978-3-319-51043-9_7/FIGURES/14.
14. R. Agrawal ve D. Mudzingwa, “A study of methodologies used in intrusion detection and prevention systems (IDPS)”, Conference Proceedings - IEEE SOUTHEASTCON, 2012, doi: 10.1109/SECON.2012.6197080.
15. J. Veeramreddy, V. Prasad, ve K. Prasad, “A Review of Anomaly based Intrusion Detection Systems”, Int J Comput Appl, c. 28, ss. 26-35, Tem. 2011, doi: 10.5120/3399-4730.
16. M. Sevri ve H. Karacan, “Two Stage Deep Learning Based Stacked Ensemble Model for Web Application Security”, KSII Transactions on Internet and Information Systems, c. 16, sy 2, ss. 632-657, Şub. 2022, doi: 10.3837/TIIS.2022.02.014.
17. N. T. Tran, V. H. Nguyen, T. Nguyen-Le, ve K. Nguyen-An, “Improving ModSecurity WAF with Machine Learning Methods”, Communications in Computer and Information Science, c. 1306, ss. 93-107, 2020, doi: 10.1007/978-981-33-4370-2_7.
18. A. Shaheed ve M. H. D. B. Kurdy, “Web Application Firewall Using Machine Learning and Features Engineering”, Security and Communication Networks, c. 2022, 2022, doi: 10.1155/2022/5280158.
19. L. Lei, M. Chen, C. He, ve D. Li, “XSS Detection Technology Based on LSTM-Attention”, 2020 5th International Conference on Control, Robotics and Cybernetics, CRC 2020, ss. 175-180, Tem. 2020, doi: 10.1109/CRC51253.2020.9253484.
20. H. Yan vd., “Cross-site scripting attack detection based on a modified convolution neural network”, Front Comput Neurosci, c. 16, s. 981739, Tem. 2022, doi: 10.3389/FNCOM.2022.981739/BIBTEX.
21. F. Mokbal, W. Dan, ve X. Wang, “Detect Cross-Site Scripting Attacks Using Average Word Embedding and Support Vector Machine”, International Journal of Network Security, c. 24, ss. 20-28, Tem. 2022, doi: 10.6633/IJNS.202201.
22. “GitHub - ajinabraham/OWASP-Xenotix-XSS-Exploit-Framework: OWASP Xenotix XSS Exploit Framework is an advanced Cross Site Scripting (XSS) vulnerability detection and exploitation framework.” Erişim: 24 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://github.com/ajinabraham/OWASP-Xenotix-XSS-Exploit-Framework
23. Coreruleset, “OWASP ModSecurity Core Rule Set”, 2022. [Çevrimiçi]. Erişim adresi: https://coreruleset.org/faq/#paranoialevel
24. J. J. Singh, H. Samuel, ve P. Zavarsky, “Impact of paranoia levels on the effectiveness of the modsecurity web application firewall”, Proceedings - 2018 1st International Conference on Data Intelligence and Security, ICDIS 2018, ss. 141-144, Tem. 2018, doi: 10.1109/ICDIS.2018.00030.
25. G. Betarte, E. Gimenez, R. Martinez, ve A. Pardo, “Improving Web Application Firewalls through Anomaly Detection”, Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018, ss. 779-784, Tem. 2019, doi: 10.1109/ICMLA.2018.00124.
26. “OWASP ruleset concepts · Cloudflare Web Application Firewall (WAF) docs”. Erişim: 23 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://developers.cloudflare.com/waf/managed-rules/reference/owasp-core-ruleset/concepts
27. “Setting up and tuning CIS WAF security | IBM Cloud Docs”. Erişim: 23 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://cloud.ibm.com/docs/cis?topic=cis-waf-configuration
28. S. Sharma, P. Zavarsky, ve S. Butakov, “Machine Learning based Intrusion Detection System for Web-Based Attacks”, Proceedings - 2020 IEEE 6th Intl Conference on Big Data Security on Cloud, BigDataSecurity 2020, 2020 IEEE Intl Conference on High Performance and Smart Computing, HPSC 2020 and 2020 IEEE Intl Conference on Intelligent Data and Security, IDS 2020, ss. 227-230, Tem. 2020, doi: 10.1109/BIGDATASECURITY-HPSC-IDS49724.2020.00048.
29. M. Zhang, B. Xu, S. Bai, S. Lu, ve Z. Lin, “A deep learning method to detect web attacks using a specially designed CNN”, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), c. 10638 LNCS, ss. 828-836, 2017, doi: 10.1007/978-3-319-70139-4_84/COVER.
30. F. A. Mereani ve J. M. Howe, “Detecting Cross-Site Scripting Attacks Using Machine Learning”, Advances in Intelligent Systems and Computing, c. 723, ss. 200-210, 2018, doi: 10.1007/978-3-319-74690-6_20/COVER.
31. M. S. Erçin ve E. Yolaçan, “SQLi ve XSS Saldırı Tespitinde Kullanılan Yeni Bir Özellik Çıkarma Yöntemi”, Journal, c. 8, sy 1, ss. 1-11, 2022.
32. “Artificial neural network based web application firewall for SQL injection | Request PDF”. Erişim: 09 Temmuz 2025. [Çevrimiçi]. Erişim adresi: https://www.researchgate.net/publication/292923440_Artificial_neural_network_based_web_application_firewall_for_SQL_injection
33. S. Rathore, P. K. Sharma, ve J. H. Park, “XSSClassifier: An Efficient XSS Attack Detection Approach Based on Machine Learning Classifier on SNSs”, Journal of Information Processing Systems, c. 13, sy 4, ss. 1014-1028, Tem. 2017, doi: 10.3745/JIPS.03.0079.
34. Y. Fang, Y. Li, L. Liu, ve C. Huang, “DeepXSS: Cross site scripting detection based on deep learning”, ACM International Conference Proceeding Series, ss. 47-51, Tem. 2018, doi: 10.1145/3194452.3194469.
35. J. Song, X. Wang, L. Jin, ve J. You, “Malicious behaviour classification in web logs based on an improved Xgboost algorithm”, International Journal of Web Engineering and Technology, c. 13, sy 4, ss. 334-362, 2018, doi: 10.1504/IJWET.2018.097560.
36. S. Abaimov ve G. Bianchi, “CODDLE: Code-Injection Detection with Deep Learning”, IEEE Access, c. 7, ss. 128617-128627, 2019, doi: 10.1109/ACCESS.2019.2939870.
37. V. Anandkumar, “Malicious-URL Detection using Logistic Regression Technique”, International Journal of Engineering Business Management, c. 9, ss. 108-113, Tem. 2019, doi: 10.31033/ijemr.
38. Ö. Kasım, “Malicious XSS Code Detection with Decision Tree”, Journal of Polytechnic, c. 23, sy 1, ss. 67-72, Tem. 2020, doi: 10.2339/POLITEKNIK.470332.
39. Ş. Bahtiyar, B. Buz, ve B. Gülçiçek, “A Hybrid Machine Learning Model to Detect Reflected XSS Attack”, Balkan Journal of Electrical and Computer Engineering, c. 9, sy 3, ss. 235-241, Tem. 2021, doi: 10.17694/BAJECE.927417.
40. B. Gogoi, T. Ahmed, ve H. Saikia, “Detection of XSS Attacks in Web Applications: A Machine Learning Approach”, International Journal of Innovative Research in Computer Science & Technology, c. 9, ss. 1-10, Tem. 2021, doi: 10.21276/ijircst.2021.9.1.1.
41. I. Odun-Ayo, W. Toro-Abasi, M. Adebiyi, ve O. Alagbe, “An implementation of real-time detection of cross-site scripting attacks on cloud-based web applications using deep learning”, Bulletin of Electrical Engineering and Informatics, c. 10, sy 5, ss. 2442-2453, Tem. 2021, doi: 10.11591/EEI.V10I5.3168.
42. R. Pallam, S. P. Konda, L. Manthripragada, ve R. A. Noone, “Detection of Web Attacks using Ensemble Learning.”, learning, c. 3, sy 4, s. 5, 2021.
43. B. Alagha, “XSS Attack Detection with N-Gram Based Prediction Model”, Eskişehir Türk Dünyası Uygulama ve Araştırma Merkezi Bilişim Dergisi, c. 4, sy 2, ss. 1-9, Tem. 2023, doi: 10.53608/ESTUDAMBILISIM.1233344.
44. F. Younas, A. Raza, N. Thalji, L. Abualigah, R. A. Zitar, ve H. Jia, “An efficient artificial intelligence approach for early detection of cross-site scripting attacks”, Decision Analytics Journal, c. 11, s. 100466, Haz. 2024, doi: 10.1016/J.DAJOUR.2024.100466.
45. “GitHub - digininja/DVWA: Damn Vulnerable Web Application (DVWA)”. Erişim: 23 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://github.com/digininja/DVWA
46. “GitHub - owasp-modsecurity/ModSecurity: ModSecurity is an open source, cross platform web application firewall (WAF) engine for Apache, IIS and Nginx. It has a robust event-based programming language which provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis.” Erişim: 23 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://github.com/owasp-modsecurity/ModSecurity
47. “GitHub - ajinabraham/OWASP-Xenotix-XSS-Exploit-Framework: OWASP Xenotix XSS Exploit Framework is an advanced Cross Site Scripting (XSS) vulnerability detection and exploitation framework.” Erişim: 23 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://github.com/ajinabraham/OWASP-Xenotix-XSS-Exploit-Framework
48. “GitHub - Mebus/cupp: Common User Passwords Profiler (CUPP)”. Erişim: 23 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://github.com/Mebus/cupp
49. “GitHub - molu8bits/modsecurity-parser: modsecurity audit log analyser and parser”. Erişim: 23 Eylül 2025. [Çevrimiçi]. Erişim adresi: https://github.com/molu8bits/modsecurity-parser
50. M. Sokolova ve G. Lapalme, “A systematic analysis of performance measures for classification tasks”, Inf Process Manag, c. 45, sy 4, ss. 427-437, Tem. 2009, doi: 10.1016/J.IPM.2009.03.002.