A Comparative Study of Explainable (XAI) Deep and Ensemble Learning Models for a Web Application Firewall Using the FWAF Dataset

Abstract

Bridging the gap between predictive accuracy and interpretability in web application security, this study benchmarks machine learning (ML) and deep learning (DL) models for intrusion detection using the publicly available FWAF dataset. We evaluate six widely used classifiers—Logistic Regression, Decision Tree, Random Forest, XGBoost, 1D-CNN, and LSTM—under a highly imbalanced setting (benign-dominant traffic). All models are trained on identical data using a unified pipeline with stratified train/validation/test splits (64%/16%/20%) and class-weight balancing to ensure a fair comparison. Models are assessed using accuracy, precision, recall, F1-score, and imbalance-aware measures (Macro-F1, PRAUC, and MCC). To improve transparency, we integrate Explainable Artificial Intelligence (XAI) techniques—specifically SHAP (SHapley Additive ExPlanations) and LIME (Local Interpretable Model-Agnostic Explanations)—to quantify feature contributions and highlight decision patterns relevant to WAF operation. An ablation study confirms that attack-indicator features (is_xss, is_lfi, is_oci, is_sqli) are derived independently from labels via regex matching, with their removal causing only marginal performance degradation (1–2% Macro-F1). Experimental results show that Random Forest achieves the strongest overall performance (99.3% accuracy, 0.949 Macro-F1, 0.935 PR-AUC), while the feature-based LSTM provides comparable results (99.2% accuracy, 0.937 Macro-F1, 0.928 PR-AUC). Ultimately, the study emphasizes the value of balancing performance with interpretability, empowering Security Operations Centers (SOC) to validate automated decisions and foster trustworthy AI-driven web application firewalls

Keywords

• Deep Learning
• Explainable Artificial Intelligence(XAI)
• Shap
• Lime
• Web Application Firewall

FWAF Veri Seti Kullanılarak Bir Web Uygulama Güvenlik Duvarı için Açıklanabilir Yapay Zeka (XAI) Destekli Derin ve Topluluk Öğrenme Modellerinin Karşılaştırmalı İncelemesi

Abstract

Web uygulaması güvenliğinde tahmin doğruluğu ve yorumlanabilirlik arasındaki boşluğu dolduran bu çalışma, açıklanabilir yapay zeka (XAI) teknikleriyle geliştirilmiş, saldırı tespiti için makine öğrenimi (ML) ve derin öğrenme (DL) modellerinin uygulanmasını araştırmaktadır. Kamuya açık FWAF veri seti, yaygın olarak kullanılan altı sınıflandırma modelini karşılaştırmak için kullanılmıştır: Lojistik Regresyon, Karar Ağacı, Rastgele Orman, XGBoost, 1D-CNN ve LSTM. Bu modeller, doğruluk, kesinlik, geri çağırma ve F1 puanı gibi standart ölçütler kullanılarak sistematik olarak değerlendirilmiştir. Model şeffaflığının kritik önem taşıyan sorununu ele almak için, analiz süreci SHAP (SHapley Additive exPlanations) ve LIME (Local Interpretable Model-Agnostic Explanations) ile güçlendirilmiş ve özelliklerin katkıları ve model karar kalıpları hakkında ayrıntılı bilgiler sağlanmıştır. Sonuçlar, XAI yöntemlerinin yüksek performanslı DL ve topluluk algoritmalarıyla entegre edilmesinin, daha şeffaf ve güvenilir saldırı tespit sistemlerinin geliştirilmesini desteklediğini göstermektedir. Sonuç olarak, çalışma, güvenilir AI tabanlı web uygulama güvenlik duvarlarının geliştirilmesi için performans ile yorumlanabilirlik arasında denge kurmanın önemini vurgulamaktadır.

Keywords

• Derin öğrenme
• açıklanabilir yapay zekâ
• SHAP
• LIME
• web uygulama güvenlik duvarı

References

1. [1] I. H. Sarker, "Machine Learning for Intrusion Detection: A Comparative Analysis," Computers & Security, vol. 108, p. 102433, 2022.
2. [2] C. Yin, Y. Zhu, J. Fei, and X. He, "A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks," IEEE Access, vol. 5, pp. 21954-21961, 2017.
3. [3] G. Kim, S. Lee, and S. Kim, "A Novel Hybrid Intrusion Detection Method Integrating Anomaly Detection with Misuse Detection," Expert Systems with Applications, vol. 41, no. 4, pp. 1690-1700, 2014.
4. [4] S. J. Almheiri, A. A. Shah, S. Abbas, M. Ahmad, and M. A. Khan, "Smart sustainable cyber security: modelling an interpretable and transparent threat detection with explainable artificial intelligence," Discover Sustainability, vol. 6, no. 1, p. 442, 2025/05/24 2025.
5. [5] N. Faizan, "FWAF: Machine Learning-driven Web Application Firewall Dataset," 2020.
6. [6] A. Sharma, S. Rani, and M. Shabaz, "A comprehensive review of explainable AI in cybersecurity: Decoding the black box," ICT Express, vol. 11, no. 6, pp. 1200-1219, 2025/12/01/ 2025.
7. [7] P. Hermosilla, S. Berríos, and H. Allende-Cid, "Explainable AI for Forensic Analysis: A Comparative Study of SHAP and LIME in Intrusion Detection Models," Applied Sciences, vol. 15, no. 13, p. 7329, 2025.
8. [8] V. Z. Mohale and I. C. Obagbuwa, "A systematic review on the integration of explainable artificial intelligence in intrusion detection systems to enhancing transparency and interpretability in cybersecurity," (in English), Frontiers in Artificial Intelligence, Systematic Review vol. Volume 8 - 2025, 2025-January-28 2025.

9. [9] M. Pawlicki, A. Pawlicka, R. Kozik, and M. Choraś, "The survey on the dual nature of xAI challenges in intrusion detection and their potential for AI innovation," Artificial Intelligence Review, vol. 57, no. 12, p. 330, 2024/10/16 2024.
10. [10] B. Sharma, L. Sharma, C. Lal, and S. Roy, "Explainable artificial intelligence for intrusion detection in IoT networks: A deep learning based approach," Expert Systems with Applications, vol. 238, p. 121751, 2024/03/15/ 2024.
11. [11] D. Gaspar, P. Silva, and C. Silva, "Explainable AI for Intrusion Detection Systems: LIME and SHAP Applicability on Multi-Layer Perceptron," IEEE Access, vol. 12, pp. 30164-30175, 2024.
12. [12] Rohith, R. Athief, N. Kishore, and R. N. Paranthaman, "Web Application Firewall Using Machine Learning," in 2024 International Conference on Advances in Computing, Communication and Applied Informatics (ACCAI), 2024, pp. 1-7.
13. [13] Y. E. Seyyar, A. G. Yavuz, and H. M. Ünver, "Detection of Web Attacks Using the BERT Model," in 2022 30th Signal Processing and Communications Applications Conference (SIU), 2022, pp. 1-4.
14. [14] J. Yang et al., "LLM-AE-MP: Web Attack Detection Using a Large Language Model with Autoencoder and Multilayer Perceptron," Expert Systems with Applications, vol. 274, p. 126982, 2025/05/15/ 2025.
15. [15] B. IŞiker and S. İ, "Machine Learning Based Web Application Firewall," in 2021 2nd International Informatics and Software Engineering Conference (IISEC), 2021, pp. 1-6.
16. [16] S. Bayrak, A. Karaca, F. Toson, M. E. Tayfur, and S. Yavas, "Unified AI Models for Network Security on Edge Devices," International Journal of Computational Intelligence Systems, vol. 18, no. 1, p. 266, 2025/10/20 2025.
17. [17] B. R. Dawadi, B. Adhikari, and D. K. Srivastava, "Deep Learning Technique-Enabled Web Application Firewall for the Detection of Web Attacks," (in English), Sensors, Article vol. 23, no. 4, p. 16, Feb 2023, Art. no. 2073.
18. [18] Y. E. Seyyar, A. G. Yavuz, and H. M. Ünver, "An Attack Detection Framework Based on BERT and Deep Learning," IEEE Access, vol. 10, pp. 68633-68644, 2022.
19. [19] S. Toprak and A. G. Yavuz, "Web Application Firewall Based on Anomaly Detection using Deep Learning," (in en), Acta Infologica, vol. 6, no. 2, pp. 219-244, December 2022.
20. [20] T. Arumuga Maria Devi and B. Akshay Kumar, "Machine Learning with Logistic Regression for Web Application Firewall," INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY (IJERT), vol. 10, no. 04, 2022.
21. [21] N. Stevanović, B. Todorović, and V. Todorović, "Web attack detection based on traps," Applied Intelligence, vol. 52, no. 11, pp. 12397-12421, 2022/09/01 2022.
22. [22] D. Wang, D. Xu, D. Yu, and G. Xu, "Time-aware sequence model for next-item recommendation," Applied Intelligence, vol. 51, no. 2, pp. 906- 920, 2021/02/01 2021.
23. [23] S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
24. [24] Y. Sun, L. Zhang, M. Yao, and J. Zhang, "Neural network models and shapley additive explanations for a beam-ring structure," Chaos, Solitons & Fractals, vol. 185, p. 115114, 2024/08/01/ 2024.
25. [25] M. T. Ribeiro, S. Singh, and C. Guestrin, ""Why Should I Trust You?": Explaining the Predictions of Any Classifier," presented at the Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California, USA, 2016. Available: https://doi.org/10.1145/2939672.2939778
26. [26] M. R. Zafar and N. Khan, "Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability," Machine Learning and Knowledge Extraction, vol. 3, no. 3, pp. 525-541, 2021.
27. [27] S. Lundberg and S.-I. Lee, A Unified Approach to Interpreting Model Predictions. 2017.