Derin Öğrenme Tekniği Kullanarak Anomali Tabanlı Web Uygulama Güvenlik Duvarı

Öz

Anomali tespiti, farklı sektörlerde ve uygulama alanlarında araştırılmaya devam etmektedir. Anomali tespitindeki temel zorluk, benzersiz özelliklere ve yeni değerlere sahip bir girdi ile karşılaşılması durumunda normallerden aykırı değerleri belirlemektir. Araştırmalar, bu görevi yerine getirmek için Makine Öğrenmesi ve Derin Öğrenme tekniklerini kullanmaya odaklanmaktadır. Internet dünyasında, bir web sitesi isteğinin kötü niyetli veya sadece normal bir istek olup olmadığını belirlemek istediğimizde yine benzer bir sınıflandırma problemiyle karşı karşıya kalmaktayız. Web Uygulama Güvenlik Duvarı (WAF) sistemleri kötü niyetli faaliyetlere ve isteklere karşı, kural tabanlı ve son yıllarda kullanılan anomali tabanlı çözüm kullanarak koruma sağlar. Bu tür çözümler bir noktaya kadar güvenlik sağlar ve kullanılan teknikler, arka uç sistemlerini savunmasız bırakan hatalı sonuçlar üretmektedirler. Bu çalışmanın odak noktası, karakter sıralaması tabanlı bir LSTM (tekli ve yığılmış olmak üzere) yapısı kullanılarak bir WAF sistemi oluşturmak ve derin öğrenme modelinin optimum sonuç üretmesi için hiper parametrelerin hangi değerleri alması gerektiğini ortaya koymaktır. Semi-supervised öğrenme yaklaşımı için PayloadAllTheThings verisetinde yer alan gerçek saldırı verilerinin yanı sıra HTTP CSIC 2010 verisetinde yer alan ve normal olarak etiketlenen veriler hem modelin öğrenmesi sırasında hem de test edilmesi adımında kullanılmıştır. Önerilen tekniğin başarı oranının analizini için F1 skor değeri baz alınmıştır. Yapılan analizler ve deneyler sonucunda elde edilen derin öğrenme modelinin F1 başarı oranının yüksek olduğu ve saldırıları tespit etme ve sınıflandırma noktasında da başarı elde edildiği gösterilmiştir. Anahtar Kelimeler: 

Anahtar Kelimeler

• Derin öğrenme
• web uygulama güvenlik duvarı
• makine öğrenmesi
• sinir ağları
• web saldırıları
• anomali tespiti
• HTTP protokolü

Web Application Firewall Based on Anomaly Detection using Deep Learning

Öz

Anomaly detection has been researched in different areas and application domains. The main difficulty is to identify the outliers from the normals in case of encountering an input that has unique features and new values. In order to accomplish this task, the research focusses on using Machine Learning and Deep Learning techniques. In the world of the Internet, we are facing a similar problem to identify whether a website request contains malicious activity or just a normal request. Web Application Firewall (WAF) systems provide such protection against malicious requests using a rule based approach. In recent years, anomaly based solutions have been integrated in addition to rule based systems. Still, such solutions can only provide security up to a point and such techniques can generate false-positive results that leave the backend systems vulnerable and most of the time rules based protection can be bypassed with simple tricks (eg. encoding, obfuscation). The main focus of the research is WAF systems that employ single and stacked LSTM layers which are based on character sequences of user supplied data and revealing hyper-parameter values for optimal results. A semi-supervised approach is used and trained with PayloadAllTheThings dataset containing real attack payloads and only normal payloads of HTTP Dataset CSIC 2010 are used. The success rate of the technique - whether the user input is identified as malicious or normal - is measured using F1 scores. The proposed model demonstrated high F1 scores and success in terms of detection and classification of the attacks. 

Anahtar Kelimeler

• Deep learning
• LSTM
• web application firewall
• machine learning
• neural networks
• web attacks
• anomaly detection
• HTTP protocol

Kaynakça

1. A. Graves (2012), Supervised Sequence Labelling with Recurrent Neural Networks. Springer, 2012th edition. google scholar
2. A. Juvonen, T. Sipola & T. Hâmâlâinen (2015), Online anomaly detection using dimensionality reduction techniques for http log analysis, Computer Networks, vol. 91, pp. 46-56. google scholar
3. A. Moradi Vartouni, S. Mehralian, M. Teshnehlab & S. Sedighian Kashi (2019). Auto-Encoder LSTM Methods for Anomaly-Based Web Application Firewall. International Journal of Information and Communication Technology. 11. 49-56. google scholar
4. A. Oza, K. Ross, R. Low & M. Stamp (2014), Http attack detection using n-gram analysis, Computers & Security, vol. 45. google scholar
5. A. Shilton, S. Rajasegarar, M. Palaniswami (2013), Combined multiclass classification and anomaly detection for large-scale wireless sensor networks, IEEE Eighth International Conference on Intelligent Sensors, Sensor Networks and Information Processing, Melbourne, Australia, pp. 491-496. google scholar
6. A. Singer & H. Wu (2011), Orientability and diffusion maps, Applied and Computational Harmonic Analysis, vol. 31, no. 1, pp. 44-58. google scholar
7. A. Singh (2017), Anomaly Detection for Temporal Data using Long Short-Term Memory (LSTM), Retrieved from http://urn.kb.se/ resolve?urn=urn:nbn:se:kth:diva-215723 google scholar
8. Acunetix Path traversal (2021), Retrieved from: https://www.acunetix.com/websitesecurity/directory-traversal/ google scholar

9. B. Mirkin (2005), Clustering For Data Mining: A Data Recovery Approacz. Chapman & Hall/CRC. google scholar
10. C. Alonso, A. Guzman, M. Beltran, R. Bordon (2009), Ldap Injection Techniques, Wireless Sensor Network, 1, 233-244, doi:10.4236/wsn.2009.14030 google scholar
11. C. Torrano-Gimnez, A. Prez-Villegas, & G. Alvarez (2010), “ The HTTP dataset CSIC 2010,” ed: Instituto deSeguridad de la Informacion (ISI). google scholar
12. Computer Fraud & Security (2020). Verizon:data breach investigations report, vol. 2020, no. 6, p. 4, 2020, ISSN: 1361-3723. google scholar
13. CWE (2006), Improper neutralization of special elements used in an os command, Retrieved from: https://cwe.mitre.org/data/definitions/78.html google scholar
14. D. Ariu, R. Tronci, & G. Giacinto (2011), Hmmpayl: An intrusion detection system based on hidden markov models, Comput. Secur., vol. 30, no. 4, pp.221-241. google scholar
15. D. Durstewitz (2017), Clustering and density estimation, pp. 85-103. google scholar
16. D. Jurafsky & J. H. Martin (2020), Speech and Language Processing. Prentice Hall. google scholar
17. D. Palka & M. Zachara (2011), Learning web application firewall - benefits and caveats, pp. 295-308. google scholar
18. F. Gers, J. Schmidhuber & F. Cummins (1999), Learning to forget: Continual prediction with LSTM, Ninth International Conference on Artificial Neural Networks ICANN 99. (Conf. Publ. No. 470), vol. 2, 850-855 vol.2. google scholar
19. F. Valeur, G. Vigna, C. Kruegel & R.A. Kemmerer (2004), Comprehensive approach tointrusion detection alert correlation, Dependable and Secure Computing, IEEE Transactions on, vol. 1, pp. 146-169. google scholar Fortinet attack vector (2021), What is an Attack Vector, Retrieved from https://www.fortinet.com/resources/cyberglossary/attack-vector. google scholar
20. G. Betarte, E.Gimenez, R. Martmez & Â. Pardo (2018). Improving Web Application Firewalls through Anomaly Detection. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), 779-784. google scholar
21. H. Xu, W. Chen, N. Zhao, Z. Li, J. Bu, Z. Li, Y. Liu, Y. Zhao, D. Pei, Y. Feng, J. Chen, Z. Wang & H. Qiao (2018), Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications, Proceedings of the 2018 World Wide Web Conference on World Wide Web. google scholar
22. I. Goodfellow, Y. Bengio & A. Courville. (2016), Deep Learning, MIT Press, Refrieved from http://www.deeplearningbook.org. google scholar
23. I. Kotenko, O. Lauta, K. Kribel & I. Saenko (2021). LSTM Neural Networks for Detecting Anomalies Caused by Web Application Cyber Attacks. 10.3233/FAIA210014. google scholar
24. J. Liang, W. Zhao & W. Ye. (2017). Anomaly-Based Web Attack Detection: A Deep Learning Approach. 80-85. 10.1145/3171592.3171594. google scholar
25. J.Hodges, R.Morgan (2002), Ldapv3, Retrieved from: https://datatracker.ietf.org/doc/html/rfc3377 google scholar
26. M. Arora & V. Kansal (2019), Character level embedding with deep convolutional neural network for text normalization of unstructured data for twitter sentiment analysis, Social Network Analysis and Mining, vol. 9. google scholar
27. M. E. Hannes Holm (2013), Estimates on the effectiveness of web application firewalls against targeted attacks, pp. 250-265. google scholar
28. M. Markou & S. Singh (2003a), Novelty detection: A review—part 1: Statistical approaches, Signal Processing, vol. 83, no. 12, pp. 2481-2497, ISSN: 0165-1684. google scholar
29. M. Markou & S. Singh (2003b), Novelty detection: A review—part 2: Neural networkbased approaches, Signal Processing, vol. 83, no. 12, pp. 2499-2521,ISSN: 0165-1684. google scholar
30. M. Nadeem, O. Marshall, S. Singh, X. Fang & X. Yuan (2016), Semi-supervised deep neural network for network intrusion detection, KSU Conference On Cybersecurıty Educatıon, Research And Practıce. google scholar
31. M. White, M. Tufano, C. Vendome & D. Poshyvanyk (2016), Deep learning code fragments for code clone detection,31st IEEE/ACM International Conferenceon Automated Software Engineering (ASE), pp. 87-98. google scholar
32. M.Wahl, T.Howes & S.Kille (1997), Ldapv,Retrieved from: https://datatracker.ietf.org/doc/html/RFC2251 google scholar
33. N. Ben-Asher & C. Gonzalez (2015), Training for the unknown: The role of feedback and similarity in detecting zero-day attacks, Procedia Manufacturing, vol. 3, pp. 1088-1095, 2015, 6th International Conference on Applied Human Factors and Ergonomics and the Affiliated Conferences. google scholar
34. N. Galbreath (2012), Libinjection. Retrieved from https://github.com/client9/libinjection (visited on 2012). google scholar
35. N. Görnitz, M. Kloft, M. Rieck, & U. Brefeld (2013), Toward supervised anomaly detection, Journal of Artificial Intelligence Research, vol. 46, pp. 235-262. google scholar
36. N. Montes, G. Betarte, Â. Pardo & R. Martmez (2018). Web Application Attacks Detection Using Machine Learning Techniques. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), 1065-1072. google scholar
37. N. Montes, G. Betarte, Â. Pardo & R. Martmez (2021). Web ApplicationAttacks Detection UsingDeep Learning. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 25th Iberoamerican Congress, CIARP 2021, 227-236. google scholar
38. N. Oliveira, I. Praça, E. Maia and O. Sousa. (2021). Intelligent Cyber Attack Detection and Classification for Network-Based Intrusion Detection Systems. Applied Sciences. 11. 1674. 10.3390/app11041674. google scholar
39. OWASP Cross site scripting (XSS) (2021). Retrieved from https://owasp.org/wwwcommunity/attacks/xss/. google scholar
40. OWASP Ldap injection (2021), Retrieved from: https://cheatsheetseries.owasp.org/cheatsheets/LDAP_Injection_Prevention_Cheat_Sheet.html google scholar
41. OWASP Php code injection (2021). Retrieved from https://owasp.org/www-community/attacks/Code_Injection . google scholar
42. OWASP Server-side template injection (2021), Retrieved from: https://owasp.org/www-project-web-security-testing-guide/ stable/4-Web_Application_Security_Testing/07-Input_Validation_Testing/18-Testing_for_Server-side_Template_Injection google scholar
43. OWASP Sql injection (2021). Retrieved from https://owasp.org/www-community/attacks/SQL_Injection. google scholar
44. Owasp TOP10 web application security risk (2021). Retrieved from https://owasp.org/www-project-top-ten/. google scholar
45. OWASP Web application firewall (2021). Retrieved from https://owasp.org/www-community/Web_Application_Firewall. google scholar
46. Payloads all the things (2021). Retrieved from https://github.com/swisskyrepo/PayloadsAllTheThings. google scholar
47. Portswigger Cross site scripting (2021), Retrieved from: https://portswigger.net/web-security/cross-site-scripting/ google scholar
48. Portswigger Path traversal (2021), Retrieved from: https://portswigger.net/web-security/file-path-traversal google scholar
49. Portswigger Sql injection cheat sheet (2021), Retrieved from: https://portswigger.net/web-security/sql-injection/cheat-sheet google scholar
50. Q. Zhu, Z. He, T. Zhang & W. Cui (2020), Improving classification performance of softmax loss function based on scalable batch-normalization, Applied Sciences, vol. 10, no. 8. google scholar
51. R. Cahuantzi & X. A. Chen & S. Güttel (2021), A comparison of LSTM and GRU networks for learning symbolic sequences. ArXiv, abs/2107.02248.. google scholar
52. R. Chalapathy & S. Chawla (2019), Deep learning for anomaly detection: A survey. arXiv: 1901.03407. google scholar
53. R. M. Cooke (1991), Experts in Uncertainty: Opinion and Subjective Probability in Science, .New York:Oxford University Press. google scholar
54. S Wold, K. Esbensen & P. Geladi (1987), Principal component analysis, Chemometrics and Intelligent Laboratory Systems, vol. 2, no. 1, pp. 37-52, ISSN: 0169-7439. google scholar
55. S. Erfani, M. Baktashmotlagh, M. Moshtaghi, V. Nguyen, C. Leckie, J. Bailey, K. Ramamohanarao (2017), From shared subspaces to shared landmarks: A robust multi-source classification approach, Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31. google scholar
56. S. Hochreiter & J. Schmidhuber (1997), Long Short-Term Memory, Neural Computation,vol. 9, no. 8, pp. 1735-1780. google scholar
57. S. Hochreiter & J. Schmidhuber (1997), Long short-term memory, Neural Comput.,vol. 9, no. 8, pp. 1735-1780. google scholar
58. S. Hochreiter (1991), Untersuchungen zu dynamischen neuronalen netzen. google scholar
59. S. Hochreiter (1998), The vanishing gradient problem during learning recurrent neural nets and problem solutions, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 6, pp. 107-116. google scholar
60. S. S. Kashi (2019), Leveraging deep neural networks for anomaly-based web application firewall, English, IET Information Security, vol. 13, 352-361(9), ISSN: 1751-8709. google scholar S. Young (2021), Designing a DMZ, SANS Institute. google scholar
61. SANS Exploiting XXE vulnerabilities (2017), Retrieved from: https://www.sans.org/blog/exploiting-xxe-vulnerabilities-in-iis-net/ google scholar
62. Statista (2021), Annual number of data breaches and exposed records in the United States from 2005 to 2020, Retrieved from https://www.statista.com/ statistics/273550/data-breaches-recorded-in-the-united-states-by-number-of-breaches-and-records-exposed/. google scholar
63. T. Alma & M. L. Das (2020), Web application attack detection using deep learning, arXiv: 2011.03181. google scholar
64. T. Liu, U. Qi, L. Shi, J. Yan (2019), Locate-then-detect: Real-time web attack detection via attention-based deep neural networks, Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, International Joint Conferences on Artificial Intelligence Organization, pp. 4725-4731. google scholar
65. T. Mikolov, I. Sutskever, K. Chen, G. Corrado & J. Dean (2013), Distributed representations of words and phrases and their compositionality, pp. 3111-3119. google scholar
66. T. Yu & H. Zhu (2020), Hyper-parameter optimization: A review of algorithms and applications, ArXiv, vol. abs/2003.05689. google scholar
67. V . Jumutc & J. A. Suykens (2014), Multi-class supervised novelty detection, IEEE Transactionson Pattern Analysis and Machine Intelligence, vol. 36, no. 12, pp. 2510- 2523. google scholar
68. WASC (2010), Web application security consortium, Retrieved from: http://projects.webappsec.org/f/WASC-TC-v2_0.pdf google scholar
69. WASC Os command injection (2009), Retrieved from: http://projects.webappsec.org/w/page/13246950/OS%5C%20Commanding google scholar
70. Y . Dong, Y. Zhang, H. Ma, Q. Wu, Q. Liu, K. Wang & W. Wang (2018), An adaptive system for detecting malicious queries in web attacks, Science China Information Sciences, vol. 61, no. 3. google scholar
71. Y . Hu, A.E. Huber, J. Anumula, & S. Liu (1998), Overcoming the vanishing gradient problem in plain recurrent networks, Retrieved from https:// openreview.net/forum?id=Hyp3i2xRb google scholar
72. Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019). Roberta: A robustly optimized bert pretraining approach, ICLR 2020 Conference. google scholar
73. Y . Pan, F. Sun, Z. Teng, J. White, C. Schmidt, J. Staples, L. Krause (2019), Detecting web attacks with end-to-end deep learning, Journal of Internet Services and Applications, vol. 10. google scholar
74. Y . Yu, X. Si, C. Hu and J. Zhang (2019), A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures, Neural Computation, vol. 31, no. 7, pp. 1235-1270. google scholar Z. Li, D. Zou, S. Xu, X. Ou, H. Jin, S. Wang, Z. Deng, Y. Zhong (2018), Vuldeepec google scholar