Derin Öğrenme Tekniği Kullanarak Anomali Tabanlı Web Uygulama Güvenlik Duvarı

Year 2022, , 219 - 244, 31.12.2022

Sezer Toprak , Ali Gökhan Yavuz

https://doi.org/10.26650/acin.1039042

Abstract

Anomali tespiti, farklı sektörlerde ve uygulama alanlarında araştırılmaya devam etmektedir. Anomali tespitindeki temel zorluk, benzersiz özelliklere ve yeni değerlere sahip bir girdi ile karşılaşılması durumunda normallerden aykırı değerleri belirlemektir. Araştırmalar, bu görevi yerine getirmek için Makine Öğrenmesi ve Derin Öğrenme tekniklerini kullanmaya odaklanmaktadır. Internet dünyasında, bir web sitesi isteğinin kötü niyetli veya sadece normal bir istek olup olmadığını belirlemek istediğimizde yine benzer bir sınıflandırma problemiyle karşı karşıya kalmaktayız. Web Uygulama Güvenlik Duvarı (WAF) sistemleri kötü niyetli faaliyetlere ve isteklere karşı, kural tabanlı ve son yıllarda kullanılan anomali tabanlı çözüm kullanarak koruma sağlar. Bu tür çözümler bir noktaya kadar güvenlik sağlar ve kullanılan teknikler, arka uç sistemlerini savunmasız bırakan hatalı sonuçlar üretmektedirler. Bu çalışmanın odak noktası, karakter sıralaması tabanlı bir LSTM (tekli ve yığılmış olmak üzere) yapısı kullanılarak bir WAF sistemi oluşturmak ve derin öğrenme modelinin optimum sonuç üretmesi için hiper parametrelerin hangi değerleri alması gerektiğini ortaya koymaktır. Semi-supervised öğrenme yaklaşımı için PayloadAllTheThings verisetinde yer alan gerçek saldırı verilerinin yanı sıra HTTP CSIC 2010 verisetinde yer alan ve normal olarak etiketlenen veriler hem modelin öğrenmesi sırasında hem de test edilmesi adımında kullanılmıştır. Önerilen tekniğin başarı oranının analizini için F1 skor değeri baz alınmıştır. Yapılan analizler ve deneyler sonucunda elde edilen derin öğrenme modelinin F1 başarı oranının yüksek olduğu ve saldırıları tespit etme ve sınıflandırma noktasında da başarı elde edildiği gösterilmiştir. Anahtar Kelimeler: 

Keywords

Derin öğrenme, web uygulama güvenlik duvarı, makine öğrenmesi, sinir ağları, web saldırıları, anomali tespiti, HTTP protokolü

Web Application Firewall Based on Anomaly Detection using Deep Learning

Abstract

Anomaly detection has been researched in different areas and application domains. The main difficulty is to identify the outliers from the normals in case of encountering an input that has unique features and new values. In order to accomplish this task, the research focusses on using Machine Learning and Deep Learning techniques. In the world of the Internet, we are facing a similar problem to identify whether a website request contains malicious activity or just a normal request. Web Application Firewall (WAF) systems provide such protection against malicious requests using a rule based approach. In recent years, anomaly based solutions have been integrated in addition to rule based systems. Still, such solutions can only provide security up to a point and such techniques can generate false-positive results that leave the backend systems vulnerable and most of the time rules based protection can be bypassed with simple tricks (eg. encoding, obfuscation). The main focus of the research is WAF systems that employ single and stacked LSTM layers which are based on character sequences of user supplied data and revealing hyper-parameter values for optimal results. A semi-supervised approach is used and trained with PayloadAllTheThings dataset containing real attack payloads and only normal payloads of HTTP Dataset CSIC 2010 are used. The success rate of the technique - whether the user input is identified as malicious or normal - is measured using F1 scores. The proposed model demonstrated high F1 scores and success in terms of detection and classification of the attacks. 

Keywords

Deep learning, LSTM, web application firewall, machine learning, neural networks, web attacks, anomaly detection, HTTP protocol

References

• A. Graves (2012), Supervised Sequence Labelling with Recurrent Neural Networks. Springer, 2012th edition. google scholar
• A. Juvonen, T. Sipola & T. Hâmâlâinen (2015), Online anomaly detection using dimensionality reduction techniques for http log analysis, Computer Networks, vol. 91, pp. 46-56. google scholar
• A. Moradi Vartouni, S. Mehralian, M. Teshnehlab & S. Sedighian Kashi (2019). Auto-Encoder LSTM Methods for Anomaly-Based Web Application Firewall. International Journal of Information and Communication Technology. 11. 49-56. google scholar
• A. Oza, K. Ross, R. Low & M. Stamp (2014), Http attack detection using n-gram analysis, Computers & Security, vol. 45. google scholar
• A. Shilton, S. Rajasegarar, M. Palaniswami (2013), Combined multiclass classification and anomaly detection for large-scale wireless sensor networks, IEEE Eighth International Conference on Intelligent Sensors, Sensor Networks and Information Processing, Melbourne, Australia, pp. 491-496. google scholar
• A. Singer & H. Wu (2011), Orientability and diffusion maps, Applied and Computational Harmonic Analysis, vol. 31, no. 1, pp. 44-58. google scholar
• A. Singh (2017), Anomaly Detection for Temporal Data using Long Short-Term Memory (LSTM), Retrieved from http://urn.kb.se/ resolve?urn=urn:nbn:se:kth:diva-215723 google scholar
• Acunetix Path traversal (2021), Retrieved from: https://www.acunetix.com/websitesecurity/directory-traversal/ google scholar
• B. Mirkin (2005), Clustering For Data Mining: A Data Recovery Approacz. Chapman & Hall/CRC. google scholar
• C. Alonso, A. Guzman, M. Beltran, R. Bordon (2009), Ldap Injection Techniques, Wireless Sensor Network, 1, 233-244, doi:10.4236/wsn.2009.14030 google scholar
• C. Torrano-Gimnez, A. Prez-Villegas, & G. Alvarez (2010), “ The HTTP dataset CSIC 2010,” ed: Instituto deSeguridad de la Informacion (ISI). google scholar
• Computer Fraud & Security (2020). Verizon:data breach investigations report, vol. 2020, no. 6, p. 4, 2020, ISSN: 1361-3723. google scholar
• CWE (2006), Improper neutralization of special elements used in an os command, Retrieved from: https://cwe.mitre.org/data/definitions/78.html google scholar
• D. Ariu, R. Tronci, & G. Giacinto (2011), Hmmpayl: An intrusion detection system based on hidden markov models, Comput. Secur., vol. 30, no. 4, pp.221-241. google scholar
• D. Durstewitz (2017), Clustering and density estimation, pp. 85-103. google scholar
• D. Jurafsky & J. H. Martin (2020), Speech and Language Processing. Prentice Hall. google scholar
• D. Palka & M. Zachara (2011), Learning web application firewall - benefits and caveats, pp. 295-308. google scholar
• F. Gers, J. Schmidhuber & F. Cummins (1999), Learning to forget: Continual prediction with LSTM, Ninth International Conference on Artificial Neural Networks ICANN 99. (Conf. Publ. No. 470), vol. 2, 850-855 vol.2. google scholar
• F. Valeur, G. Vigna, C. Kruegel & R.A. Kemmerer (2004), Comprehensive approach tointrusion detection alert correlation, Dependable and Secure Computing, IEEE Transactions on, vol. 1, pp. 146-169. google scholar Fortinet attack vector (2021), What is an Attack Vector, Retrieved from https://www.fortinet.com/resources/cyberglossary/attack-vector. google scholar
• G. Betarte, E.Gimenez, R. Martmez & Â. Pardo (2018). Improving Web Application Firewalls through Anomaly Detection. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), 779-784. google scholar
• H. Xu, W. Chen, N. Zhao, Z. Li, J. Bu, Z. Li, Y. Liu, Y. Zhao, D. Pei, Y. Feng, J. Chen, Z. Wang & H. Qiao (2018), Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications, Proceedings of the 2018 World Wide Web Conference on World Wide Web. google scholar
• I. Goodfellow, Y. Bengio & A. Courville. (2016), Deep Learning, MIT Press, Refrieved from http://www.deeplearningbook.org. google scholar
• I. Kotenko, O. Lauta, K. Kribel & I. Saenko (2021). LSTM Neural Networks for Detecting Anomalies Caused by Web Application Cyber Attacks. 10.3233/FAIA210014. google scholar
• J. Liang, W. Zhao & W. Ye. (2017). Anomaly-Based Web Attack Detection: A Deep Learning Approach. 80-85. 10.1145/3171592.3171594. google scholar
• J.Hodges, R.Morgan (2002), Ldapv3, Retrieved from: https://datatracker.ietf.org/doc/html/rfc3377 google scholar
• M. Arora & V. Kansal (2019), Character level embedding with deep convolutional neural network for text normalization of unstructured data for twitter sentiment analysis, Social Network Analysis and Mining, vol. 9. google scholar
• M. E. Hannes Holm (2013), Estimates on the effectiveness of web application firewalls against targeted attacks, pp. 250-265. google scholar
• M. Markou & S. Singh (2003a), Novelty detection: A review—part 1: Statistical approaches, Signal Processing, vol. 83, no. 12, pp. 2481-2497, ISSN: 0165-1684. google scholar
• M. Markou & S. Singh (2003b), Novelty detection: A review—part 2: Neural networkbased approaches, Signal Processing, vol. 83, no. 12, pp. 2499-2521,ISSN: 0165-1684. google scholar
• M. Nadeem, O. Marshall, S. Singh, X. Fang & X. Yuan (2016), Semi-supervised deep neural network for network intrusion detection, KSU Conference On Cybersecurıty Educatıon, Research And Practıce. google scholar
• M. White, M. Tufano, C. Vendome & D. Poshyvanyk (2016), Deep learning code fragments for code clone detection,31st IEEE/ACM International Conferenceon Automated Software Engineering (ASE), pp. 87-98. google scholar
• M.Wahl, T.Howes & S.Kille (1997), Ldapv,Retrieved from: https://datatracker.ietf.org/doc/html/RFC2251 google scholar
• N. Ben-Asher & C. Gonzalez (2015), Training for the unknown: The role of feedback and similarity in detecting zero-day attacks, Procedia Manufacturing, vol. 3, pp. 1088-1095, 2015, 6th International Conference on Applied Human Factors and Ergonomics and the Affiliated Conferences. google scholar
• N. Galbreath (2012), Libinjection. Retrieved from https://github.com/client9/libinjection (visited on 2012). google scholar
• N. Görnitz, M. Kloft, M. Rieck, & U. Brefeld (2013), Toward supervised anomaly detection, Journal of Artificial Intelligence Research, vol. 46, pp. 235-262. google scholar
• N. Montes, G. Betarte, Â. Pardo & R. Martmez (2018). Web Application Attacks Detection Using Machine Learning Techniques. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), 1065-1072. google scholar
• N. Montes, G. Betarte, Â. Pardo & R. Martmez (2021). Web ApplicationAttacks Detection UsingDeep Learning. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 25th Iberoamerican Congress, CIARP 2021, 227-236. google scholar
• N. Oliveira, I. Praça, E. Maia and O. Sousa. (2021). Intelligent Cyber Attack Detection and Classification for Network-Based Intrusion Detection Systems. Applied Sciences. 11. 1674. 10.3390/app11041674. google scholar
• OWASP Cross site scripting (XSS) (2021). Retrieved from https://owasp.org/wwwcommunity/attacks/xss/. google scholar
• OWASP Ldap injection (2021), Retrieved from: https://cheatsheetseries.owasp.org/cheatsheets/LDAP_Injection_Prevention_Cheat_Sheet.html google scholar
• OWASP Php code injection (2021). Retrieved from https://owasp.org/www-community/attacks/Code_Injection . google scholar
• OWASP Server-side template injection (2021), Retrieved from: https://owasp.org/www-project-web-security-testing-guide/ stable/4-Web_Application_Security_Testing/07-Input_Validation_Testing/18-Testing_for_Server-side_Template_Injection google scholar
• OWASP Sql injection (2021). Retrieved from https://owasp.org/www-community/attacks/SQL_Injection. google scholar
• Owasp TOP10 web application security risk (2021). Retrieved from https://owasp.org/www-project-top-ten/. google scholar
• OWASP Web application firewall (2021). Retrieved from https://owasp.org/www-community/Web_Application_Firewall. google scholar
• Payloads all the things (2021). Retrieved from https://github.com/swisskyrepo/PayloadsAllTheThings. google scholar
• Portswigger Cross site scripting (2021), Retrieved from: https://portswigger.net/web-security/cross-site-scripting/ google scholar
• Portswigger Path traversal (2021), Retrieved from: https://portswigger.net/web-security/file-path-traversal google scholar
• Portswigger Sql injection cheat sheet (2021), Retrieved from: https://portswigger.net/web-security/sql-injection/cheat-sheet google scholar
• Q. Zhu, Z. He, T. Zhang & W. Cui (2020), Improving classification performance of softmax loss function based on scalable batch-normalization, Applied Sciences, vol. 10, no. 8. google scholar
• R. Cahuantzi & X. A. Chen & S. Güttel (2021), A comparison of LSTM and GRU networks for learning symbolic sequences. ArXiv, abs/2107.02248.. google scholar
• R. Chalapathy & S. Chawla (2019), Deep learning for anomaly detection: A survey. arXiv: 1901.03407. google scholar
• R. M. Cooke (1991), Experts in Uncertainty: Opinion and Subjective Probability in Science, .New York:Oxford University Press. google scholar
• S Wold, K. Esbensen & P. Geladi (1987), Principal component analysis, Chemometrics and Intelligent Laboratory Systems, vol. 2, no. 1, pp. 37-52, ISSN: 0169-7439. google scholar
• S. Erfani, M. Baktashmotlagh, M. Moshtaghi, V. Nguyen, C. Leckie, J. Bailey, K. Ramamohanarao (2017), From shared subspaces to shared landmarks: A robust multi-source classification approach, Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31. google scholar
• S. Hochreiter & J. Schmidhuber (1997), Long Short-Term Memory, Neural Computation,vol. 9, no. 8, pp. 1735-1780. google scholar
• S. Hochreiter & J. Schmidhuber (1997), Long short-term memory, Neural Comput.,vol. 9, no. 8, pp. 1735-1780. google scholar
• S. Hochreiter (1991), Untersuchungen zu dynamischen neuronalen netzen. google scholar
• S. Hochreiter (1998), The vanishing gradient problem during learning recurrent neural nets and problem solutions, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 6, pp. 107-116. google scholar
• S. S. Kashi (2019), Leveraging deep neural networks for anomaly-based web application firewall, English, IET Information Security, vol. 13, 352-361(9), ISSN: 1751-8709. google scholar S. Young (2021), Designing a DMZ, SANS Institute. google scholar
• SANS Exploiting XXE vulnerabilities (2017), Retrieved from: https://www.sans.org/blog/exploiting-xxe-vulnerabilities-in-iis-net/ google scholar
• Statista (2021), Annual number of data breaches and exposed records in the United States from 2005 to 2020, Retrieved from https://www.statista.com/ statistics/273550/data-breaches-recorded-in-the-united-states-by-number-of-breaches-and-records-exposed/. google scholar
• T. Alma & M. L. Das (2020), Web application attack detection using deep learning, arXiv: 2011.03181. google scholar
• T. Liu, U. Qi, L. Shi, J. Yan (2019), Locate-then-detect: Real-time web attack detection via attention-based deep neural networks, Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, International Joint Conferences on Artificial Intelligence Organization, pp. 4725-4731. google scholar
• T. Mikolov, I. Sutskever, K. Chen, G. Corrado & J. Dean (2013), Distributed representations of words and phrases and their compositionality, pp. 3111-3119. google scholar
• T. Yu & H. Zhu (2020), Hyper-parameter optimization: A review of algorithms and applications, ArXiv, vol. abs/2003.05689. google scholar
• V . Jumutc & J. A. Suykens (2014), Multi-class supervised novelty detection, IEEE Transactionson Pattern Analysis and Machine Intelligence, vol. 36, no. 12, pp. 2510- 2523. google scholar
• WASC (2010), Web application security consortium, Retrieved from: http://projects.webappsec.org/f/WASC-TC-v2_0.pdf google scholar
• WASC Os command injection (2009), Retrieved from: http://projects.webappsec.org/w/page/13246950/OS%5C%20Commanding google scholar
• Y . Dong, Y. Zhang, H. Ma, Q. Wu, Q. Liu, K. Wang & W. Wang (2018), An adaptive system for detecting malicious queries in web attacks, Science China Information Sciences, vol. 61, no. 3. google scholar
• Y . Hu, A.E. Huber, J. Anumula, & S. Liu (1998), Overcoming the vanishing gradient problem in plain recurrent networks, Retrieved from https:// openreview.net/forum?id=Hyp3i2xRb google scholar
• Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019). Roberta: A robustly optimized bert pretraining approach, ICLR 2020 Conference. google scholar
• Y . Pan, F. Sun, Z. Teng, J. White, C. Schmidt, J. Staples, L. Krause (2019), Detecting web attacks with end-to-end deep learning, Journal of Internet Services and Applications, vol. 10. google scholar
• Y . Yu, X. Si, C. Hu and J. Zhang (2019), A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures, Neural Computation, vol. 31, no. 7, pp. 1235-1270. google scholar Z. Li, D. Zou, S. Xu, X. Ou, H. Jin, S. Wang, Z. Deng, Y. Zhong (2018), Vuldeepec google scholar