Encrypted malware detection methodology without decryption using deep learning-based approaches

Year 2024, Volume: 8 Issue: 3, 498 - 509, 28.07.2024

Abhay Pratap Singh , Mahendra Singh Karamjit Bhatia Heman Pathak

https://doi.org/10.31127/tuje.1416933

Abstract

The encrypted or https traffic on Internet accounts for the safe and secure
communication between users and servers. However, cyber attackers are also exploiting https traffic to disguise their malignant activities. Detection of network threats in https traffic is a tiresome task for security experts owing to the convoluted nature of encrypted traffic on the web. Conventional detection techniques decrypt the network content, check it for threats, re-encrypt the network content, and then send it to the server. But this approach jeopardizes the secrecy of data and user. In recent time, deep learning (DL) has emerged as one of the most fruitful AI methods that diminishes the manual resolution of features to enhance classification accuracy. A DL based strategy is suggested for recognition of threat in encrypted communication without using decryption. The three DL algorithms, as used by the proposed approach are, multilayer perceptron (MLP), long short-term memory (LSTM) and 1-D convolutional neural network (1-D CNN), which are experimented on the CTU-
13 malware dataset containing flow-based attributes of network traffic. The outcome of the experiment exhibits that MLP based approach performs better in comparison to 1-D CNN and LSTM based ones and other existing approaches. Thus, the secrecy of the data is maintained and the capability of identifying threats in encrypted communication is augmented.

Keywords

Malware, Encrypted traffic, Deep learning, Network security

Thanks

Respected sir, As per your suggestions, I have submitted my manuscript in word format, cited papers this journal along with similarity report. Thanks for kind cooperation

References

• Papadogiannaki, E., & Ioannidis, S. (2021). A survey on encrypted network traffic analysis applications, techniques, and countermeasures. ACM Computing Surveys (CSUR), 54(6), 1-35. https://doi.org/10.1145/3457904
• Singh, A. P., & Singh, M. (2021). A comparative review of malware analysis and detection in HTTPs traffic. International Journal of Computing and Digital Systems, 10(1), 111-123. http://dx.doi.org/10.12785/ijcds/100111
• Ayas, M. Ş. (2021). A brief review on attack design and detection strategies for networked cyber-physical systems. Turkish Journal of Engineering, 5(1), 1-7. https://doi.org/10.31127/tuje.640282
• Grier, C., Ballard, L., Caballero, J., Chachra, N., Dietrich, C. J., Levchenko, K., ... & Voelker, G. M. (2012, October). Manufacturing compromise: the emergence of exploit-as-a-service. In Proceedings of the 2012 ACM Conference on Computer and Communications Security, 821-832. https://doi.org/10.1145/2382196.238228
• Mishra, N., & Pandya, S. (2021). Internet of things applications, security challenges, attacks, intrusion detection, and future visions: A systematic review. IEEE Access, 9, 59353-59377. https://doi.org/10.1109/ACCESS.2021.3073408
• Zhu, T., Weng, Z., Fu, L., & Ruan, L. (2020). A web shell detection method based on multiview feature fusion. Applied Sciences, 10(18), 6274. https://doi.org/10.3390/app10186274
• Zhao, Y., Yang, Y., Tian, B., Yang, J., Zhang, T., & Hu, N. (2021). Edge intelligence based identification and classification of encrypted traffic of Internet of Things. IEEE Access, 9, 21895-21903. https://doi.org/10.1109/ACCESS.2021.3056216
• Wang, P., Ye, F., Chen, X., & Qian, Y. (2018). Datanet: Deep learning based encrypted network traffic classification in sdn home gateway. IEEE Access, 6, 55380-55391. https://doi.org/10.1109/ACCESS.2018.2872430
• Atli, B. G., Miche, Y., Kalliola, A., Oliver, I., Holtmanns, S., & Lendasse, A. (2018). Anomaly-based intrusion detection using extreme learning machine and aggregation of network traffic statistics in probability space. Cognitive Computation, 10(5), 848-863. https://doi.org/10.1007/s12559-018-9564-y
• Guo, L., Wu, Q., Liu, S., Duan, M., Li, H., & Sun, J. (2020). Deep learning-based real-time VPN encrypted traffic identification methods. Journal of Real-Time Image Processing, 17(1), 103-114. https://doi.org/10.1007/s11554-019-00930-6
• Anderson, B., & McGrew, D. (2016, October). Identifying encrypted malware traffic with contextual flow data. In Proceedings of the 2016 ACM workshop on Artificial Intelligence and Security, 35-46. https://doi.org/10.1145/2996758.2996768
• Anderson, B., Paul, S., & McGrew, D. (2018). Deciphering malware’s use of TLS (without decryption). Journal of Computer Virology and Hacking Techniques, 14, 195-211. https://doi.org/10.1007/s11416-017-0306-6
• Anderson, B., & McGrew, D. (2017, August). Machine learning for encrypted malware traffic classification: accounting for noisy labels and non-stationarity. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1723-1732. https://doi.org/10.1145/3097983.3098163
• McGrew, D., & Anderson, B. (2016). Enhanced telemetry for encrypted threat analytics. In 2016 IEEE 24th International Conference on Network Protocols (ICNP), 1-6. https://doi.org/10.1109/ICNP.2016.7785325
• Shekhawat, A. S., Di Troia, F., & Stamp, M. (2019). Feature analysis of encrypted malicious traffic. Expert Systems with Applications, 125, 130-141. https://doi.org/10.1016/j.eswa.2019.01.064
• Hamad, M., Durad, M. H., & Yousaf, M. (2018). Mitigation of the effect of standard networks attacks in SSL encrypted traffic by encrypted traffic analysis. VFAST Transactions on Mathematics, 6(1), 15-22. https://doi.org/10.21015/vtm.v8i1.578
• Dai, R., Gao, C., Lang, B., Yang, L., Liu, H., & Chen, S. (2019, November). SSL malicious traffic detection based on multi-view features. In Proceedings of the 2019 9th International Conference on Communication and Network Security, 40-46. https://doi.org/10.1145/3371676.3371697
• Scarbrough, B. (2021). Malware Detection in Encrypted TLS Traffic Through Machine Learning. Global Information Assurance Certification Paper.
• Zheng, R., Liu, J., Li, K., Liao, S., & Liu, L. (2020, August). Detecting malicious tls network traffic based on communication channel features. In 2020 IEEE 8th International Conference on Information, Communication and Networks (ICICN), 14-19. https://doi.org/10.1109/ICICN51133.2020.9205087
• Luo, Z. M., & Xu, S. B. (2020). Scheme for identifying malware traffic with TLS data based on machine learning. Chinese Journal of Network and Information Security, 6(1), 77-83.
• Wang, W., Sun, C. S., & Ye, J. N. (2021). A method for TLS malicious traffic identification based on machine learning. Advances in Science and Technology, 105, 291-301. https://doi.org/10.4028/www.scientific.net/AST.105.291
• Gomez, G., Kotzias, P., Dell’Amico, M., Bilge, L., & Caballero, J. (2023). Unsupervised detection and clustering of malicious tls flows. Security and Communication Networks, 2023(1), 3676692. https://doi.org/10.1155/2023/3676692
• Yu, T., Zou, F., Li, L., & Yi, P. (2019). An encrypted malicious traffic detection system based on neural network. In 2019 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), 62-70. https://doi.org/10.1109/CyberC.2019.00020
• Zhou, Z., Bin, H., Li, J., Yin, Y., Chen, X., Ma, J., & Yao, L. (2022). Malicious encrypted traffic features extraction model based on unsupervised feature adaptive learning. Journal of Computer Virology and Hacking Techniques, 18(4), 453-463. https://doi.org/10.1007/s11416-022-00429-y
• Jie, F. (2020, September). Research on malicious TLS traffic identification based on hybrid neural network. In 2020 International Conference on Advance in Ambient Computing and Intelligence (ICAACI), 42-46. https://doi.org/10.1109/ICAACI50733.2020.00013
• Bakhshi, T., & Ghita, B. (2021). Anomaly detection in encrypted internet traffic using hybrid deep learning. Security and Communication Networks, 2021(1), 5363750. https://doi.org/10.1155/2021/5363750
• Bazuhair, W., & Lee, W. (2020, January). Detecting malign encrypted network traffic using perlin noise and convolutional neural network. In 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), 0200-0206. https://doi.org/10.1109/CCWC47524.2020.9031116
• Zeng, Y., Gu, H., Wei, W., & Guo, Y. (2019). Deep-Full-Range: A deep learning based network encrypted traffic classification and intrusion detection framework. IEEE Access, 7, 45182-45190. https://doi.org/10.1109/ACCESS.2019.2908225
• Ctu-13 dataset. https://www.stratosphereips.org/datasets-ctu13
• CICFlowmeter. https://www.unb.ca/cic/reserach/applications.html
• Zhang, C., Chen, Y., Meng, Y., Ruan, F., Chen, R., Li, Y., & Yang, Y. (2021). A novel framework design of network intrusion detection based on machine learning techniques. Security and Communication Networks, 2021(1), 6610675. https://doi.org/10.1155/2021/6610675
• Pontes, C. F., De Souza, M. M., Gondim, J. J., Bishop, M., & Marotta, M. A. (2021). A new method for flow-based network intrusion detection using the inverse Potts model. IEEE Transactions on Network and Service Management, 18(2), 1125-1136. https://doi.org/10.1109/TNSM.2021.3075503
• Başarslan, M. S., & Kayaalp, F. (2023). Sentiment analysis with ensemble and machine learning methods in multi-domain datasets. Turkish Journal of Engineering, 7(2), 141-148. https://doi.org/10.31127/tuje.1079698
• Rezaei, S., & Liu, X. (2019). Deep learning for encrypted traffic classification: An overview. IEEE Communications Magazine, 57(5), 76-81. https://doi.org/10.1109/MCOM.2019.1800819
• Dirik, M. (2023). Machine learning-based lung cancer diagnosis. Turkish Journal of Engineering, 7(4), 322-330. https://doi.org/10.31127/tuje.1180931
• Sharma, A., Malacaria, P., & Khouzani, M. H. R. (2019, June). Malware detection using 1-dimensional convolutional neural networks. In 2019 IEEE European symposium on security and privacy workshops (EuroS&PW), 247-256. https://doi.org/10.1109/EuroSPW.2019.00034
• Azizjon, M., Jumabek, A., & Kim, W. (2020, February). 1D CNN based network intrusion detection with normalization on imbalanced data. In 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), 218-224. https://doi.org/10.1109/ICAIIC48513.2020.9064976
• Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., ... & Farhan, L. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8, 1-74. https://doi.org/10.1186/s40537-021-00444-8
• Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
• Dang, D., Di Troia, F., & Stamp, M. (2021). Malware classification using long short-term memory models. Cryptography and Security,1-16. https://doi.org/10.48550/arXiv.2103.02746