ANOMALY DETECTION WITH API CALLS BY USING MACHINE LEARNING: SYSTEMATIC LITERATURE REVIEW

Abstract

API, in other words system calls are critical data sources for monitoring the operation of systems and applications, and the data obtained from these calls provides a wealth of information for anomaly detection. API calls are the basic building blocks of the interaction between the oper- ating system and user applications, and analysis of these calls provides important data for securing the system. Anomaly detection is crucial for system security and performance. ML models learn nor- mal and abnormal behaviors by processing large amounts of data and use this information to detect anomalies in new data. When anomaly detection using system calls is combined with ML algorithms, it can make more precise and accurate detections. In this paper, we focus on anomaly detection with machine learning methods using API calls. We present a SLR on the topic as well as a SoK by provid- ing basic knowledge. The main goal is to describe, synthesize, and compare security advancements in anomaly detection using API calls with ML algorithms by examining them through the lens of vari- ous research questions. More than 30 research papers were retrieved using search phrases identified from common and reputable databases, and those relevant to the topic were included in the SLR us- ing different screening criteria. In addition, the reviewed studies were compared in terms of different metrics such as dataset, platform, success parameter, used ML method, and features.

Keywords

• API call
• system call
• anomaly detection
• machine learning

References

1. S. Garg, S. Batra, A novel ensembled technique for anomaly detection, International Journal of Communication Systems 30 (11) (2017) e3248.
2. S. Ranshous, S. Shen, D. Koutra, S. Harenberg, C. Faloutsos, N. F. Samatova, Anomaly detection in dynamic networks: a survey, Wiley Interdisciplinary Reviews: Computational Statistics 7 (3) (2015) 223–247.
3. M. Ahmed, A. N. Mahmood, M. R. Islam, A survey of anomaly detection techniques in financial domain, Future Gener- ation Computer Systems 55 (2016) 278–288.
4. D. Alsalman, A comparative study of anomaly detection techniques for iot security using amot (adaptive machine learn- ing for iot threats), IEEE Access (2024).
5. B. Jin, S. Sahni, A. Shevat, Designing Web APIs: Building APIs That Developers Love, ” O’Reilly Media, Inc.”, 2018.
6. A. Almaleh, R. Almushabb, R. Ogran, Malware api calls detection using hybrid logistic regression and rnn model, Applied Sciences 13 (9) (2023) 5439.
7. Y. Li, F. Kang, H. Shu, X. Xiong, Y. Zhao, R. Sun, Apiaso: A novel api call obfuscation technique based on address space obscurity, Applied Sciences 13 (16) (2023) 9056.
8. F. Osamor, B. Wellman, Deep learning-based hybrid model for efficient anomaly detection, International Journal of Advanced Computer Science and Applications 13 (4) (2022).

9. U. S. Shanthamallu, A. Spanias, C. Tepedelenlioglu, M. Stanley, A brief survey of machine learning methods and their sensor and iot applications, in: 2017 8th International Conference on Information, Intelligence, Systems & Applications (IISA), IEEE, 2017, pp. 1–8.
10. I. Muhammad, Z. Yan, Supervised machine learning approaches: A survey, ICTACT Journal on Soft Computing 5 (3) (2015).
11. I. Rish, et al., An empirical study of the naive bayes classifier, in: IJCAI 2001 workshop on empirical methods in artificial intelligence, Vol. 3, Citeseer, 2001, pp. 41–46.
12. E. Min, J. Long, Q. Liu, J. Cui, W. Chen, Tr-ids: Anomaly-based intrusion detection through text-convolutional neural network and random forest, Security and Communication Networks 2018 (1) (2018) 4943509.
13. K. Beyer, J. Goldstein, R. Ramakrishnan, U. Shaft, When is “nearest neighbor” meaningful?, in: Database The- ory—ICDT’99: 7th International Conference Jerusalem, Israel, January 10–12, 1999 Proceedings 7, Springer, 1999, pp. 217–235.
14. H. Liu, B. Lang, Machine learning and deep learning methods for intrusion detection systems: A survey, applied sciences 9 (20) (2019) 4396.
15. Y. Liu, X. Hao, B. Zhang, Y. Zhang, Simplified long short-term memory model for robust and fast prediction, Pattern Recognition Letters 136 (2020) 81–86. S. Yang, A. Jin, W. Nie, C. Liu, Y. Li, Research on ssa-lstm-based slope monitoring and early warning model, Sustain- ability 14 (16) (2022) 10246.
16. J. Bernal, K. Kushibar, D. S. Asfaw, S. Valverde, A. Oliver, R. Mart´ı, X. Llado´, Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review, Artificial intelligence in medicine 95 (2019) 64–81.
17. Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324.
18. R. Yamashita, M. Nishio, R. K. G. Do, K. Togashi, Convolutional neural networks: an overview and application in radiology, Insights into imaging 9 (2018) 611–629.
19. G. Yao, T. Lei, J. Zhong, A review of convolutional-neural-network-based action recognition, Pattern Recognition Letters 118 (2019) 14–22.
20. A. Akagic, I. Dzˇafic´, Enhancing smart grid resilience with deep learning anomaly detection prior to state estimation, Engineering Applications of Artificial Intelligence 127 (2024) 107368.
21. G. Duan, Y. Fu, M. Cai, H. Chen, J. Sun, Dongting: A large-scale dataset for anomaly detection of the linux kernel, Journal of Systems and Software 203 (2023) 111745.
22. S. L. Rocha, F. L. L. de Mendonca, R. S. Puttini, R. R. Nunes, G. D. A. Nze, Dcids—distributed container ids, Applied Sciences 13 (9301) (2023) 9301.
23. A. Chaudhari, B. Gohil, U. P. Rao, A novel hybrid framework for cloud intrusion detection system using system call sequence analysis, Cluster Computing (2023) 1–17.
24. D. Zhan, K. Tan, L. Ye, X. Yu, H. Zhang, Z. He, An adversarial robust behavior sequence anomaly detection approach based on critical behavior unit learning, IEEE Transactions on Computers (2023).
25. S. Ahn, H. Yi, H. Bae, S. Yoon, Y. Paek, Data embedding scheme for efficient program behavior modeling with neural networks, IEEE Transactions on Emerging Topics in Computational Intelligence 6 (4) (2022) 982–993.
26. A. Karamanou, P. Brimos, E. Kalampokis, K. Tarabanis, Exploring the quality of dynamic open government data using statistical and machine learning methods, Sensors 22 (24) (2022) 9684.
27. D. Cotroneo, L. De Simone, P. Liguori, R. Natella, Fault injection analytics: A novel approach to discover failure modes in cloud-computing systems, IEEE transactions on dependable and secure computing 19 (3) (2020) 1476–1491.
28. Y. Wang, Y. Jiang, J. Lan, Intrusion detection using few-shot learning based on triplet graph convolutional network, Journal of Web Engineering 20 (5) (2021) 1527–1552.
29. C. Kim, M. Jang, S. Seo, K. Park, P. Kang, Intrusion detection based on sequential information preserving log embedding methods and anomaly detection algorithms, IEEE Access 9 (2021) 58088–58101.
30. R. R. Karn, P. Kudva, H. Huang, S. Suneja, I. M. Elfadel, Cryptomining detection in container clouds using system calls and explainable machine learning, IEEE transactions on parallel and distributed systems 32 (3) (2020) 674–691.
31. O. M. Ezeme, A. Azim, Q. H. Mahmoud, Peskea: Anomaly detection framework for profiling kernel event attributes in embedded systems, IEEE Transactions on Emerging Topics in Computing 9 (2) (2020) 957–971.
32. F. J. Mora-Gimeno, H. Mora-Mora, B. Volckaert, A. Atrey, Intrusion detection system based on integrated system calls graph and neural networks, IEEE Access 9 (2021) 9822–9833.
33. I. Kohyarnejadfard, D. Aloise, M. R. Dagenais, M. Shakeri, A framework for detecting system performance anomalies using tracing data analysis, Entropy 23 (8) (2021) 1011.
34. H. Kim, S. Ahn, W. R. Ha, H. Kang, D. S. Kim, H. K. Kim, Y. Paek, Panop: Mimicry-resistant ann-based distributed nids for iot networks, IEEE Access 9 (2021) 111853–111864.
35. S. K. Peddoju, H. Upadhyay, J. Soni, N. Prabakar, Natural language processing based anomalous system call sequences detection with virtual memory introspection, International Journal of Advanced Computer Science and Applications 11 (5) (2020).
36. Y. Shin, K. Kim, Comparison of anomaly detection accuracy of host-based intrusion detection systems based on different machine learning algorithms, International Journal of Advanced Computer Science and Applications 11 (2) (2020).
37. Z. Liu, N. Japkowicz, R. Wang, Y. Cai, D. Tang, X. Cai, A statistical pattern based feature extraction method on system call traces for anomaly detection, Information and Software Technology 126 (2020) 106348.
38. T. Ergen, S. S. Kozat, Unsupervised anomaly detection with lstm neural networks, IEEE transactions on neural networks and learning systems 31 (8) (2019) 3127–3141.
39. L. Liu, C. Chen, J. Zhang, O. De Vel, Y. Xiang, Insider threat identification using the simultaneous neural learning of multi-source logs, IEEE Access 7 (2019) 183162–183176.
40. Q. Chen, R. Luley, Q. Wu, M. Bishop, R. W. Linderman, Q. Qiu, Anrad: A neuromorphic anomaly detection framework for massive concurrent data streams, IEEE transactions on neural networks and learning systems 29 (5) (2017) 1622– 1636.
41. S. Lv, J. Wang, Y. Yang, J. Liu, Intrusion prediction with system-call sequence-to-sequence model, IEEE Access 6 (2018) 71413–71421.
42. O. M. Ezeme, Q. H. Mahmoud, A. Azim, Dream: deep recursive attentive model for anomaly detection in kernel events, IEEE Access 7 (2019) 18860–18870.
43. W. Haider, J. Hu, Y. Xie, X. Yu, Q. Wu, Detecting anomalous behavior in cloud servers by nested-arc hidden semi- markov model with state summarization, IEEE Transactions on Big Data 5 (3) (2018) 305–316.