A Framework for Studying New Approaches to Anomaly Detection

Year 2016, Volume: 5 Issue: 2, 39 - 50, 01.06.2016

Esra Nergis Yolacan David R. Kaeli

Abstract

In this work, we describe a new framework for an anomaly-based intrusion detection system using system call traces. System calls provide an interface between an application and the operating system’s kernel. Since a program frequently requests services via system calls, a trace of these system calls provides a rich profile of program behavior. But we need to use efficient and effective methods while extracting the underlying behavior. In this paper we present an illustrative example to describe how to apply our proposed approach on system call traces for cyber security. We discuss the details of system call anomaly detection by considering various normal behaviors in program traces. Test and detection results show the proposed approach provides fast and accurate anomaly detection by applying context-aware behavior learning.

Keywords

Intrusion detection, anomaly, system call traces

References

• V. J. Hodge and J. Austin, “A survey of outlier detection methodologies,” Artificial Intelligence Review, vol. 22, no. 2, pp. 85–126, 2004.
• D. C. Montgomery, C. L. Jennings, and M. Kulahci, Introduc- tion to time series analysis and forecasting. Sons, 2011, vol. 526. John Wiley &
• A. Kl¨aser, M. Marszałek, C. Schmid, and A. Zisserman, “Hu- man focused action localization in video,” in Trends and Topics in Computer Vision.
• G. Aloysius and D. Binu, “An approach to products placement in supermarkets using prefixspan algorithm,” Journal of King Saud University-Computer and Information Sciences, vol. 25, no. 1, pp. 77–87, 2013.
• T.-c. Fu, “A review on time series data mining,” Engineering Applications of Artificial Intelligence, vol. 24, no. 1, pp. 164– 181, 2011.
• I. Kinde, J. Wu, N. Papadopoulos, K. W. Kinzler, and B. Vo- gelstein, “Detection and quantification of rare mutations with massively parallel sequencing,” Proceedings of the National Academy of Sciences, vol. 108, no. 23, pp. 9530–9535, 2011.
• K.-P. Chan and A.-C. Fu, “Efficient time series matching by wavelets,” in Data Engineering, 1999. Proceedings., 15th International Conference on.
• Z. Xing, J. Pei, and E. Keogh, “A brief survey on sequence classification,” ACM SIGKDD Explorations Newsletter, vol. 12, no. 1, pp. 40–48, 2010.
• X. D. Hoang, J. Hu, and P. Bertok, “A multi-layer model for anomaly intrusion detection using program sequences of system calls,” in Networks, ICON2003. The 11th IEEE International Conference on.
• X. Hoang and J. Hu, “An efficient hidden markov model training scheme for anomaly intrusion detection of server applications based on system calls,” in Networks, 2004.(ICON 2004). Pro- ceedings. 12th IEEE International Conference on, vol. 2. IEEE, 2004, pp. 470–474.
• C. Kruegel, D. Mutz, F. Valeur, and G. Vigna, “On the detection of anomalous system call arguments,” in Computer Security– ESORICS 2003.
• D. Mutz, F. Valeur, G. Vigna, and C. Kruegel, “Anomalous system call detection,” ACM Transactions on Information and System Security (TISSEC), vol. 9, no. 1, pp. 61–93, 2006.
• F. Maggi, M. Matteucci, and S. Zanero, “Detecting intrusions through system call sequence and argument analysis,” Depend- able and Secure Computing, IEEE Transactions on, vol. 7, no. 4, pp. 381–395, 2010.
• G. Tandon and P. Chan, “Learning rules from system call arguments and sequences for anomaly detection,” in ICDM Workshop on Data Mining for Computer Security (DMSEC), 2003, pp. 20–29.
• D.-K. Kang, D. Fuller, and V. Honavar, “Learning classifiers for misuse and anomaly detection using a bag of system calls representation,” in Information Assurance Workshop, 2005. IAW’05. Proceedings from the Sixth Annual IEEE SMC. IEEE, 2005, pp. 118–125.
• Y. Liao and V. R. Vemuri, “Use of k-nearest neighbor classifier for intrusion detection,” Computers & Security, vol. 21, no. 5, pp. 439–448, 2002.
• N. Ye and Q. Chen, “An anomaly detection technique based on a chi-square statistic for detecting intrusions into information systems,” Quality and Reliability Engineering International, vol. 17, no. 2, pp. 105–112, 2001.
• Z. Zhang and H. Shen, “Application of online-training svms for real-time intrusion detection with different considerations,” Computer Communications, vol. 28, no. 12, pp. 1428–1442, 2005.
• S. M. Varghese and K. P. Jacob, “Process profiling using frequencies of system calls,” in Availability, Reliability and Se- curity, 2007. ARES 2007. The Second International Conference on.
• W.-H. Chen, S.-H. Hsu, and H.-P. Shen, “Application of svm and ann for intrusion detection,” Computers & Operations Research, vol. 32, no. 10, pp. 2617–2634, 2005.
• W. Wang, X. Zhang, and S. Gombault, “Constructing attribute weights from computer audit data for effective intrusion detec- tion,” Journal of Systems and Software, vol. 82, no. 12, pp. 1974–1981, 2009.
• S. Forrest, S. A. Hofmeyr, A. Somayaji, and T. A. Longstaff, “A sense of self for unix processes,” in Security and Privacy, 1996. Proceedings., 1996 IEEE Symposium on. pp. 120–128.
• S. A. Hofmeyr, S. Forrest, and A. Somayaji, “Intrusion detection using sequences of system calls,” Journal of computer security, vol. 6, no. 3, pp. 151–180, 1998.
• UNM. (2013) Unm system call dataset. [Online; accessed 28-November-2013]. [Online]. Available: \url{http://www.cs. unm.edu/$∼$immsec/systemcalls.htm}
• S. Brooks, A. Gelman, G. Jones, and X.-L. Meng, Handbook of Markov Chain Monte Carlo.
• D.-Y. Yeung and Y. Ding, “Host-based intrusion detection using dynamic and static behavioral models,” Pattern recognition, vol. 36, no. 1, pp. 229–243, 2003.
• Y. Du, H. Wang, and Y. Pang, “A hidden markov models-based anomaly intrusion detection method,” in Intelligent Control and Automation, 2004. WCICA 2004. Fifth World Congress on, vol. 5.
• IEEE, 2004, pp. 4348–4351.
• W. Khreich, E. Granger, R. Sabourin, and A. Miri, “Combining hidden markov models for improved anomaly detection,” in Communications, 2009. ICC’09. IEEE International Conference on.
• J. Hu, X. Yu, D. Qiu, and H.-H. Chen, “A simple and efficient hidden markov model scheme for host-based anomaly intrusion detection,” Network, IEEE, vol. 23, no. 1, pp. 42–47, 2009.
• C. Warrender, S. Forrest, and B. Pearlmutter, “Detecting intru- sions using system calls: Alternative data models,” in Security and Privacy, 1999. Proceedings of the 1999 IEEE Symposium on.
• E. Eskin, W. Lee, and S. J. Stolfo, “Modeling system calls for intrusion detection with dynamic window sizes,” in DARPA Information Survivability Conference & Exposition II, 2001. DISCEX’01. Proceedings, vol. 1.
• IEEE, 2001, pp. 165–175.
• W. Lee and D. Xiang, “Information-theoretic measures for anomaly detection,” in Security and Privacy, 2001. S&P 2001. Proceedings. 2001 IEEE Symposium on. IEEE, 2001, pp. 130– 143.
• K. M. Tan and R. A. Maxion, “Determining the operational limits of an anomaly-based intrusion detector,” Selected Areas in Communications, IEEE Journal on, vol. 21, no. 1, pp. 96– 110, 2003.
• L. R. Rabiner, “A tutorial on hidden markov models and selected applications in speech recognition,” Proceedings of the IEEE, vol. 77, no. 2, pp. 257–286, 1989.
• G. Schwarz, “Estimating the dimension of a model,” The annals of statistics, vol. 6, no. 2, pp. 461–464, 1978.
• E. N. Yolacan, J. G. Dy, and D. R. Kaeli, “System call anomaly detection using multi-hmms,” in Software Security and Reliability-Companion (SERE-C), 2014 IEEE Eighth Interna- tional Conference on.