BT Sistemlerinde Veri Madenciliği Yöntemlerini Kullanarak Anomali Algılama: Karar Destek Uygulaması

Year 2018, Volume: 22 Issue: 4, 1109 - 1123, 01.08.2018

Ferdi Sönmez Metin Zontul Oğuz Kaynar Hayati Tutar

https://doi.org/10.16984/saufenbilder.365931

Cited By: 1

Abstract

 Anomali tespiti üzerine çeşitli çalışmalar olmasına
rağmen, büyük ölçekli ağ ortamları için uygun yaklaşımların eksikliği nedeniyle
basit ve etkili anormali tespit yaklaşımları hala gereklidir. Mevcut
çalışmalardaki analiz yöntemlerinde, genellikle ön tanımlı analiz tekniklerinin
kullanıldığı, kapsam dışı durum ve olasılıkların dikkate alınmadığı ve
danışmansız öğrenen yapay sinir ağları (NN) metotlarının yeterince
kullanılmadığı görülmektedir. Alternatif olarak bu çalışmada özörgütlemeli
harita ağları kullanımı tercih edilmiştir. Diğer çalışmalarda, genellikle ağ
trafiğinden elde dilen veriler analiz edilirken, bu çalışmada diğer bilişim
sistemi verilerinin de analizine ve alternatif olabilecek çözüm önerilerine yer
verilmektedir. Ayrıca, büyük ölçekli ağ ortamlarında analiz edilecek verilerin
büyük boyutlu olmasına bağlı olarak, analiz çalışmalarında işlemlerin daha
hızlı gerçekleştirilebilmesi amacıyla uygulamada Bellek İçi Veritabanı
Sistemleri (BİVTS) kullanılmıştır. Bilişim sitemlerindeki yönetim araçlarından
edinilen uygulama log verilerinin analizi sonrasında anomali tespitinin %96
oranında başarı ile gerçekleştiği gözlenmiştir. Çalışmanın işletme ve
kullanıcılarına bilişim sistemlerini izleme ve güvenlik takibi işlemlerinde ön
tanımlama ihtiyacını ortadan kaldırarak, yoğun iş yükünü azaltma yönünde fayda
sağlayacağı düşünülmektedir. Böylelikle, önemli bir maliyet kaleminin ortadan
kalkacağı da düşünülmektedir. Ayrıca, öngörülemeyen hususlara bağlı güvenlik
açıkları ve problemlerin, uygulama ile tespiti ve böylelikle birçok saldırı ve
problemin önceden engellenileceği öngörülmektedir.

Keywords

Veri Analizi, Anomali Tespiti, Yapay Sinir Ağları, SOM, Bellek İçi Veritabanı Sistemleri

Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application

Abstract

Although there are various
studies on anomaly detection, simple and effective anomaly detection approaches
are still necessary due to the lack of appropriate approaches for large-scale
network environments. In the existing analysis methods, it is seen that the
methods of preliminary analysis are generally used, the extrapolations and probabilities
are not taken into account and the unsupervised neural network (NN) methods are
not used enough. As an alternative, the use of the Self-Organizing Maps has
been preferred in the study. In other studies, analysis of data obtained from
network traffic is analyzed, here, analysis of other information systems data
and suggestions for alternative solutions are given, too. In addition,
in-memory database systems have been used in practice in order to enable faster
processing in analysis studies, due to the large size of data to be analyzed in
large-scale network environments. An analysis of the application log data
obtained from the management tools in the information systems was carried out.
After anomaly detection results obtained and the verification test results are
compared, it is found out that anomaly detection process is successful by 96%.
The advantage offered for the company and users at IT and security monitoring
processes is to eliminate the need for pre-qualification and to reduce the heavy
workload. By this way, it is thought that a significant cost item is
eliminated. It is also contemplated that the security vulnerabilities and
problems associated with unpredictable issues will be detected through practice
and thus many attacks and problems will be prevented in advance.

Keywords

Data Analysis, Anomaly Detection, Artificial Neural Networks, Self-Organizing Maps, In-Memory Database Systems

References

• M. V. O. Assis, J. J. P. C. Rodrigues, M. L. Proença. “A seven-dimensional flow analysis to help autonomous network management”, Information Sciences, 278, 900-913, 2014, doi: 10.1016/j.ins.2014.03.102.
• A. Coluccia, A. D’Alconzo, F. Ricciato. “Distribution-based anomaly detection via generalized likelihood ratio test: A general Maximum Entropy approach”, Computer Networks, 57(17), ss.3446-3462, 2013, http://dx.doi.org/10.1016/j.comnet.2013.07.028.
• F. Mata, P. Żuraniewski, M. Mandjes, M. Mellia. “Anomaly detection in diurnal data”, Computer Networks, 60, ss. 187-200, 2014.
• M. A. Rassam, A. Zainal, M. A. Maarof. “An Efficient Distributed Anomaly Detection Model for Wireless Sensor Networks”, AASRI Procedia, 5, ss. 9-14, 2013, doi: 10.1016/j.aasri.2013.10.052.
• S. Anil, R. Remya. “A hybrid method based on genetic algorithm, self-organised feature map, and support vector machine for better network anomaly detection", 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), Tiruchengode, ss.1-5, 2013, doi: 10.1109/ICCCNT. 2013.6726604
• A. Juvonen, T. Sipola, T. Hämäläinen. “Online anomaly detection using dimensionality reduction techniques for HTTP log analysis”, Computer Networks, 91(14), ss.46-56, 2015, doi: 10.1016/j.comnet. 2015.07.019.
• I. Fronza, A. Sillitti, G. Succi, M. Terho, J. Vlasenko. “Failure prediction based on log files using Random Indexing and Support Vector Machines”, Journal of Systems and Software, 86(1), ss.2-11, 2013, doi: 10.1016/j.jss.2012.06.025.
• D. Olszewski. “Fraud detection using self-organizing map visualizing the user profiles”, Knowledge-Based Systems, 70, 324-334, 2014, doi: 10.1016/j.knosys.2014.07.008.
• C. Modi, D. Patel, B. Borisaniya, H. Patel, A. Patel, M. Rajarajan. “A survey of intrusion detection techniques in Cloud”, Journal of Network and Computer Applications, 36(1), ss.42-57, 2013, doi: 10.1016/j.jnca.2012.05.003.
• A. Botta, A. Dainotti, A. Pescapé. “A tool for the generation of realistic network workload for emerging networking scenarios”, Computer Networks, 56(15), ss.3531-3547, 2012, doi: 10.1016/j. comnet.2012.02.019.
• D. A. S. Resul, I. Turkoglu, I., M. Poyraz. “Analyzing of system errors for increasing a web server performance by using web usage mining”, IU-Journal of Electrical & Electronics Engineering, 7(2), ss.379-386, 2007.
• S. A. Ünlü. “Ağ Üzerinden Yavaşlama Tabanlı Anomali Tespiti”, Tez Çalışması, TOBB Ekonomi Ve Teknoloji Üniversitesi, Fen Bilimleri Enstitüsü, 2011.
• P. Ma. “Log Analysis-Based Intrusion Detection via Unsupervised Learning”, Master of Science, School of Informatics, University of Edinburgh, 2003.
• C. Chiu, Y. Ku, T. Lie, Y. Chen. “Internet auction fraud detection using social network analysis and classification tree approaches”, Int. J. Electron. Commer, 15 (3), ss.123–147, 2011.
• A. Li, L. Gu, K. Xu. "Fast Anomaly Detection for Large Data Centers," 2010 IEEE Global Telecommunications Conference GLOBECOM, Miami, ABD, 2010, doi: 10.1109/GLOCOM. 2010.5683551
• Y. Kanda, K. Fukuda, T. Sugawara. "A Flow Analysis for Mining Traffic Anomalies", 2010 IEEE International Conference on Communications, Cape Town, 2010, doi: 10.1109/ ICC.2010.5502463
• S. Molnar, Z. Moczar. "Three-Dimensional Characterization of Internet Flows," 2011 IEEE International Conference on Communications (ICC), Kyoto, 2011, doi: 10.1109/icc. 2011.5963476
• P. P. Cortez, M. Rio, M. Rocha, P. Sousa. "Internet Traffic Forecasting using Neural Networks," The 2006 IEEE International Joint Conference on Neural Network Proceedings, Vancouver, BC, 2006, doi: 10.1109/IJCNN. 2006.247142
• A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E.D. Kolaczyk, N. Taft. “Structural analysis of network traffic flow”s, SIGMETRICS Perform. Eval. Rev., ss.32, 61–72, 2004.
• C. Yoohee, K. Yihan. “Case study of an anomalous traffic detection on the aggregation points of enterprise network”, International Conference on Advanced Communication Technology (ICACT), Seul, 2011.
• A. Chandola, V. Chandola, V. Kumar. “Anomaly Detection: A Survey”, ACM Comput. Surv., 41(3), 2009, doi: 10.1145/1541880. 1541882
• N. Carneiro, G. Figueira, M. Costa. “A data mining based system for credit-card fraud detection in e-tail”, Decision Support Systems, 95, ss.91-101, 2017, doi: 10.1016/j.dss.2017.01.002.
• V. Kumar. "Parallel and distributed computing for cybersecurity" IEEE Distributed Systems Online, 6(10), 2005. doi: 10.1109/MDSO. 2005.53 .
• K. A. Smith. Introduction to neural networks and data mining for business applications. Eruditions Publishing, Melbourne, 1999.
• D. Swagatam, D. Ajith, K. Amit, ‘‘Automatic kernel clustering with a multi-elitist particle swarm optimization algorithm’’, Pattern: Recognition Letters, 29(5), ss.688–699, 2008.
• J.Z. Lei, A.A. Ghorbani.”Improved competitive learning neural networks for network intrusion and fraud detection”, Neurocomputing, 75 (1), 135-145, 2012, doi: 10.1016/j.neucom. 2011.02.021.
• T. Fawcett, ROC Graphs: Notes and Practical Considerations for Data Mining Researchers, Tech. Rep. HPL-2003-4, HP Labs, 2003
• A. Mitrokotsa, N. Komninos, C. Douligeris. “Intrusion Detection with Neural Networks and Watermarking Techniques for MANET,” IEEE computer society, ss.1-10, 2008.
• W. Wanga, H. Wang, B. Wang, Yaping Wang, Jiajun Wang. “Energy-aware and self-adaptive anomaly detection scheme based on network tomography in mobile ad hoc networks,” Information Sciences 220, ss.580–602, 2013.
• S. Haykin, Neural Networks and Learning Machine, 3E, Pearson Education Inc., New Jersey, 2009.
• L. Cao. “Support vector machines experts for time series forecasting”, Neurocomputing, 51, ss.321-329, doi:10.1016/S0925-2312(02) 00577-5.
• F. Sönmez, Ş. Bülbül. “Intelligent Software Model Design for Estimating Deposit Banks Profitability with Soft Computing Techniques”, Neural Network World, ss.319-345, 2015, doi: 10.14311/NNW.2015.25.017.
• D. Altaş, A. M. Çilingirtürk, V. Gülpınar. “Analyzing the process of the artificial neural networks by the help of the social network analysis”, New Knowledge Journal of Science. 2(2), ss.80–91, 2013.
• B. Yıldız, S. Akkoç. “Banka Finansal Başarısızlıklarının Sinirsel Bulanık Ağ Yöntemi ile Öngörüsü”, BDDK Bankacılık ve Finansal Piyasalar, 3(1), ss.9-36, 2009.
• L. A. Zadeh. “The Roles of Fuzzy Logic and Soft Computing in the Conception, Design and Deployment of Intelligent Systems”, BT Technology Journal, 14(4), ss.32-36, 1994.
• T. K. Kohonen. “The self-organizing map”, Proceedings of the IEEE, 78 (9), ss.1464–1480, 1990.
• T. K. Kohonen, S. Kaski, K. Lagus, J. Saloj¨arvi, J. Honkela, V. Paatero, A. Saarela. “Self Organization of a Massive Document Collection”, IEEE Transactions on Neural Networks, 11(3), ss.574–585, 2000.
• Bullinaria, J. A. Introduction to neural networks. University of Birmingham, UK, 2004.
• L. Yang, Z. Ouyang, Y. Shi. “A Modified Clustering Method Based on Self-Organizing Maps and Its Applications”, Procedia Computer Science, 9, ss.1371-1379, 2012, doi: doi.org/10.1016/j.procs.2012. 04.151.
• J. A. Kangas, T. K. Kohonen, J. T. Jorma. “Variants of self-organizing maps”, IEEE transactions on neural networks, 1(1), ss.93-99, 1990.
• N. R Pal, J. C. Bezdek, E. C. K. Tsao. “Generalized clustering networks and Kohonen's self-organizing scheme”, IEEE transactions on Neural Networks, 4 (4), ss.549-557,1993.
• B. Hammer, T. Villmann. “Generalized relevance learning vector quantization”, Neural Networks, 15(8–9), 1059-1068, 2002, doi: 10.1016/S0893-6080(02)00079-5.
• T. M. Martinetz, S. G.Berkovich, K. J. Schulten. “Neural-gas network for vector quantization and its application to time-series prediction", Neural Networks, IEEE Transactions on, 4(4), ss.558-569, 1993, doi: 10.1109/72.238311.
• G. M. Afify, A. E. Bastawissy, O. M. Hegazy. “A hybrid filtering approach for storage optimization in main-memory cloud database”, Egyptian Informatics Journal, 16(3), ss.329-337, 2015, doi: 10.1016/j.eij.2015.06.007.
• A. T. Kabakus, R. Kara. “A performance evaluation of in-memory databases”, Journal of King Saud University - Computer and Information Sciences, 29(4), ss.520-525, 2017, doi:10.1016/j.jksuci.2016.06.007.
• T. Lahiri, M. A. Neimat, S. Folkman. “Oracle TimesTen: An In-Memory Database for Enterprise Applications”, IEEE Data Eng. Bull., 36(2), ss.6-13, 2013.
• P. Jaroslav. “NoSQL databases: a step to database scalability in web environment”, International Journal of Web Information Systems, 9(1), ss.69-82, 2013.
• P. Chao, D. He, S. Sadiq, K. Zheng, X. Zhou. "A performance study on large-scale data analytics using disk-based and in-memory database systems," 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, ss. 247-254, 2017, doi: 10.1109/BIGCOMP. 2017.7881706
• Y. Wang, G. Zhong, L. Kun, L. Wang, H. Kai, F. Guo. "The Performance Survey of in Memory Database", 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS), Melbourne, Avustralya, ss.815-820, 2015, doi: 10.1109/ICPADS.2015.109.
• S.-Y. Huang, R.-H. Tsaih, F. Yu. “Topological pattern discovery and feature extraction for fraudulent financial reporting”, Expert Syst. Appl. 41 (9) , ss.4360–4372, 2014.
• P. C. González, J.D. Velásquez. “Characterization and detection of taxpayers with false invoices using data mining techniques”, Expert Syst. Appl., 40 (5), ss.1427–1436, 2013.
• S. Jha, M. Guillen, J.C. Westland. “Employing transaction aggregation strategy to detect credit card fraud”, Expert Syst. Appl., 39, ss.12650–12657, 2012.
• D. Olszewski. “A probabilistic approach to fraud detection in telecommunications”, Knowledge-Based Systems, 26, ss.246–258, 2012.
• V. D. Kumar, S. Radhakrishnan. "Intrusion detection in MANET using Self Organizing Map (SOM)", 2014 International Conference on Recent Trends in Information Technology, Chennai, 2014, doi: 10.1109/ICRTIT.2014.6996118.
• W. Khreich, E. Granger, A. Miri, R. Sabourin. “Iterative Boolean combination of classifiers in the ROC space: An application to anomaly detection with HMMs”, Pattern Recognition, 43 (8), ss.2732-2752, 2010, doi: 10.1016/j.patcog.2010.03.006.
• G. Kim, S. Lee, S. Kim. “A novel hybrid intrusion detection method integrating anomaly detection with misuse detection”, Expert Systems with Applications, 41(4), ss.1690-1700, 2014.