Classifier Selection in Resource Limited Hardware: Decision Analysis and Resolution Approach

Öz

Digitalization, Industry 4.0 and Internet of things (IoT) have become more popular in the recent years. Most of these systems depend on micro-controllers and sensors. These micro-controllers and sensors are mostly cheap, low RAM and low CPU systems; thus, they are resource constrained environments. In this study, a supervised learning classifier comparison technique suitable for resource constrained environments is proposed. This technique, Decision Analysis and Resolution (DAR), is originated in the domain of Software Engineering. First, DAR is explained using an example of car buying scenario. Then 11 off-the-shelf classifiers are compared using DAR for low RAM and less powerful CPU environments in an intrusion detection scenario. This scenario simulated on well-known KDD99 intrusion detection dataset. All the experiments are realized using python scikit-learn package. According to the experiments, Decision Tree classifier is the most suitable to implement for resource constrained environments with a small lead. Results for the other three classifiers (Bagging, Multi Layer Perceptron, Random Forest) are also very similar. To aid the reproducibility of the experiments, the whole source code of the study is provided in the popular open source repository https://github.com/ati-ozgur/classifier-comparison-using-DAR.

Anahtar Kelimeler

• Classifier Selection
• Decision Analysis and Resolution
• Machine Learning
• Performance Metrics
• Resource Limited Environment

Destekleyen Kurum

Jacobs University

Kaynakça

1. Amatriain, X. & Basilico, J. (2012), ‘Netflix recommendations: Beyond the 5 stars (part 1)’, https://netflixtechblog.com/ netflix-recommendations-beyond-the-5-stars-part-1-55838468f429. Axelsson, S. (1999), The base-rate fallacy and its implications for the difficulty of intrusion detection, in ‘In Proceedings of the 6th ACM Conference on Computer and Communications Security’, pp. 1–7. Basheleishvi̇li̇, I., Bardaveli̇dze, A. & Tsiramua, S. (2019), ‘The development of a model for decision support system of assessment and selection of university academic staff’, Journal of Intelligent Systems: Theory and Applications 2(2), 18–23. Faydalı, R. & Erkan, E. F. (2020), ‘Makine seçim probleminin bulanık VIKOR yöntemiyle i̇ncelenmesi’, Journal of Intelligent Systems: Theory and Applications 3(1), 7–12. Çınar, A. & Özer Uygun (2019), ‘Selecting green supplier using intuitionistic fuzzy AHP’, Journal of Intelligent Systems: Theory and Applications 2(2), 24– 31. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M. & Édouard Duchesnay (2011), ‘Scikit-learn: Machine learning in python’, Journal of Machine Learning Research 12(85), 2825–2830. URL: http://jmlr.org/papers/v12/pedregosa11a.html Research, G. V. (2019), ‘Microcontroller market size, share & trends analysis report’, https://www.grandviewresearch.com/industry-analysis/ microcontroller-market. Last Accessed August 2019. Sahingoz, O. K. (2019), ‘A clustering approach for intrusion detection with big data processing on parallel computing platform’, Balkan Journal of Electrical and Computer Engineering 7, 286 – 293. Saleh, N. & Hussein, N. (2019), ‘Artificial intelligence in corneal topography’, Journal of Intelligent Systems: Theory and Applications 2(1), 1–6. Taşcı, E. (2019), ‘A meta-ensemble classifier approach: Random rotation forest’, Balkan Journal of Electrical and Computer Engineering 7, 182 – 187. Team, C. P. (2006), Cmmi for development, version 1.2, Technical report. Wilson, J. & Jungner, G. (1968), Principles and practice of screening for disease, Technical report, World Health Organization. Yavuz, H. S., Çevi̇kalp, H. & Edi̇zkan, R. (2016), ‘A comprehensive comparison of features and embedding methods for face recognition’, Turkish Journal of Electrical Engineering and Computer Science 24, 313 – 340. Yılmaz, A. (2020), ‘Assessment of mutation susceptibility in DNA sequences with word vectors’, Journal of Intelligent Systems: Theory and Applications 3(1), 1–6. Özgür, A. & Erdem, H. (2012), ‘Saldırı tespit sistemlerinde kullanılan kolay erişilen makine Öğrenme algoritmalarının karşılaştırılması’, Bilişim Teknolojileri Dergisi 5, 41–48. URL: http://btd.gazi.edu.tr/dergi/sayi/volume5-2-5.pdf Özgür, A. & Erdem, H. (2016), ‘A review of kdd99 dataset usage in intrusion detection and machine learning between 2010 and 2015’, PeerJ Preprints . Özgür, A. & Erdem, H. (2018), ‘Feature selection and multiple classifier fusion using genetic algorithms in intrusion detection systems’, Journal of the Faculty of Engineering and Architecture of Gazi University 33, 0 – 0. Özgür, A., Nar, F. & Erdem, H. (2018), ‘Sparsity-driven weighted ensemble classifier’, International Journal of Computational Intelligence Systems 11, 962– 978. URL: https://doi.org/10.2991/ijcis.11.1.73