Computer Vision Based AutoML Platform

Abstract

The rapid increase in data production, thanks to technological developments and scientific research, leads to the development of Machine Learning (ML) and similar new data analysis tools. It was announced that Amazon Web Services (AWS), a cloud service provider, stored 500EB of data in 2021 [1]. ML is an alternative to traditional engineering methods and does not require field knowledge of the problem to obtain a solution. However, the implementation of ML Algorithms can be complex depending on the content of the data set, and expert knowledge is the most important factor to use these algorithms effectively. Various methods have been developed to find a solution to this problem. There are many different areas and problems that machine learning can be applied to. We have limited our research to problems that can be solved using computer vision and AutoML. We have used AutoML and computer vision-based solutions to solve object classification, detection and segmentation problems. Our goal is to develop a platform that will work without the intervention of any expert. Users can load their datasets, choose the method they want, and train their models according to the problem they choose without any other intervention. After the training process is over, they can use their models in real time by transferring them over the platform in real time with their own hardware.

Keywords

• AutoML
• Deep Learning
• Computer Vision
• Machine Learning
• Object Classification
• Object Detection

Thanks

This study was carried out within the scope of the thesis of the National Defence University, Atatürk Institute of Strategic Studies and Graduate Education.

Bigisayarlı Görü Tabanlı AutoML Platformu

Abstract

Teknolojik gelişmeler ve bilimsel araştırmalarla sayesinde veri üretimindeki hızlı artış, Makine Öğrenimi (ML) vb. yeni veri analiz araçlarının geliştirilmesine neden olmaktadır. Bir bulut servis sağlayıcısı olan Amazon Web Hizmetleri’nin(AWS) sadece 2021 yılında 500EB’lik veri depolandığı açıklandı. ML, geleneksel mühendislik yöntemlerine bir alternatiftir ve çözüm elde etmek için sorunun saha bilgisini gerektirmez. Bununla birlikte, ML Algoritmaları uygulanması veri setinin içeriğine göre kompleks olabilmektedir ve bu algoritmaları etkin bir şekilde kullanmak için uzman bilgisi en önemli etkendir. Bu soruna çözüm bulmak için çeşitli yöntemler geliştirilmiştir. Makine öğreniminin uygulanabileceği birçok farklı alan ve sorun bulunmaktadır. Çalışmada bilgisayarlı görü ve AutoML kullanılarak çözüm elde edilebilemek hedeflenmiştir. Bu anlamda çalışmada obje sınıflandırma, tespit etme ve segmentasyon sorunlarını çözmek için AutoML ve bilgisayarlı görü tabanlı çözümler kullanılmıştır. Hedefimiz, herhangi bir uzmanın müdahelesi olmadan çalışacak bir platform geliştirmektir. Kullanıcılar verisetlerini yükleyip, istedikleri yöntemi seçip ve başka hiçbir müdahale de bulunmadan seçtikleri sorun özellinde modellerini eğitebilemektedirler. Eğitim süreci bittikten sonra, kendi donanımlarıyla gerçek zamanlı bir şekilde platform üzerinden aktarım yapıp modellerini gerçek zamanlı bir şekilde kullanabilmektedirler.

Keywords

• AutoML
• Bilgisayarlı Görü
• Derin Öğrenme
• Makine Öğrenmesi
• Nesne Sınıflandırma

References

1. Adadi A. A survey on data‐efficient algorithms in big data era. Journal of Big Data 2021; 24: 8(1).
2. Borgi T, Zoghlami N, Abed M, Naceur, MS. Big data for operational efficiency of transport and logistics: a review. In 2017 6th IEEE International conference on Advanced Logistics and Transport (ICALT) 2017; pp. 113-120.
3. Simeone O. A very brief introduction to machine learning with applications to communication systems. IEEE Transactions on Cognitive Communications and Networking 2018; 4(4): 648-664.
4. Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D. A survey of methods for explaining black box models. ACM computing surveys (CSUR) 2018; 51(5): 1-42.
5. Buhrmester V, Münch D, Arens, M. Analysis of explainers of black box deep neural networks for computer vision: A survey. Machine Learning and Knowledge Extraction 2021; 3(4): 966-989.
6. Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, ... & Farhan L. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. Journal of big Data 2021; 8: 1-74..
7. Lowe DG. Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference on computer vision 1999; 2: 1150-1157.
8. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Communications of the ACM 2017İ 60(6): 84-90.

9. Hospedales T, Antoniou A, Micaelli P, Storkey A. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence 2021; 44(9): 5149-5169.
10. Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, ... & He Q. A comprehensive survey on transfer learning. Proceedings of the IEEE 2020; 109(1): 43-76.
11. Zeng Y, Zhang J. A machine learning model for detecting invasive ductal carcinoma with Google Cloud AutoML Vision. Computers in biology and medicine 2020; 122: 103861.
12. Marcu D, Mirela D. Sentiment Analysis From Images-Comparative Study of SAI-G and SAI-C Models' Performances Using AutoML Vision Service from Google Cloud and Clarifai Platform. International Journal of Computer Science & Network Security 2021; 21(9): 179-184.
13. Bottou L, Cortes C, Denker JS, Drucker H, Guyon I, Jackel LD, ... & Vapnik V. Comparison of classifier methods: a case study in handwritten digit recognition. In Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3-Conference C: Signal Processing 1994; 2: 77-82.
14. Yang J, Shi R, Ni B. Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) 2021: pp. 191-195.
15. Singh A, Amutha J, Nagar J, Sharma S, Lee CC. AutoML-ID: Automated machine learning model for intrusion detection using wireless sensor network. Scientific Reports 2022; 12(1): 9074.
16. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference 2015; Part III 18: pp. 234-241.
17. Saikia T, Marrakchi Y, Zela A, Hutter F, Brox T. Autodispnet: Improving disparity estimation with automl. In Proceedings of the ieee/cvf international conference on computer vision 2019: pp. 1812-1823.
18. Gijsbers P, Bueno ML, Coors S, LeDell E, Poirier S, Thomas J, ... & Vanschoren J. Amlb: an automl benchmark. arXiv preprint 2022; arXiv:2207.12560.
19. LeDell E, Poirier S. H2o automl: Scalable automatic machine learning. In Proceedings of the AutoML Workshop at ICML 2020.
20. Vikhar PA. Evolutionary algorithms: A critical review and its future prospects. In 2016 International conference on global trends in signal processing, information computing and communication (ICGTSPICC) 2016: pp. 261-265.
21. Liang J, Meyerson E, Hodjat B, Fink D, Mutch K, Miikkulainen R. Evolutionary neural automl for deep learning. In Proceedings of the Genetic and Evolutionary Computation Conference 2019: pp. 401-409.
22. Dridi S. Reinforcement Learning-A Systematic Literature Review 2022.
23. He Y, Lin J, Liu Z, Wang H, Li LJ, Han S. Amc: Automl for model compression and acceleration on mobile devices. In Proceedings of the European conference on computer vision (ECCV) 2018: pp. 784-800.
24. Galanopoulos A, Ayala-Romero JA, Leith DJ, Iosifidis G. AutoML for video analytics with edge computing. In IEEE INFOCOM 2021-IEEE Conference on Computer Communications 2021: pp. 1-10.
25. Bolhasani H, Jassbi SJ. Deep learning accelerators: a case study with MAESTRO. Journal of Big Data 2020; 7, 1-11.
26. Gupta S, Akin B. Accelerator-aware neural network design using automl. arXiv preprint 2020: arXiv:2003.02838.
27. Forsyth DA, Mundy JL, di Gesú V, Cipolla R, LeCun Y, Haffner P, ... & Bengio Y. Object recognition with gradient-based learning. Shape, contour and grouping in computer vision 1999: 319-345.
28. Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images 2009.
29. Fukushima K. Cognitron: A self-organizing multilayered neural network. Biological cybernetics, 1975; 20(3-4): 121-136.
30. Shah A, Kadam E, Shah H, Shinde S, Shingade S. Deep residual networks with exponential linear unit. In Proceedings of the third international symposium on computer vision and the internet 2016: pp. 59-65.
31. Gholamalinezhad H, Khosravi H. Pooling methods in deep neural networks, a review. arXiv preprint 2022; arXiv:2009.07485.
32. Redmon J, Farhadi A. Yolov3: An incremental improvement. arXiv preprint 2018; arXiv:1804.02767.
33. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition 2016: pp. 779-788.
34. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, ... & Zitnick CL. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference 2014; 6(12), pp. 740-755.
35. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition 2017: pp. 2117-2125.
36. Zhong Y, Wang J, Peng J, Zhang L. Anchor box optimization for object detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2020: pp. 1286-1294.
37. Yolov3 Weights Retrieved January 2, 2023 from https://pjreddie.com/darknet/yolo/
38. Ma Y, Mosskull A, Xiang A. 3D Semantic Segmentation for Autonomous Cars.
39. Zhang Z, Fidler S, Urtasun R. Instance-level segmentation for autonomous driving with deep densely connected mrfs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016: pp. 669-677.
40. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, ... & Schiele B. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition 2016: pp. 3213-3223.
41. Yuan Y, Chen X, Wang J. Object-contextual representations for semantic segmentation. In Computer Vision–ECCV 2020: 16th European Conference, 2020; 6(16): pp. 173-190.
42. Yan H, Zhang C, Wu M. Lawin transformer: Improving semantic segmentation transformer with multi-scale representations via large window attention 2022; arXiv:2201.01615.
43. Choi S, Kim JT, Choo J. Cars can't fly up in the sky: Improving urban-scene segmentation via height-driven attention networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2020: pp. 9373-9383.
44. Dumoulin V, Visin F. A guide to convolution arithmetic for deep learning 2016; arXiv:1603.07285.
45. UNet Model Retrieved February 10, 2023 from: https://github.com/hamdaan19/UNet-Multiclass
46. Yolov3 Pytorch İmplementation Retrieved January 2, 2023: github. GitHub Retrieved from: https://github.com/eriklindernoren /PyTorch-YOLOv3