KPI based performance estimation in production systems using deep learning techniques

Öz

Firms in the manufacturing sector need to constantly monitor their performance in order to maintain their development under competitive conditions in the market. In this study, eleven KPIs are determined to measure the production performance by taking into account the factory assets. The proposed system is designed in which the relevant KPIs are obtained via the instantaneous data received from the CNC machines in a production system. The main objective of this study is to measure and estimate production performance. In this way, it is aimed to provide a proactive approach for the assets whose performance is monitored by the decision-makers. LSTM and LightGBM models which are deep learning techniques are proposed for the estimation of performance indicators. The approximately three-month time series OEE (Overall Equipment Effectiveness) values of the sample CNC machine are used for estimation. The estimation performance of methods is compared over performance metrics (MSE, MAE, etc.). The results indicated that LightGBM outperforms LSTM for all performance metrics.

Anahtar Kelimeler

• Key performance indicators
• Performance estimation
• Deep learning
• LSTM
• LightGBM

Proje Numarası

1170452

Derin öğrenme teknikleri kullanılarak üretim sistemlerinde KPI tabanlı performans tahminleme

Öz

İmalat sektöründe yer alan firmaların, piyasadaki rekabet koşullarında gelişimlerini sürdürebilmeleri için performanslarını sürekli izlemeleri gerekmektedir. Bu çalışmada, fabrika varlıkları dikkate alınarak üretim performansını ölçmek için on bir adet anahtar performans göstergesi belirlenmiştir. Önerilen sistem, bir üretim sistemindeki CNC makinelerinden alınan anlık veriler ile ilgili KPI'ların elde edildiği yapıda tasarlanmıştır. Bu çalışmanın temel amacı, üretim performansını ölçmek ve bir sonraki değerlerini tahmin etmektir. Bu sayede karar vericiler tarafından performansı izlenen varlıklara proaktif bir yaklaşım sağlanması amaçlanmaktadır. Performans göstergelerinin tahmin edilmesi için derin öğrenme teknikleri olan LSTM ve LightGBM modelleri önerilmiştir. Tahminleme için örnek bir CNC makinesinin yaklaşık üç aylık zaman serisi OEE (Toplam Ekipman Etkinliği) değerleri kullanılmıştır. Yöntemlerin tahmin performansları, çeşitli metrikler (MSE, MAE vb.) üzerinden karşılaştırılmıştır. Sonuçlar, LightGBM'nin tüm performans ölçümleri için LSTM'den daha iyi performans gösterdiğini göstermiştir

Anahtar Kelimeler

• Anahtar performans göstergeleri
• Performans tahminleme
• Derin öğrenme
• LSTM
• LightGBM

Destekleyen Kurum

Türkiye Bilimsel ve Teknolojik Araştırma Kurumu'nun (TÜBİTAK)

Proje Numarası

1170452

Teşekkür

Bu çalışmada kullanılan veriler, Türkiye Bilimsel ve Teknolojik Araştırma Kurumu'nun (TÜBİTAK) 1170452 no'lu “Otomotiv Endüstrisi için Akıllı Üretim Yönetim Sistemi-IOTOPRO” proje çerçevesinde elde edilmiştir.

Kaynakça

1. 1. Domínguez, E., Pérez, B., Rubio, Á. L., Zapata, M. A., A taxonomy for key performance indicators management, Computer Standards & Interfaces, 64, 24-40, 2019.
2. 2. Samir, K., Khabbazi, M. R., Maffei, A., Onori, M. A., Key performance indicators in cyber-physical production systems, Procedia CIRP, 72, 498-502, 2018.
3. 3. Senkuvienė, I., Jankauskas, K., Kvietkauskas, H., Using manufacturing measurement visualization to improve performance, Mechanics, 20 (1), 99-107, 2014.
4. 4. Uddin, M. K., Puttonen, J., Martinez Lastra, J. L., Context-sensitive optimisation of the key performance indicators for FMS, International Journal of Computer Integrated Manufacturing, 28 (9), 958-971, 2015.
5. 5. Riexinger, G., Holtewert, P., Bruns, A., Wahren, S., Tran, K., Bauernhansl, T., KPI-focused simulation and management system for eco-efficient design of energy-intensive production systems, Procedia CIRP, 29, 68-73, 2015.
6. 6. Küçükaltan, B., Irani, Z., Aktas, E., A decision support model for identification and prioritization of key performance indicators in the logistics industry, Computers in Human Behavior, 65, 346-358, 2016.
7. 7. Wohlers, B., Dziwok, S., Schmelter, D., Lorenz, W., Improving Quality Control of Mechatronic Systems Using KPI-Based Statistical Process Control, International Conference on Applied Human Factors and Ergonomics, 398-410, 2018.
8. 8. Skylakha, S., Sakthivel, P., Arunselvan, K. S., Empirical study on application of machine learning techniques for resource allocation in health care using KPI, The Journal of Supercomputing, 76 (4), 2266-2274, 2020.

9. 9. Sikora, M., Szczyrba, K., Wróbel, Ł., Michalak, M., Monitoring and maintenance of a gantry based on a wireless system for measurement and analysis of the vibration level, Eksploatacja i Niezawodność, 21 (2), 341-350, 2019.
10. 10. Ma, Z., Zeng, H., Guo, J., Gu, T., Mao, S., & Yang, T., The application of CNN-LightGBM algorithm in remaining useful life prediction, In 2020 7th International Conference on Information Science and Control Engineering (ICISCE), 1411-1418, 2020.
11. 11. Wang, J., Zhang, J., & Wang, X., Bilateral LSTM: A two-dimensional long short-term memory model with multiply memory units for short-term cycle time forecasting in re-entrant manufacturing systems, IEEE Transactions on Industrial Informatics, 14 (2), 748-758, 2017.
12. 12. Essien, A., Giannetti, C., A deep learning model for smart manufacturing using convolutional LSTM neural network autoencoders, IEEE Transactions on Industrial Informatics, 16 (9), 6069-6078, 2020.
13. 13. Shehadeh, A., Alshboul, O., Al Mamlook, R. E., Hamedat, O., Machine learning models for predicting the residual value of heavy construction equipment: An evaluation of modified decision tree, LightGBM, and XGBoost regression, Automation in Construction, 129, 103827, 2021.
14. 14. Wang, X., Xu, N., Meng, X., Chang, H., Prediction of Gas Concentration Based on LSTM-LightGBM Variable Weight Combination Model, Energies, 15 (3), 827, 2022.
15. 15. Cao, Y., & Gui, L., Multi-step wind power forecasting model using LSTM networks, similar time series and LightGBM, In 2018 5th International Conference on Systems and Informatics (ICSAI), 192-197, 2018.
16. 16. He, Z., Yu, S., Application of LightGBM and LSTM combined model in vegetable sales forecast, In Journal of Physics: Conference Series, 1693 (1), 012110, 2020.
17. 17. Weng, T., Liu, W., Xiao, J., Supply chain sales forecasting based on lightGBM and LSTM combination model, Industrial Management & Data Systems, 265-279, 2019.
18. 18. Ganatra, N., Patel, A., A comprehensive study of deep learning architectures, applications and tools, International Journal of Computer Sciences and Engineering, 6 (12), 701-705, 2018.
19. 19. Doğan, F., Türkoğlu, İ., Derin öğrenme algoritmalarının yaprak sınıflandırma başarımlarının karşılaştırılması, Sakarya University Journal of Computer and Information Sciences, 1 (1), 10-21, 2018.
20. 20. Şeker, A., Diri, B., Balık, H. H., Derin öğrenme yöntemleri ve uygulamaları hakkında bir inceleme, Gazi Mühendislik Bilimleri Dergisi, 3 (3), 47-64, 2017.
21. 21. Siami-Namini, S., Namin, A. S., Forecasting economics and financial time series: ARIMA vs. LSTM, arXiv preprint arXiv:1803.06386, 2018.
22. 22. Burcu, C., 2019, LSTM ağları ile türkçe kök bulma, Bilişim Teknolojileri Dergisi 12 (3), 183-193, 2019.
23. 23. Ju, Y., Sun, G., Chen, Q., Zhang, M., Zhu, H., Rehman, M. U., A model combining convolutional neural network and LightGBM algorithm for ultra-short-term wind power forecasting, Ieee Access, 7, 28309-28318, 2019.
24. 24. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Liu, T. Y., Lightgbm: A highly efficient gradient boosting decision tree, Advances in neural information processing systems, 30, 2017.
25. 25. The European Standard EN 15341:2007, Maintenance Key Performance Indicators, British Standards Institution, 2007.
26. 26. Klimberg, R. K., Sillup, G. P., Boyle, K. J., Tavva, V. (2010). Forecasting performance measures–what are their practical meaning?, In Advances in business and management forecasting, 7, 137-147, 2010.
27. 27. Chicco, D., Warrens, M. J., Jurman, G., The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation, PeerJ Computer Science, 7, 623, 2021.