Prediction of Failure Categories in Plastic Extrusion Process with Deep Learning

Öz

Today’s manufacturing vision necessitates extracting insights from the data collected in real-time from manufacturing processes. Predicting failures with the predictive analysis of the collected process data and preventing these failures by taking necessary actions before they occur is a key factor in ensuring quality at the desired level, increasing productivity, and reducing costs in production systems. In the literature on predictive analysis of process data, machine learning and deep learning methods have attracted considerable attention, especially in recent years. This study has addressed a multi-class failure classification problem in the plastic extrusion process with a real case study. Classification models have been developed based on Long Short-term Memory (LSTM) as a deep learning method and Multilayer Perceptron (MLP) and Logistic Regression (LR) as machine learning methods to predict the failure categories. In the case study, real data taken from the extrusion process of one of the leading insulation companies operated in Izmir has been used. The final dataset includes actual measurements of seven parameters related to temperature and pressure and failure categories as the target variable. Three failure categories have been identified to define Category 0 (No failure), Category 1 (Filter change), and Category 2 (Feeding failures) states, and coded as 0,1 and 2 in the models, respectively. LSTM, MLP, and LR’s performance to predict the failure categories have been evaluated and compared based on accuracy, precision, recall, and F1 Score measures. LSTM is the highest performing among the three methods, with 100% prediction accuracy for each failure category. On the other hand, LR and MLP have achieved considerable and close results except for Category 1.

Anahtar Kelimeler

• Deep learning
• failure prediction
• machine learning
• plastic extrusion process

Plastik Ekstrüzyon Sürecinde Derin Öğrenme İle Hata Kategorilerinin Tahmini

Öz

Günümüz üretim anlayışı, imalat süreçlerinden gerçek zamanlı olarak toplanan süreç verisinden kestirim yapabilmeyi gerektirmektedir. Toplanan süreç verilerinin kestirimci analizi ile hataların tahmin edilmesi ve gerekli aksiyonların alınmasıyla hataların ortaya çıkmadan önlenmesi, üretim sistemlerinde kalitenin istenilen seviyede sağlanması, verimliliğin artırılması ve maliyetlerin azaltılmasında kilit bir faktördür. Makine öğrenmesi ve derin öğrenme yöntemleri, süreç verilerinin kestirimci analizinde, özellikle son dönemlerde büyük ilgi görmektedir. Bu çalışmada plastik ekstrüzyon sürecinde çok sınıflı hata sınıflandırma problemi bir gerçek hayat örneğiyle ele alınmıştır. Problemin çözümü için derin öğrenme yöntemlerinden Uzun-Kısa Süreli Bellek (LSTM) ve makine öğrenmesi yöntemlerinden Çok Katmanlı Algılayıcı (MLP) ve Lojistik Regresyon (LR) kullanılmıştır. Çalışmanın uygulama kısmında, İzmir'de faaliyet gösteren Türkiye'nin önde gelen yalıtım firmalarından birinin plastik ekstrüzyon sürecinden alınan gerçek veriler kullanılmıştır. Nihai veri seti, süreçten alınan sıcaklık ve basınçla ilişkili yedi parametrenin gerçek ölçümlerini ve hedef değişken olarak hata kategorilerini içermektedir. Modellerde Kategori 0 (Hata yok), Kategori 1 (Filtre değişimi) ve Kategori 2 (Besleme hataları) durumlarını tanımlamak için üç hata kategorisi belirlenmiş ve sırasıyla 0,1 ve 2 olarak kodlanmıştır. LSTM, MLP ve LR'nin hata kategorilerini tahmin etme performansı, tahmin doğruluğu, kesinlik, duyarlılık ve F1 skoru metriklerine göre değerlendirilmiş ve karşılaştırılmıştır. LSTM, her hata kategorisi için %100 tahmin doğruluğu ile en yüksek performansa sahip olmuştur. LR ve MLP, Kategori 1 dışındaki hata kategorileri tahminlerinde başarılı ve birbirine yakın sonuçlar elde etmiştir.

Anahtar Kelimeler

• Derin öğrenme
• Hata tahmini
• Makine öğrenmesi
• plastik ekstrüzyon süreci

Kaynakça

1. Al Rozuq, R. A. M. I., Al Robaidi, A. M. I. N. 2013. Application of neural network ANN to predict XLPE cable in extrusion processes. Journal of Materials Sciences and Applications, 2013.
2. Bandara, K., Bergmeir, C., Smyl, S. 2020. Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach. Expert Systems with Applications, 140, 112896.
3. Cadavid, J. P. U., Lamouri, S., Grabot, B., Pellerin, R., Fortin, A. 2020. Machine learning applied in production planning and control: a state-of-the-art in the era of industry 4.0. Journal of Intelligent Manufacturing, 1-28.
4. Caesarendra, W., Widodo, A., Yang, B. S. 2010. Application of relevance vector machine and logistic regression for machine degradation assessment. Mechanical Systems and Signal Processing, 244, 1161-1171.
5. Cirak, B., Kozan, R. 2009. Prediction of the coating thickness of wire coating extrusion processes using artificial neural network ANN. Modern Applied Science, 37, 52-66.
6. De Menezes, F. S., Liska, G. R., Cirillo, M. A., Vivanco, M. J. 2017. Data classification with binary response through the Boosting algorithm and logistic regression. Expert Systems with Applications, 69, 62-73.
7. Dreiseitl, S., Ohno-Machado, L. 2002. Logistic regression and artificial neural network classification models: a methodology review. Journal of biomedical informatics, 355-6, 352-359.
8. Fallah, N., Mitnitski, A., Rockwood, K. 2011. Applying neural network Poisson regression to predict cognitive score changes. Journal of Applied Statistics, 389, 2051-2062.

9. Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., Schmidhuber, J. 2016. LSTM: A search space odyssey. IEEE transactions on neural networks and learning systems, 2810, 2222-2232.
10. Gyimothy, T., Ferenc, R., Siket, I. 2005. Empirical validation of object-oriented metrics on open source software for fault prediction. IEEE Transactions on Software engineering, 3110, 897-910.
11. Hochreiter, S., Schmidhuber, J. 1997. Long short-term memory. Neural computation, 9(8), 1735-1780.
12. Hore, S., Chatterjee, S., Sarkar, S., Dey, N., Ashour, A. S., Balas-Timar, D., Balas, V. E. 2016. Neural-based prediction of structural failure of multistoried RC buildings. Structural Engineering and Mechanics, 583, 459-473.
13. Hou, T. H. T., Liu, W. L., Lin, L. 2003. Intelligent remote monitoring and diagnosis of manufacturing processes using an integrated approach of neural networks and rough sets. Journal of Intelligent Manufacturing, 142, 239-253.
14. Huang, H. X., Liao, C. M. 2002. Prediction of parison swell in plastics extrusion blow molding using a neural network method. Polymer testing, 217, 745-749.
15. Janssens, O., Slavkovikj, V., Vervisch, B., Stockman, K., Loccufier, M., Verstockt, S., de Walle, R.V. Van Hoecke, S. 2016. Convolutional neural network based fault detection for rotating machinery. Journal of Sound and Vibration, 377, 331-345.
16. Jing, C., Hou, J. 2015. SVM and PCA based fault classification approaches for complicated industrial process. Neurocomputing, 167, 636-642.
17. Jing, L., Zhao, M., Li, P., Xu, X. 2017. A convolutional neural network based feature learning and fault diagnosis method for the condition monitoring of gearbox. Measurement, 111, 1-10.
18. Konar, P., Chattopadhyay, P. 2011. Bearing fault detection of induction motor using wavelet and Support Vector Machines SVMs. Applied Soft Computing, 116, 4203-4211.
19. Kutyłowska, M. 2015. Neural network approach for failure rate prediction. Engineering Failure Analysis, 47, 41-48.
20. Le Thi, H. A., Le, H. M., Phan, D. N., & Tran, B. 2020. Stochastic DCA for minimizing a large sum of DC functions with application to multi-class logistic regression. Neural Networks, 132, 220-231.
21. Liukkonen, M., Hiltunen, T., Havia, E., Leinonen, H., Hiltunen, Y. 2009. Modeling of soldering quality by using artificial neural networks. IEEE Transactions on electronics packaging manufacturing, 322, 89-96.
22. Malhotra, P., Vig, L., Shroff, G., Agarwal, P. 2015., Long short term memory networks for anomaly detection in time series, Proceedings of European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Presses universitaires de Louvain. 22-24 April 2015, pp. 89-94.
23. Malhotra, R., Singh, Y. 2011. On the applicability of machine learning techniques for object oriented software fault prediction. Software Engineering: An International Journal, 11, 24-37.
24. Meyes, R., Donauer, J., Schmeing, A., Meisen, T. 2019. A Recurrent Neural Network Architecture for Failure Prediction in Deep Drawing Sensory Time Series Data. Procedia Manufacturing, 34, 789-797.
25. Moghar, A., Hamiche, M. 2020. Stock market prediction using LSTM recurrent neural network. Procedia Computer Science, 170, 1168-1173.
26. Morariu, C., Răileanu, S., Borangiu, T., Anton, F. 2018, June. A distributed approach for machine learning in large scale manufacturing systems. In International Workshop on Service Orientation in Holonic and Multi- Agent Manufacturing pp. 41-52. Springer, Cham.
27. Nabipour, M., Nayyeri, P., Jabani, H., Shahab, S., Mosavi, A. 2020. Predicting stock market trends using machine learning and deep learning algorithms via continuous and binary data; a comparative analysis. IEEE Access, 8, 150199-150212.
28. Oh, Y., Ransikarbum, K., Busogi, M., Kwon, D., Kim, N. 2019. Adaptive SVM-based real-time quality assessment for primer-sealer dispensing process of sunroof assembly line. Reliability Engineering System Safety, 184, 202-212.
29. Orrù, P. F., Zoccheddu, A., Sassu, L., Mattia, C., Cozza, R., Arena, S. 2020. Machine learning approach using MLP and SVM algorithms for the fault prediction of a centrifugal pump in the oil and gas industry. Sustainability, 12(11), 4776.
30. Quintana, G., Garcia-Romeu, M. L., Ciurana, J. 2011. Surface roughness monitoring application based on artificial neural networks for ball-end milling operations. Journal of Intelligent Manufacturing, 224, 607-617.
31. Razaviarab, N., Sharifi, S., Banadaki, Y. M. 2019. Smart additive manufacturing empowered by a closed-loop machine learning algorithm, In Nano-, Bio-, Info-Tech Sensors and 3D Systems III, International Society for Optics and Photonics, Vol. 10969 2009, p. 109690H.
32. Shao, S. Y., Sun, W. J., Yan, R. Q., Wang, P., Gao, R. X. 2017. A deep learning approach for fault diagnosis of induction motors in manufacturing. Chinese Journal of Mechanical Engineering, 306, 1347-1356.
33. Singh, Y., Kaur, A., Malhotra, R. 2009. Comparative analysis of regression and machine learning methods for predicting fault proneness models. International journal of computer applications in technology, 352-4, 183-193.
34. Smagulova, K., & James, A. P. 2019. A survey on LSTM memristive neural network architectures and applications. The European Physical Journal Special Topics, 228(10), 2313-2324.
35. Tan, Z., Pan, P. 2019. Network Fault Prediction Based on CNN-LSTM Hybrid Neural Network. In 2019 International Conference on Communications, Information System and Computer Engineering CISCE pp. 486-490. IEEE.
36. Tao, F., Qi, Q., Liu, A., Kusiak, A. 2018. Data-driven smart manufacturing. Journal of Manufacturing Systems, 48, 157-169.
37. Venkatesan, P., & Anitha, S. 2006. Application of a radial basis function neural network for diagnosis of diabetes mellitus. Current Science, 91(9), 1195-1199.
38. Wang, J., Ma, Y., Zhang, L., Gao, R. X., Wu, D. 2018. Deep learning for smart manufacturing: Methods and applications. Journal of Manufacturing Systems, 48, 144-156.
39. Ye, Q., Yang, X., Chen, C., Wang, J. 2019. River Water Quality Parameters Prediction Method Based on LSTM-RNN Model. In 2019 Chinese Control And Decision Conference CCDC pp. 3024-3028. IEEE.
40. Yilmaz, I., Kaynar, O. 2011. Multiple regression, ANN (RBF, MLP) and ANFIS models for prediction of swell potential of clayey soils. Expert systems with applications, 38(5), 5958-5966.
41. Zhang, S., Wang, Y., Liu, M., Bao, Z. 2017a. Data-based line trip fault prediction in power systems using LSTM networks and SVM. IEEE Access, 6, 7675-7686.
42. Zhang, Y., Xiong, R., He, H., Liu, Z. 2017b, July. A LSTM-RNN method for the lithuim-ion battery remaining useful life prediction. In 2017 Prognostics and System Health Management Conference PHM-Harbin pp. 1-4. IEEE.
43. Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., Gao, R. X. 2019. Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 115, 213-237.