Application of Machine Learning Methods with Dimension Reduction Techniques for Fault Prediction in Molding Process

Abstract

Significant advances in digital technology and advanced analytical tools have had a substantial impact on the production environment and laid the foundation for Industry 4.0 and intelligent production concepts. Predictive engineering is one of the key pillars of smart manufacturing that necessitates the collection and analysis of real-time data with an anticipatory point of view through advanced analytical techniques. In the literature, machine learning-based methods have received a great deal of attention to extract valuable information from process data for fault detection. In this study, fault prediction problem was addressed in a molding process that includes successive steps by applying machine learning methods with dimension reduction techniques. The techniques of Principal Component Analysis (PCA), and Isometric Feature Mapping (Isomap) were first utilized for dimension reduction. Then, the data was analyzed for fault prediction with several machine learning techniques, namely, Support Vector Machine (SVM), Neural Network (NN), and Logistic Regression (LR). The dataset for our analysis includes sensor data captured during the molding process of a wheel rim manufacturer. Several criteria, including accuracy, area under curve (AUC), Type I, and Type II error, were employed to assess the predictive performance of the methods applied, including and the model variants reinforced with PCA and Isomap. Our study demonstrates that all predictive model variants have performed with high accuracy, ranging between 92.16% (LR) and 98.04% (PCA-NN). PCA and Isomap improved the accuracy and Type-I error measures of all models; however, no such improvement was obtained on the Type-II error rates.

Keywords

• Dimension Reduction,Fault Prediction,Industry 4.0,Machine Learning,Molding Process

Application of Machine Learning Methods with Dimension Reduction Techniques for Fault Prediction in Molding Process

Abstract

Teknolojide yaşanan gelişmeler ve büyük veriyi kullanarak değer yaratmaya imkân sağlayan ileri analitik teknikler, üretim sistemleri üzerinde önemli etkiler yaratmış, Endüstri 4.0 ve akıllı imalat sistemlerinin temelini oluşturmuştur. Akıllı imalat sistemlerinin en kritik yapı taşlarından birini oluşturan kestirimci mühendislik, süreçlerden gerçek zamanlı doğru verinin toplanmasını, bu verinin ileri analitik teknikler uygulanarak öngörücü bir bakış açısıyla analiz edilmesini ve bu sayede değer yaratılmasını gerektirmektedir. Makine öğrenmesi yöntemleri, süreç verileri kullanılarak anlamlı bilgi elde etme ve hata tahmini yapmada yoğun ilgi görmektedir. Bu çalışmada, birbiri ardına gerçekleşen çok sayıda alt süreç içeren döküm sürecinde hata tahminleme problemi, makine öğrenmesi yöntemleri ve boyut indirgeme teknikleri uygulanarak ele alınmıştır. Veri setinin boyutunu indirgemek için öncelikle Temel Bileşenler Analizi (TBA) ve İzometrik Özellik Haritalama (İÖH) teknikleri uygulanmıştır. Hata tahminlemesi için, indirgenmiş veri setine, makine öğrenmesi yöntemlerinden Destek Vektör Makineleri (DVM), Yapay Sinir Ağları (YSA) ve Lojistik Regresyon (LR) uygulanmıştır. Çalışmanın veri setini, bir jant üreticisi firmanın döküm sürecinden elde edilen gerçek ölçüm değerleri oluşturmaktadır. Uygulanan makine öğrenmesi yöntemlerinin performansı ve boyut indirgeme yöntemlerinin bu yöntemlerin performansı üzerindeki etkileri, tahmin doğruluğu, eğri altındaki alan, Tip-1 hata ve Tip-2 hata ölçütleri kullanılarak değerlendirilmiştir. Tahminleyici modellerde doğruluk oranının %92,16 (LR) - 98,04 (TBA-YSA) aralığında elde edildiği görülmüştür. TBA ve İÖH tüm modellerin tahmin doğruluğu ve Tip-1 hata ölçütlerinde iyileşme sağlarken, Tip-2 hata ölçütünde aynı başarıyı yakalayamamıştır.

Keywords

• Boyut İndirgeme,Hata Tahmini,Endüstri 4.0,Makine Öğrenmesi,Döküm Süreci

References

1. T. Niesen, C. Houy, P. Fettke, and P. Loos, “Towards an integrative big data analysis framework for data-driven risk management in industry 4.0,” 49th Hawaii International Conference on System Sciences (HICSS), January 2016, 5065-5074. IEEE.
2. Y. Oh, K. Ransikarbum, M. Busogi, D. Kwon, and N. Kim, “Adaptive SVM-based real-time quality assessment for primer-sealer dispensing process of sunroof assembly line,” Reliability Engineering & System Safety, vol. 184, pp. 202–212, 2019.
3. Q. Qi, and F. Tao, “Digital twin and big data towards smart manufacturing and industry 4.0: 360 degree comparison,”, Ieee Access, vol. 6, pp. 3585-3593, 2018.
4. F. Tao, Q. Qi, A. Liu, and A. Kusiak, “Data-driven smart manufacturing,” Journal of Manufacturing Systems, vol. 48, pp. 157-169, 2018.
5. A. Kusiak, “Smart manufacturing,” International Journal of Production Research., vol. 56, no.1-2, pp. 508-517, 2018.
6. H. N. Dai, H. Wang, G. Xu, J. Wan, and M. Imran, “Big data analytics for manufacturing internet of things: opportunities, challenges and enabling technologies,” Enterprise Information Systems, pp. 1-25, 2019.
7. Z. Li, Y. Wang, and K.-S. Wang, “Intelligent predictive maintenance for fault diagnosis and prognosis in machine centers: Industry 4.0 scenario,” Advances in Manufacturing, vol. 5, no. 4, pp. 377–387, 2017.
8. S. Hou, and Y. Li, “Short-term fault prediction based on support vector machines with parameter optimization by evolution strategy,” Expert Systems with Applications, vol. 36, no. 10, pp. 12383-12391, 2009.

9. I. Santos, J. Nieves, Y. K. Penya, and P. G. Bringas, “Optimising Machine-Learning-Based Fault Prediction in Foundry Production,” Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living Lecture Notes in Computer Science, pp. 554–561, 2009.
10. Q. Zhu, Y. Jia, D. Peng, and Y. Xu, “Study and Application of Fault Prediction Methods with Improved Reservoir Neural Networks,” Chinese Journal of Chemical Engineering, vol. 22, no. 7, pp. 812–819, 2014.
11. H.-Q. Wang, Y.-N. Cai, G.-Y. Fu, M. Wu, and Z.-H. Wei, “Data-driven fault prediction and anomaly measurement for complex systems using support vector probability density estimation,” Engineering Applications of Artificial Intelligence, vol. 67, pp. 1–13, 2018.
12. Y. Bai, Z. Sun, B. Zeng, J. Long, L. Li, J. V. de Oliveira, and C. Li, “A comparison of dimension reduction techniques for support vector machine modeling of multi-parameter manufacturing quality prediction,” Journal of Intelligent Manufacturing, vol. 30, no. 5, pp. 2245-2256, 2019.
13. Z. Zhang, Y. Wang, and K. Wang, “Intelligent fault diagnosis and prognosis approach for rotating machinery integrating wavelet transform, principal component analysis, and artificial neural networks,” The international journal of advanced manufacturing technology, vol. 68, no.(1-4), pp.763-773, 2013.
14. İ. Kabasakal, F. Demircan Keskin, A. Koçak, and H. Soyuer. “A Prediction Model for Fault Detection in Molding Process Based On Logistic Regression Technique,” in Proceedings of the International Symposium for Production Research 2019, August 28-30, 2019. Wien, Austria.
15. T. Wuest, D. Weimer, C. Irgens, and K. D. Thoben, “Machine learning in manufacturing: advantages, challenges, and applications,” Production & Manufacturing Research, vol. 4, no. 1, pp. 23-45, 2016.
16. R. Malhotra. “Comparative analysis of statistical and machine learning methods for predicting faulty modules,” Applied Soft Computing, vol. 21, pp. 286-297, 2014.
17. D. Thukaram, H. P. Khincha, and H. P. Vijaynarasimha, “Artificial neural network and support vector machine approach for locating faults in radial distribution systems,” IEEE Transactions on Power Delivery, vol. 20, no. 2, pp. 710-721, 2005.
18. J. Wang, X. Wu, and C. Zhang, “Support vector machines based on K-means clustering for real-time business intelligence systems,” International Journal of Business Intelligence and Data Mining, vol. 1, no. 1, pp. 54-64, 2005.
19. G. Rubio, H. Pomares, I. Rojas, and L. J. Herrera, “A heuristic method for parameter selection in LS-SVM: Application to time series prediction International Journal of Forecasting, vol. 27, no. 3, pp. 725-739, 2011.
20. Alvi, S. B., Martin, R., and Gottschling, J., “Efficient Use of Hybrid Adaptive Neuro-Fuzzy Inference System Combined with Nonlinear Dimension Reduction Method in Production Processes”, Computer Science & Information Technology (CS & IT), vol. 7, pp. 29-43, 2017.
21. M. W.Craven, and J. W. Shavlik, Using neural networks for data mining,” Future generation computer systems, vol.13, no. (2-3), pp. 211-229, 1997.
22. A. Tavanaei, M. Ghodrati, S. R. Kheradpisheh, T. Masquelier, and A. Maida, “Deep learning in spiking neural networks,” Neural Networks, vol. 111, pp. 47-63, 2018.
23. J. Han, J. Pei, and M. Kamber, “Data mining: concepts and techniques,” Elsevier, 2011
24. S. B. Kotsiantis, I. D. Zaharakis, P. E. Pintelas, Machine learning: a review of classification and combining techniques. Artificial Intelligence Review, vol. 26, no. 3, pp. 159-190, 2006.
25. U. Orhan, M. Hekim, and M. Ozer, “EEG signals classification using the K-means clustering and a multilayer perceptron neural network model,” Expert Systems with Applications, vol. 38, no. 10, pp. 13475-13481, 2011.
26. S. N. Oğulata, C. Şahin, and R. Erol, “Neural network-based computer-aided diagnosis in classification of primary generalized epilepsy by EEG signals,” Journal of medical systems, vol. 33, no. 2, pp. 107-112, 2009.
27. J.E. King, “Binary logistic regression,” in Best Practices in Quantitative Methods, J.W.Osborne, Ed., Sage Publications, USA, pp. 358-384, 2008.
28. F. S. De Menezes, G. R. Liska, M. A. Cirillo, and M. J. Vivanco, “Data classification with binary response through the Boosting algorithm and logistic regression,” Expert Systems with Applications, vol. 69, pp. 62-73, 2017.
29. D. Lieber, M. Stolpe, B. Konrad, J. Deuse, and K. Morik, “Quality prediction in interlinked manufacturing processes based on supervised & unsupervised machine learning,” Procedia CIRP, vol. 7, pp.193-198, 2013
30. H. Abdi, and L. J. Williams, “Principal component analysis,” Wiley interdisciplinary reviews: computational statistics, vol. 2, no. 4, pp. 433-459, 2010.
31. J. B. Tenenbaum, V. De Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science, vol. 290.5500, 2319-2323, 2000.
32. T. Benkedjouh, K. Medjaher, N. Zerhouni, and S. Rechak, “Remaining useful life estimation based on nonlinear feature reduction and support vector regression,” Engineering Applications of Artificial Intelligence., 26(7), 1751-1760, 2013.
33. A. Diez-Olivan, J. Del Ser, D. Galar and B. Sierra, “Data fusion and machine learning for industrial prognosis: Trends and perspectives towards industry 4.0,” Information Fusion, vol. 50, no. 92-111, 2019.
34. X. Gao and J. Hou, “An improved SVM integrated GS-PCA fault diagnosis approach of Tennessee Eastman process,” Neurocomputing, vol. 174, pp. 906–911, 2016.
35. Z. Yin and J. Hou, “Recent advances on SVM based fault diagnosis and process monitoring in complicated industrial processes,” Neurocomputing, vol. 174, pp. 643–650, 2016.
36. H. Lahdhiri, M. Said, K. B. Abdellafou, O. Taouali and M. F. Harkat, “Supervised process monitoring and fault diagnosis based on machine learning methods,” The International Journal of Advanced Manufacturing Technology, vol. 102, no. (5-8), pp. 2321-2337, 2019.
37. W. Mao, L. He, Y. Yan, and J. Wang, “Online sequential prediction of bearings imbalanced fault diagnosis by extreme learning machine,” Mechanical Systems and Signal Processing, vol. 83, pp. 450–473, 2017.
38. P. K. Kankar, S. C. Sharma and S. P. Harsha, “Fault diagnosis of ball bearings using machine learning methods,” Expert Systems with applications, vol. 38, no. 3, pp. 1876-1886, 2011.
39. A. Kusiak and W. Li, “The prediction and diagnosis of wind turbine faults,” Renewable Energy, vol. 36, no. 1, 16-23, 2011.
40. N. R. Sakthivel, B. B. Nair, M. Elangovan, V. Sugumaran, and S. Saravanmurugan, “Comparison of dimensionality reduction techniques for the fault diagnosis of mono block centrifugal pump using vibration signals, Engineering Science and Technology, an International Journal, vol. 17, no.1, pp. 30-38, 2014.
41. X. Jin, F. Yuan, T. W. Chow, and M. Zhao, “Weighted local and global regressive mapping: A new manifold learning method for machine fault classification, ” Engineering Applications of Artificial Intelligence, vol. 30, pp.118-128, 2014.
42. B. H. M. Sadeghi. “A BP-neural network predictor model for plastic injection molding process,” Journal of materials processing technology, vol. 103, no. 3, pp. 411-416, 2000.
43. B. Ribeiro, “Support vector machines for quality monitoring in a plastic injection molding process. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 3 no. 3, pp. 401-410, 2005.
44. S. Kim, and S. Kim, K. R. Ryu, “Deep Learning Experiments with Skewed Data for Defect Prediction in Plastic Injection Molding,” IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA), October 2018, 1-2. IEEE.
45. S. Nasiri, and M. R. Khosravani, “Faults and failures prediction in injection molding process”, The International Journal of Advanced Manufacturing Technology, vol. 103, no. (5-8), pp. 2469-2484, 2019.
46. S. Taghizadeh, A. Özdemir, and O. Uluer, “Warpage prediction in plastic injection molded part using artificial neural network” IJSTM, 37(M2), 2013.
47. T. T. Wong, Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation. Pattern Recognition, vol. 48, no. 9, pp. 2839-2846, 2015.