Failure Prediction Using Ensemble Learning: A Comparative Study with Synthetic and Real-World Datasets

Yıl 2025, Cilt: 25 Sayı: 4, 785 - 797, 04.08.2025

Aslı Beyza Çiftpınar , Pelin Kanar , Zeynep Idil Erzurum Cıcek

https://doi.org/10.35414/akufemubid.1571811

Öz

The ability to predict and prevent machine failures is a crucial task for businesses on a global scale at a time of increasing dependence on automation and technology. This paper primarily addressed a novel failure prediction model approach based on ensemble learning. Commonly used machine learning models including Decision Trees, K-Nearest Neighborhood, Support Vector Machines, and Logistic Regression and two different ensemble learning strategies were used: bagging and majority voting. The SZVAV real-life failure dataset provided by Lawrence Berkeley National Laboratory and the AI4I2020 Predictive Maintenance synthetic dataset were utilized to evaluate the performance of the proposed ensemble models. The preprocessing stage included the application of oversampling since there is an imbalance problem in both datasets. In this context, a comparison of three oversampling techniques was also presented for the datasets considered in the study. As a result of the tests, it was seen that the proposed models are superior to individual machine learning methods and Random Forest, which is an ensemble model itself, for the considered datasets. In addition, the proposed ensemble models were compared with the original failure prediction models previously presented in the literature on the AI4I2020 dataset, and it was reported that more successful results are obtained with the proposed approach.

Anahtar Kelimeler

Failure prediction , Machine learning , Ensemble learning , Oversampling

Destekleyen Kurum

Eskisehir Technical University, Turkey Scientific Research Projects Committee

Proje Numarası

22LÖP171

Teşekkür

This study is supported by TUBITAK 2209-A - Research Project Support Programme for Undergraduate Students and Eskisehir Technical University, Turkey Scientific Research Projects Committee (22LÖP171).

Topluluk Öğrenmesi Kullanılarak Arıza Tahmini: Sentetik ve Gerçek Dünya Veri Setleriyle Karşılaştırmalı Bir Çalışma

Öz

Makine arızalarını tahmin etme ve önleme yeteneği, otomasyon ve teknolojiye olan bağımlılığın arttığı bir zamanda küresel ölçekte işletmeler için kritik bir görevdir. Bu çalışma öncelikle topluluk öğrenmeye dayalı özgün bir arıza tahmin modeli yaklaşımını ele almaktadır. Karar Ağaçları, K-En Yakın Komşuluk, Destek Vektör Makineleri ve Lojistik Regresyon dahil olmak üzere yaygın olarak kullanılan makine öğrenmesi modelleri ve iki farklı topluluk öğrenme stratejisi kullanılmıştır: torbalama ve çoğunluk oylaması. Lawrence Berkeley Ulusal Laboratuvarı tarafından sağlanan SZVAV gerçek yaşam arıza veri seti ve AI4I2020 Tahmini Bakım sentetik veri seti, önerilen topluluk modellerinin performansını değerlendirmek için kullanılmıştır. Her iki veri setinde de bir dengesizlik sorunu olduğu için ön işleme aşaması aşırı örnekleme uygulamasını içermektedir. Bu bağlamda, çalışmada ele alınan veri setleri için üç aşırı örnekleme tekniğinin bir karşılaştırması da sunulmuştur. Testler sonucunda, ele alınan verisetleri için önerilen modellerin bireysel makine öğrenmesi yöntemlerinden ve kendisi bir topluluk modeli olan Rastgele Orman'dan üstün olduğu görülmüştür. Ayrıca önerilen topluluk modelleri, AI4I2020 veri seti üzerinden literatürde daha önce sunulan orijinal hasar tahmin modelleri ile karşılaştırılmış ve önerilen yaklaşımla daha başarılı sonuçlar elde edildiği raporlanmıştır.

Anahtar Kelimeler

Arıza tahmini , Makine öğrenmesi , Topluluk öğrenmesi , Aşırı örnekleme

Destekleyen Kurum

Eskişehir Teknik Üniversitesi

Proje Numarası

22LÖP171

Kaynakça

• Akgül, G., Çelik, A. A., Ergül Aydın, Z. and Kamışlı Öztürk, Z., 2020, Hipotiroidi hastalığı teşhisinde sınıflandırma algoritmalarının kullanımı. Bilişim Teknolojileri Dergisi, 13, 255 – 268. https://doi.org/10.17671/gazibtd.710728.
• Andre, A. B., Beltrame, E. and Wainer, J., 2013. A combination of support vector machine and k-nearest neighbors for machine fault detection. Applied Artificial Intelligence, 27, 36–49. https://doi.org/10.1080/08839514.2013.747370
• Arslan, B. and Tiryaki, H., 2020. Prediction of railway switch point failures by artificial intelligence methods. Turkish Journal of Electrical Engineering and Computer Sciences, 28, 1044–1058. https://doi.org/10.3906/elk-1906-66
• Ay, A.K. and Yolacan, E. N., 2022. Yeniden Örnekleme Metotlarının Kredi Kartı Sahtecilik Tespiti için Topluluk Öğrenmesine Kapsamlı Analizi. Afyon Kocatepe Üniversitesi Fen ve Mühendislik Bilimleri Dergisi, 22, 1005-1015. https://doi.org/10.35414/akufemubid.1066453
• Aydın, M. A., 2022. Müşteri kaybı tahmininde sınıf dengesizliği problemi, Politeknik Dergisi, 25, 351 – 360. https://doi.org/10.2339/politeknik.734916
• Baptista, M., Sankararaman, S., Medeiros, I. P. de, Nascimento, C., Prendinger, H. and Henriques, E. M., 2018. Forecasting fault events for predictive maintenance using data-driven techniques and ARMA modeling. Computers Industrial Engineering, 115, 41–53. https://doi.org/10.1016/j.cie.2017.10.033
• Beretta, M., Vidal, Y., Sepulveda, J., Porro, O. and Cusidó, J., 2020. Improved Ensemble Learning for Wind Turbine Main Bearing Fault Diagnosis. Applied Sciences, 11(16), 7523. https://doi.org/10.3390/app11167523
• Bousdekis, A., Lepenioti, K., Apostolou, D. and Mentzas G. 2019, Decision making in predictive maintenance: Literature review and research agenda for industry 4.0. IFAC-PapersOnLine, 52, 607–612. https://doi.org/10.1016/j.ifacol.2019.11.226
• Breiman, L., 2001. Random forests. Machine Learning 45, 5–32. https://doi.org/10.1023/A:1010933404324
• Bukhsh, Z.A., Saeed, A., Stipanovic, A. and Doree, A. G., 2019. Predictive maintenance using tree-based classification techniques: A case of railway switches. Transportation Research Part C: Emerging Technologies, 101, 35–54. https://doi.org/10.1016/j.trc.2019.02.001
• Cakir, M., Guvenc, M. A. and Mistikoglu, S., 2021. The experimental application of popular machine learning algorithms on predictive maintenance and the design of iot based condition monitoring system. Computers Industrial Engineering, 15, 106948. https://doi.org/10.1016/j.cie.2020.106948
• Chen, C. H., Tsung, C. K. and Yu, S. S., 2022. Designing a hybrid equipment-failure diagnosis mechanism under mixed-type data with limited failure samples. Appl Sci, 12, 9286. https://doi.org/10.3390/app12189286
• Chen, C., Zhu, Z. H., Shi, J., Lu, N. and Jiang, B., 2021. Dynamic predictive maintenance scheduling using deep learning ensemble for system health prognostics. IEEE Sensors Journal, 21, 26878–26891. https://doi.org/10.1109/JSEN.2021.3119553.
• Cortes, C. and Vapnik, V., 1995. Support-vector networks. Machine Learning, 20 (3), 273–297. https://doi.org/10.1007/BF00994018
• Demir, I. and Karaboga, H.A., 2021. Modeling mathematics achievement with deep learning methods. Sigma J Eng Nat Sci, 39, 33–410. https://doi.org/10.14744/sigma.2021.00039
• Dundar, D. R. , Saricicek, I., Cinar, E. and Yazici, A., 2021. Kestirimci bakimda makine ogrenmesi: Literatür arastirmasi. ESOGÜ Müh Mim Fak Derg, 29, 256–276. https://doi.org/10.31796/ogummf.873963
• Erzurum Cicek, Z. I. and Kamisli Ozturk, Z, 2022. Prediction of fatal traffic accidents using one-class SVMs: a case study in Eskisehir, Turkey, International Journal of Crashworthiness, 27 (5), 1433–1443. https://doi.org/10.1080/13588265.2021.1959168
• Fan. C., Lin, Y., Piscitelli, M. S., Chiosa, R., Wang, H., Capozzoli, A. and Ma, Y., 2023. Leveraging graph convolutional networks for semi-supervised fault diagnosis of HVAC systems in data-scarce contexts. Building Simulation, 16, 1499-1517. https://doi.org/10.1007/s12273-023-1041-1
• Fan, C., Wu, Q., Zhao, Y. and Mo. L., 2024. Integrating active learning and semi-supervised learning for improved data-driven HVAC fault diagnosis performance. Applied Energy, 356, 122356. https://doi.org/10.1016/j.apenergy.2023.122356
• Fernandes, S., Antunes, M., Santiago, A. R., Barraca, J. P., Gomes, D., Aguiar, R. L., 2020. Forecasting appliances failures: A machine-learning approach to predictive maintenance. Information, 11(4), 208. https://doi.org/10.3390/info11040208.
• Gencer, A., Yumusak, R., Ozcan, E. and Eren, T., 2021. An artificial neural network model for maintenance planning of metro trains. Journal of Polytechnic, 24, 811–820. https://doi.org/10.2339/politeknik.693223
• Genuer, R., Poggi, J.-M. and Tuleau-Malot, C., 2010. Variable selection using random forests, Pattern Recognition Letters, 31, 2225–2236. https://doi.org/10.1016/j.patrec.2010.03.014
• Ghasemkhani, B., Aktas, O. and Birant, D., 2023. Balanced k-star: An explainable machine learning method for internet-of-things- enabled predictive maintenance in manufacturing. Machines, 11, 322. https://doi.org/10.3390/machines11030322
• Gohel, H. A., Upadhyay, H., Lagos, L., Cooper, K. and Sanzetenea, A., 2020. Predictive maintenance architecture development for nuclear infrastructure using machine learning, Nuclear Engineering and Technology, 52, 1436–1442. https://doi.org/10.1016/j.net.2019.12.029.
• Gungor, O., Rosing, T. and Aksanli, B., 2022. Stewart: Sacking ensemble for white-box adversarial attacks towards more resilient data-driven predictive maintenance. Computers in Industry, 140, 103660. https://doi.org/10.1016/j.compind.2022.103660.
• Gungor, O., Rosing, T.S. and Aksanli, B., 2022. Dowell: Diversity-induced optimally weighted ensemble learner for predictive maintenance of industrial internet of things devices. IEEE Internet of Things Journal, 9, 3125–3134. https://doi.org/10.1109/JIOT.2021.3097269.
• Hung, Y.-H., 2021. Improved ensemble-learning algorithm for predictive maintenance in the manufacturing process, Applied Sciences, 11(15), 6832. https://doi.org/10.3390/app11156832.
• Iantovics, L.B. and Enachescu, C., 2022. Method for data quality assessment of synthetic industrial data. Sensors, 22, 1608. https://doi.org/10.3390/s22041608.
• Janssens, O., Loccufier, M. and Van Hoecke, S., 2019. Thermal imaging and vibration-based multisensor fault detection for rotating machinery. IEEE Transactions on Industrial Informatics, 15, 434–444. https://doi.org/10.1109/TII.2018.
• Kaya, Y., Kuncan, F. and Ertunc, H. M., 2022. A new automatic bearing fault size diagnosis using time-frequency images of cwt and deep transfer learning methods. Turkish Journal of Electrical Engineering and Computer Sciences, 30, 1851–1867. https://doi.org/10.55730/1300-0632.3909.
• Khalil, A. F. and Rostam, S., 2024. Machine Learning-based Predictive Maintenance for Fault Detection in Rotating Machinery: A Case Study. Engineering, Technology & Applied Science Research, 14, 13181-13189. https://doi.org/10.48084/etasr.6813
• Khan, P. W., Yeun, C. Y. and Byun, Y. C., 2023. Fault detection of wind turbines using SCADA data and genetic algorithm-based ensemble learning. Engineering Failure Analysis, 148, 107209. https://doi.org/10.1016/j.engfailanal.2023.107209
• Lee, W. J., Wu, H., Yun, H., Kim, H., Jun, M.B., and Sutherland, J. W., 2019. Predictive maintenance of machine tool systems using artificial intelligence techniques applied to machine condition data. Procedia CIRP, 80, 506-511. https://doi.org/10.1016/j.procir.2018.12.019
• Liao, Y., Li, M., Sun, Q. and Li P., 2025. Advanced stacking models for machine fault diagnosis with ensemble trees and SVM. Applied Intelligence, 55, 251. https://doi.org/10.1007/s10489-024-06206-2.
• Matzka, S., 2020. Explainable artificial intelligence for predictive maintenance applications. 2020 Third international conference on artificial intelligence for industries (ai4i). 69–74. https://doi.org/10.1109/AI4I49448.2020.00023.
• Mei, Y., Sun, Y., Li, F., Xu, X., Zhang, A. and Shen, J., 2022. Probabilistic prediction model of steel to concrete bond failure under high temperature by machine learning. Engineering Failure Analysis, 142, 106786. https://doi.org/10.1016/j.engfailanal.2022.106786.
• Mienye, I.D. and Sun, Y., 2022. A survey of ensemble learning: Concepts, algorithms, applications, and prospects. IEEE Access, 10, 99129–99149. https://doi.org/10.1109/ACCESS.2022.3207287.
• Mohammed, R., Rawashdeh, J. and Abdullah, M., 2020. Machine learning with oversampling and undersampling techniques: Overview study and experimental results. 2020 11th International Conference on Information and Communication Systems (ICICS). 243–248. https://doi.org/10.1109/ICICS49469.2020.239556.
• Mota, B., Faria, P. and Ramos, C., 2022. Predictive maintenance for maintenance-effective manufacturing using machine learning approaches. International workshop on soft computing models in industrial and environmental applications. 13– 22. https://doi.org/10.1007/978-3-031-18050-7_2
• Mujib, A. and Djatna, T., 2020 Ensemble learning for predictive maintenance on wafer stick machine using IoT sensor data. 2020 International Conference on Computer Science and Its Application in Agriculture (ICOSICA). 1-5. https://doi.org/10.1109/ICOSICA49951.2020.9243180.
• Patra, K., Sethi, R. N. and Behera, D. K., 2022. Anomaly detection in rotating machinery using autoencoders based on bidirectional LSTM and GRU Neural Networks. Turkish Journal of Electrical Engineering and Computer Sciences, 30, 1637–1653. https://doi.org/10.55730/1300-0632.3870
• Phillips, J., Cripps, E., Lau, J. W. and Hodkiewicz, M., 2015. Classifying machinery condition using oil samples and binary logistic regression. Mechanical Systems and Signal Processing, 60-61, 316–325. https://doi.org/10.1016/j.ymssp.2014.12.020
• Raschka, S., 2015. Python Machine Learning, Packt Publishing Ltd. Raza, J., Liyanage, J. P., Al Atat, H. and Lee, J., 2010. A comparative study of maintenance data classification based on neural networks, logistic regression and support vector machines. Journal of Quality in Maintenance Engineering, 16, 303–318. https://doi.org/10.1108/13552511011072934.
• Saihood, Q. and Sonuc E., 2023. A practical framework for early detection of diabetes using ensemble machine learning models. Turkish Journal of Electrical Engineering and Computer Sciences, 31, 722–738. https://doi.org/10.55730/1300-0632.4013.
• Shamayleh, A. J. F. and Awad, M., 2020. Iot based predictive maintenance management of medical equipment. J Med Syst., 44, 72. https://doi.org/10.1007/s10916-020-1534-8.
• Shaheen, A., Hammad, M., Elmedany, W., Ksantini, R. and Sharif, S., 2023. Machine failure prediction using joint reserve intelligence with feature selection technique. International Journal of Computers and Applications, 45, 638–646. https://doi.org/10.1080/1206212X.2023.2260619.
• Shashidhar Kaparthi, D. B., 2020. Designing predictive maintenance systems using decision tree-based machine learning techniques. International Journal of Quality & Reliability Management, 37, 4, 659-686. https://doi.org/10.1108/IJQRM-04-2019-0131.
• Torcianti, A. and Matzka, S., 2021. Explainable artificial intelligence for predictive maintenance applications using a local surrogate model. 2021 4th International conference on artificial intelligence for industries (ai4i), Laguna Hills, CA, USA, 86–88. https://doi.org/10.1109/AI4I51902.2021.00029.
• Vuttipittayamongkol, P. and Arreeras, T., 2022. IEEE Data-driven industrial machine failure detection in imbalanced environments. 2022 IEEE international conference on industrial engineering and engineering management (IEEM), Kuala Lumpur, Malaysia, 1224–1227. https://doi.org/10.1109/IEEM55944.2022.9989673
• Wu, H., Huang, A. and Sutherland, J. W., 2020. Avoiding environmental consequences of equipment failure via an LSTM-based model for predictive maintenance. Procedia Manufacturing, 43, 666–673. https://doi.org/10.1016/j.promfg.2020.02.131
• Zhang, L., Cheng, Y., Zhang, J., Chen, H., Cheng, H., and Gou, W., 2023. Refrigerant charge fault diagnosis strategy for VRF systems based on stacking ensemble learning. Building and Environment, 234, 10209. https://doi.org/10.1016/j.buildenv.2023.110209.
• Zhang, M., Ge, W., Tang, R. and Liu, P., 2023. Hard Disk Failure Prediction Based on Blending Ensemble Learning. Applied Sciences, 13, 3288. https://doi.org/10.3390/app13053288.
• Zhu, T., Ran, Y. and Wen, Y., 2019. A Survey of Predictive Maintenance: Systems, Purposes and Approaches- Arxiv.org. https://doi.org/10.48550/arXiv.1912.07383
• Zonta, T. , Costa, C. A. da, Rosa Righi, R. da, Lima, M. J. de, Trindade, E. S. da and Li, G. P., 2020. Predictive maintenance in the industry 4.0: A systematic literature review. Computers Industrial Engineering, 150, 106889. https://doi.org/10.1016/j.cie.2020.106889.
• AI4I 2020 Predictive Maintenance Dataset [Dataset]. (2020). UCI Machine Learning Repository. https://doi.org/10.24432/C5HS5C.
• Granderson, J. G. L., 2019. Inventory of datasets for afdd evaluation, https://data.openei.org/files/910/lbnldatasynthesisinventory.pdf