Emerging Trends in Classification with Imbalanced Datasets: A Bibliometric Analysis of Progression

Year 2022, Volume: 15 Issue: 3, 275 - 288, 31.07.2022

Abdullah Maraş , Çiğdem Erol

https://doi.org/10.17671/gazibtd.1019015

Cited By: 1

Abstract

Imbalanced or unbalanced datasets are defined as the highly skewed distribution of target variable in the field of machine learning. Imbalanced datasets have greatly caught the attention of researchers due to their negative effect on machine learning models in the last decade. Researchers develop various solutions to the problems of imbalanced datasets and contribute to the literature.
The increasing number of articles makes it difficult to follow the literature. Review articles contribute to the solution of this problem. The goal of this study is to conduct a bibliometric analysis to find solutions for classification with imbalanced datasets. Bibliometric analysis is a quantitative technique based on extracting statistics from databases. This work is the first bibliometric analysis to address the problem of imbalanced datasets.
In this study, data on imbalanced datasets were obtained from the Scopus database with the R Bibliometrix package version 3.1.4, and recent studies and new approaches were summarized. Data on 16255 publications between 1957-2021 were collected by using selected keywords. This collection mainly comprises 8871 articles, 6987 conference papers, and 175 reviews with 1, 66 average citations per year per document. Among the most cited countries, the United States has 106139 total citations followed by China with 13839 citations and Germany has 9524 citations.  

Keywords

Imbalanced learning, classification, sampling methods, cost-sensitive learning, evaluation metrics, bibliometric

Dengesiz Veri Kümeleriyle Sınıflandırmada Gelişen Trendler: İlerlemenin Bibliyometrik Analizi

Abstract

Dengesiz veri kümeleri, makine öğrenimi alanında hedef değişkenin oldukça çarpık dağılımı olarak tanımlanmaktadır. Dengesiz veri kümeleri, makine öğrenimi modelleri üzerindeki olumsuz etkilerinden dolayı son on yılda araştırmacıların dikkatini büyük ölçüde çekmiştir. Araştırmacılar dengesiz veri kümeleri sorunlarına çeşitli çözümler geliştirip literatürde paylaşmaktadır.
Artan makale sayısı literatürü takip etmeyi zorlaştırmaktadır. Derleme makaleleri bu sorunun çözümüne katkıda bulunur. Bu çalışmada, dengesiz veri kümeleriyle sınıflandırmadaki çözüm önerilerini bulmak için bibliyometrik bir analiz yapılması amaçlanmaktadır. Bibliyometrik analiz, veri tabanlarından istatistik çıkarmaya dayalı nicel bir tekniktir. Bu çalışma, dengesiz veri kümeleri problemini ele alan ilk bibliyometrik analizi olma niteliğindedir.
Bu çalışmada, Scopus veri tabanından, dengesiz veri kümeleriyle ilgili veri, R Bibliometrix package version 3.1.4 ile elde edilerek son çalışmalar ve yeni yaklaşımlar özetlendi. Seçilen anahtar kelimeler ile 1957-2021 yılları arasında 16255 yayına ilişkin veriler toplandı. Bu koleksiyon temel olarak 8871 makale, 6987 konferans bildirisi ve 175 derlemeden oluşmaktadır ve belge başına atıf sayısı yılda ortalama 1,66’dır. En çok atıf yapılan ülkeler arasında 106139 toplam atıf ile Amerika Birleşik Devletleri'ni, 13839 atıf ile Çin ve 9524 atıf ile Almanya takip etmektedir.

Keywords

dengesiz öğrenim, sınıflandırma, örnekleme yöntemleri, maliyet duyarlı çözüm, değerlendirme metrikleri, bibliyometrik

References

• T. O. Ayodele, “Types of Machine Learning Algorithms”, New Advances in Machine Learning, 3, Yagang Zhang, Intech, Rijeka, Croatia, 19-48, 2010.
• G. E. Melo-Acosta, F. Duitama-Muñoz & J. D. Arias-Londoño, “Fraud Detection in Big Data Using Supervised and Semi-Supervised Learning Techniques”, IEEE Colombian Conference on Communications and Computing (COLCOM), Cartagena, Colombia, 1-6, 2017.
• D. Zhang, H. Huang, Q. Chen, & Y. Jiang, “A Comparison Study of Credit Scoring Models”, Third International Conference on Natural Computation (ICNC 2007), Haikou, China, 1, 15-18, 2007.
• A. Maraş & S. Aydin, “Intercorrelation between Singular Spectrum of EEG Sub-Bands and Emotional States”, National Conference on Electrical, Electronics and Biomedical Engineering (ELECO), Bursa, Turkey, 486-490, 2016.
• H. Asri, H. Mousannif, H. Al Moatassime & T. Noel, “Using Machine Learning Algorithms for Breast Cancer Risk Prediction and Diagnosis”, Procedia Computer Science, 83, 1064-1069, 2016.
• V. A. Kumari & R. Chitra, “Classification of Diabetes Disease Using Support Vector Machine”, International Journal of Engineering Research and Applications, 3(2), 1797-1801, 2013.
• J. Burez & D. Van den Poel, “Handling Class Imbalance in Customer Churn Prediction”, Expert Systems with Applications, 36(3), 4626-4636, 2009.
• M. Anjaria & R. M. R. Guddeti, “A Novel Sentiment Analysis of Social Networks Using Supervised Learning”, Social Network Analysis and Mining, 4(1), 181, 2014.
• R. Caruana & A. Niculescu-Mizil, “An Empirical Comparison of Supervised Learning Algorithms”, Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh Pennsylvania, USA, 161-168, 2006.
• V. García, R. A. Mollineda, & J. S. Sánchez, “A New Performance Evaluation Method for Two-Class Imbalanced Problems”, Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), Orlando, FL, USA, 917-925, 2008.
• Y. Sun, A. K. Wong & M. S. Kamel, “Classification of Imbalanced Data: A Review”, International Journal of Pattern Recognition and Artificial Intelligence, 23(04), 687-719, 2009.
• J. L. Leevy, T. M. Khoshgoftaar, R. A. Bauder & N. Seliya, “A Survey on Addressing High-Class Imbalance in Big Data”, Journal of Big Data, 5(1), 1-30, 2018.
• Internet: Google (2021) Google Trends, http://www.google.com/trends/, 15.08.2021.
• F. Provost, “Machine Learning from Imbalanced Data Sets 101”, Proceedings of the AAAI’2000 Workshop on Imbalanced Data Sets, Austin, Texas, USA, 68, 1-3, 2000.
• N. V. Chawla, N. Japkowicz & A. Koltcz, Proceedings of the ICML’2003 Workshop on Learning from Imbalanced Data Sets, Washington, DC, USA, 2003.
• M. Zięba & J. M. Tomczak, “Boosted SVM with Active Learning Strategy for Imbalanced Data”, Soft Computing, 19(12), 3357-3368, 2015.
• H. He & E. A. Garcia, “Learning from Imbalanced Data”, IEEE Transactions on knowledge and data engineering, 21(9), 1263-1284, 2009.
• S. Wang, Z. Li, W. Chao & Q. Cao, “Applying Adaptive Over-Sampling Technique based on Data Density and Cost-Sensitive SVM to Imbalanced Learning”, The 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, QLD, Australia, 1-8, 2012.
• S. Belarouci & M. A. Chikh, “Medical Imbalanced Data Classification”, Advances in Science, Technology and Engineering Systems Journal, 2(3), 116-124, 2017.
• Y. Yan, T. Yang, Y. Yang & J. Chen, “A Framework of Online Learning with Imbalanced Streaming Data”, Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, California USA, 2817-2823, 2017.
• A. Orriols-Puig & E. Bernadó-Mansilla, “Evolutionary rule-based Systems for Imbalanced Data Sets”, Soft Computing, 13(3), 213, 2009.
• G. M. Weiss, “Mining with Rarity: a Unifying Framework”, ACM Sigkdd Explorations Newsletter, 6(1), 7-19, 2004.
• T. Jo & N. Japkowicz, “Class imbalances versus small disjuncts”. ACM Sigkdd Explorations Newsletter, 6(1), 40-49, 2004.
• R. C. Prati, G. E. Batista & M. C. Monard, “Class Imbalances versus Class Overlapping: An Analysis of a Learning System Behavior”, Mexican International Conference on Artificial Intelligence, Mexico City, Mexico, 312-321, 2004.
• T. M. Khoshgoftaar, J. Van Hulse & A. Napolitano, “Comparing Boosting and Bagging Techniques with Noisy and Imbalanced Data”, IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 41(3), 552-568, 2010.
• A. S. More & D. P. Rana, “Review of Random Forest Classification Techniques to Resolve Data Imbalance”, IEEE 1st International Conference on Intelligent Systems and Information Management (ICISIM), Aurangabad, India, 72-78, 2017.
• J. J. Ng & K. H. Chai, “A Bibliometric Analysis of Project Management Research”, IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), Singapore, 976-980, 2015.
• Internet: Scopus (2022) , https://www.scopus.com/, 26.05.2022.
• F. Machado & C. D. Martes, “Project Management Success: A Bibliometric Analisys”, Revista de Gestão e Projetos-GeP, 6(1), 28-44, 2015.
• M. Aria & C. Cuccurullo, “Bibliometrix: An R-tool for Comprehensive Science Mapping Analysis”, Journal of Informetrics, 11(4), 959-975, 2017.
• N. V. Chawla, K. W. Bowyer, L. O. Hall & W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-Sampling Technique”, Journal of Artificial Intelligence Research, 16, 321-357, 2002.
• H. Han, W. Y. Wang & B. H. Mao, “Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning”, Advances in Intelligent Computing. ICIC. Lecture Notes in Computer Science, Hefei, China, 878-887, 2005.
• N. V. Chawla, A. Lazarevic, L. O. Hall & K. W. Bowyer, “SMOTEBoost: Improving Prediction of the Minority Class in Boosting”, European Conference on Principles of Data Mining and Knowledge Discovery , Cavtat-Dubrovnik, Croatia, 107-119, 2003.
• M. Galar, A. Fernandez, E. Barrenechea, H. Bustince & F. Herrera, “A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-based Approaches”, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4), 463-484, 2011.
• X. Y. Liu, J. Wu & Z. H. Zhou, “Exploratory Undersampling for Class-Imbalance Learning”, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 39(2), 539-550, 2008.
• V. López, A. Fernandez, S. Garcia, V. Palade, & F. Herrera, “An Insight into Classification with Imbalanced Data: Empirical Results and Current Trends on Using Data Intrinsic Characteristics”, Information Sciences, 250, 113-141, 2013.
• R. Akbani, S. Kwek & N. Japkowicz, “Applying support vector machines to imbalanced datasets”, European Conference on Machine Learning, Pisa, Italy, 39-50, 2004.
• K. Veropoulos, C. Campbell & N. Cristianini, “Controlling the sensitivity of support vector machines”, Proceedings of the International Joint Conference on AI, 55–60, 1999.
• C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse & A. Napolitano, “RUSBoost: A Hybrid Approach to Alleviating Class Imbalance”, IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 40(1), 185-197, 2009.
• Z. H. Zhou & X. Y. Liu, “Training Cost-Sensitive Neural Networks with Methods Addressing the Class Imbalance Problem”, IEEE Transactions on Knowledge and Data Engineering, 18(1), 63-77, 2005.
• A. Estabrooks, T. Jo & N. Japkowicz, “A Multiple Resampling Method for Learning from Imbalanced Data Sets”, Computational Intelligence, 20(1), 18-36, 2004.
• Y. Tang, Y. Q. Zhang, N. V. Chawla & S. Krasser, “SVMs Modeling for Highly Imbalanced Classification”, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 39(1), 281-288, 2008.
• C. Bunkhumpornpat, K. Sinapiromsaran & C. Lursinsap, “Safe-Level-Smote: Safe-Level-Synthetic Minority Over-Sampling Technique for Handling the Class Imbalanced Problem”, Pacific-Asia Conference on Knowledge Discovery and Data Mining, Bangkok, Thailand, 475-482, 2009.
• S. Wang, & X. Yao, “Diversity Analysis on Imbalanced Data Sets by Using Ensemble Models”, IEEE Symposium on Computational Intelligence and Data Mining, Nashville, TN, USA 324-331, 2009.
• M. A. Mazurowski, P. A. Habas, J. M. Zurada, J. Y. Lo, J. A. Baker & G. D. Tourassi, “Training Neural Network Classifiers for Medical Decision Making: The Effects of Imbalanced Datasets on Classification Performance”, Neural Networks, 21(2-3), 427-436, 2008.
• M. Galar, A. Fernández, E. Barrenechea, & F. Herrera, “EUSBoost: Enhancing Ensembles for Highly Imbalanced Data-Sets by Evolutionary Undersampling”, Pattern Recognition, 46(12), 3460-3471, 2013.
• A. Fernández, S. Garcia, F. Herrera & N. V. Chawla, “SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary”, Journal of Artificial Intelligence Research, 61, 863-905, 2018.
• I. Triguero, M. Galar, S. Vluymans, C. Cornelis, H. Bustince, F. Herrera & Y. Saeys, “Evolutionary Undersampling for Imbalanced Big Data Classification”, IEEE Congress on Evolutionary Computation (CEC), Sendai, Japan, 715-722, 2015.
• J. A. Sáez, J. Luengo, J. Stefanowski & F. Herrera, “SMOTE–IPF: Addressing the Noisy and Borderline Examples Problem in Imbalanced Classification by a Re-sampling Method with Filtering”, Information Sciences, 291, 184-203, 2015.
• L. Lusa, “Improved Shrunken Centroid Classifiers for High-Dimensional Class-Imbalanced Data”, BMC Bioinformatics, 14(1), 1-13, 2013.
• I. Brown & C. Mues, “An Experimental Comparison of Classification Algorithms for Imbalanced Credit Scoring Data Sets”, Expert Systems with Applications, 39(3), 3446-3453, 2012.
• E. Garfield, “Historiographic Mapping of Knowledge Domains Literature”, Journal of Information Science, 30(2), 119-145, 2004.
• F. Fernández-Navarro, C. Hervás-Martínez, C. García-Alonso & M. Torres-Jiménez, “Determination of Relative Agrarian Technical Efficiency by a Dynamic Over-Sampling Procedure Guided by Minimum Sensitivity”, Expert Systems with Applications, 38(10), 12483-12490, 2011.
• G. Kovács, “An Empirical Comparison and Evaluation of Minority Oversampling Techniques on a Large Number of Imbalanced Datasets”, Applied Soft Computing, 83, 105662, 2019.
• S. Del Río, V. López, J. M. Benítez & F. Herrera, “On the Use of Mapreduce for Imbalanced Big Data Using Random Forest”, Information Sciences, 285, 112-137, 2014.
• L. Yijing, G. Haixiang, L. Xiao, L. Yanan & L. Jinling, “Adapted Ensemble Classification Algorithm based on Multiple Classifier System and feature selection for classifying multi-class imbalanced data”, Knowledge-Based Systems, 94, 88-104, 2016.
• Q. Kang, X. Chen, S. Li & M. Zhou, “A Noise-Filtered Under-sampling Scheme for Imbalanced Classification”, IEEE Transactions on Cybernetics, 47(12), 4263-4274, 2016.
• T. Hasanin & T. Khoshgoftaar, “The Effects of Random Undersampling with Simulated Class Imbalance for Big Data”, IEEE International Conference on Information Reuse and Integration (IRI), Salt Lake City, UT, USA, 70-79, 2018.
• T. Hasanin, T. M. Khoshgoftaar, J. Leevy & N. Seliya, “Investigating Random Undersampling and Feature Selection on Bioinformatics Big Data”, IEEE Fifth International Conference on Big Data Computing Service and Applications (BigDataService), Newark, CA, USA, 346-356, 2019.
• N. B. Abdel-Hamid, S. ElGhamrawy, A. El Desouky & H. Arafat, “A Dynamic Spark-Based Classification Framework for Imbalanced Big Data”, Journal of Grid Computing, 16(4), 607-626, 2018.
• H. Esfahani, K. Tavasoli & A. Jabbarzadeh, “Big Data and Social Media: A Scientometrics Analysis”, International Journal of Data and Network Science, 3(3), 145-164, 2019.
• N. V. Chawla, N. Japkowicz & A. Kotcz, “Special Issue on Learning from Imbalanced Data Sets”, ACM SIGKDD Explorations Newsletter, 6(1), 1-6, 2004.
• S. M. Abd Elrahman & A. Abraham, A Review of Class Imbalance Problem, Journal of Network and Innovative Computing, 1, 332-340, 2013.
• C. Su, S. Ju, Y. Liu & Z. Yu, “Improving Random Forest and Rotation Forest for Highly Imbalanced Datasets”, Intelligent Data Analysis, 19(6), 1409-1432, 2015.
• F. Bulut, “Sınıflandırıcı topluluklarının dengesiz veri kümeleri üzerindeki performans analizleri”, Bilişim Teknolojileri Dergisi, 9(2), 153, 2016.
• A. Fernández, S. García, & F. Herrera, “Addressing the Classification with Imbalanced Data: Open Problems and New Challenges on Class Distribution”, International Conference on Hybrid Artificial Intelligence Systems, Wroclaw, Poland, 1-10, 2011.
• D. J. Dittman, T. M. Khoshgoftaar & A. Napolitano,”The Effect of Data Sampling When Using Random Forest on Imbalanced Bioinformatics Data”, IEEE International Conference on Information Reuse and Integration, San Francisco, CA, USA, 457-463, 2015.