ENHANCING MULTI-CLASS TEXT CLASSIFICATION WITH APRIORI-BASED FEATURE SELECTION

Abstract

: In the field of Natural Language Processing, selecting the right features is crucial for reducing unnecessary model complexity, speeding up training, and improving the ability to generalize. However, the multi-class text classification problem makes it challenging for models to generalize well, which complicates feature selection. This paper investigates how feature selection impacts model performance for multi-class text classification, using a dataset of projects completed by TÜBİTAK TEYDEB between 2009 and 2022. The study employs LSTM, a deep learning method, to classify the projects into nine different industries based on various attributes. The paper proposes a new feature selection approach based on the Apriori algorithm, which reduces the number of attribute combinations considered and makes model training more efficient. Model performance is evaluated using metrics like accuracy, loss, validation scores, and test scores. The key findings are that feature selection significantly affects model performance, and different feature sets have varying impacts on performance.

Keywords

• Feature Selection
• LSTM
• NLP
• Text Classification

References

1. Dogra, V., Singh, A., Verma, S., Kavita, Jhanjhi, N. Z., Talib, M. N., Understanding of data preprocessing for dimensionality reduction using feature selection techniques in text classification, in: Intelligent Computing and Innovation on Data Science: Proceedings of ICTIDS, Springer, Singapore, pp. 455-464, 2021.
2. Thirumoorthy, K., Muneeswaran, K., “Feature selection for text classification using machine learning approaches.” National Academy Science Letters, 45(1), 51-56, 2022.
3. Amazal, H., Ramdani, M., Kissi, M. (2020). “Towards a feature selection for multi-label text classification in big data.” Proceedings of Smart Applications and Data Analysis: Third International Conference, Marrakesh, Morocco, June 25–26, 2020, pp. 187-199
4. Naik, D. A., Mythreyan, S., Seema, S., “Relevance Feature Discovery in Text Mining Using NLP”. in: 3rd International Conference for Emerging Technology, IEEE, pp. 1-6, 2022.
5. Dowlagar, S., Mamidi, R. “Does a Hybrid Neural Network Based Feature Selection Model Improve Text Classification?”, arXiv preprint arXiv:2101.09009, 2021. https://doi.org/10.48550/arXiv.2101.09009
6. Hussain, S. F., Babar, H. Z. U. D., Khalil, A., Jillani, R. M., Hanif, M., Khurshid, K. “A fast non-redundant feature selection technique for text data”, IEEE Access, 8, 181763-181781, 2020.
7. Belkarkor, S., Hafidi, I., Nachaoui, M., Feature Selection for Text Classification Using Genetic Algorithm, in: International Conference of Machine Learning and Computer Science Applications, Cham: Springer Nature Switzerland, pp. 69-80, 2022.
8. Zheng, W., “A comparative study of feature selection methods.” International Journal on Natural Language Computing, 7(5), 01-09, 2018.

9. Xiaochuan, T., Xiaochuan, T., Yuanshun, D., Yanping, X., “Feature selection based on feature interactions with application to text categorization.” Expert Systems with Applications, 120, 207-216, 2019. doi: 10.1016/J.ESWA.2018.11.018
10. TÜBİTAK TEYDEP Proje Değerlendirme ve İzleme Sistemi (2023, Nov. 23). Tamamlanmış Proje Sorgula [Online]. Available: https://eteydeb.tubitak.gov.tr/
11. re — Regular expression operations. (2024, Jan. 05). Regular expression operations [Online]. Available: https://docs.python.org/3/library/re.html
12. Van Houdt, G., Mosquera, C., Nápoles, G., “A review on the long short-term memory model”, Artificial Intelligence Review, 53(8), 5929–5955, 2020. https://doi.org/10.1007/s10462-020-09838-1
13. PO, U., Udanor, C. N., Bakpo, F. S., “Deep Learning Algorithms for predicting the geographical locations of Pandemic Disease Patients from Global Positioning System (GPS) Trajectory Datasets”. https://doi.org/10.21203/rs.3.rs-2770308/v1
14. Rainio, O., Teuho, J., Klén, R. “Evaluation metrics and statistical tests for machine learning”. Scientific Reports, 14(1), 6086. 2024.