Metin Ağırlıklandırma Şemalarının NMF-Tabanlı Konu Analizi Alanında Başarımlarının Karşılaştırılması

Abstract

Günümüzde üstel bir şekilde üretilen metin verisinin analiz edilebilmesi, bu verinin belirli bir ağırlıklandırma yaklaşımı ile büyük boyutlu seyrek bir matrise çevrilmesi ile mümkün olmaktadır. İdeal bir metin ağırlıklandırma yaklaşımının 3 temel bileşeni bulunmakta olup; bunlar Terim Frekansı, Doküman Frekansı ve Vektör Normalizasyonu bileşenleridir. Bu üç bileşenin çarpımı ile bir kelimenin bir metin için önemini ifade eden sayısal değerler elde edilir. Ancak bu değerlerin ham hali metin verisinin anlamsal olarak analiz edilmesi için uygun değildir. Bu amaçla çeşitli yöntemler bulunmakta olup, büyük metin koleksiyonlarında bahsedilen konuları bulmayı amaçlayan Konu Analizi bu yöntemlerden bir tanesidir. Bu amaçla konu analizinde bir girdi matrisini hem rastgele hem de deterministik başlangıç değeri ile iki veya daha fazla matrisin çarpımına dönüştürmeyi hedefleyen Negatif Olmayan Matris Ayrışımı (NMF) yönteminden sıklıkla faydalanılır. Bu çalışmada, literatürde kullanılan metin ağırlıklandırma yöntemlerinin ve başlangıç değer yaklaşımlarının NMF yöntemi üzerinde etkilerinin bulunması amacıyla, Vikipedi özgür internet ansiklopedisinden elde edilen 20.000 makale üzerinde denemeler yapılmıştır. Denemelerde kullanılacak küme sayısının elde edilmesi için analitik bir yöntem yardımıyla bir üst sınır kullanılmıştır. Elde edilen sonuçlara göre, NMF üzerinde en iyi başarıma “lnc” ve “nnc” ağırlıklandırma şemalarıyla ulaşılmıştır. Buda konu analizi alanında “lnc” veya “nnc” ağırlıklandırma şemalarının kullanılmasıyla daha başarılı sonuçlar elde edileceğini göstermiştir.

Keywords

• Konu Analizi
• Metin Ağırlıklandırma Şemaları
• Negatif Olmayan Matris Ayrışımı
• Başarım Karşılaştırması

Performance Comparison of Text Weighting Schemas on NMF-Based Topic Analysis

Abstract

Nowadays, it is feasible to analyze text data that is being generated at an exponential rate by transforming it into a sparse matrix of big size using a certain weighting method. A comprehensive text weighting approach consists of three fundamental components: Term Frequency, Document Frequency, and Vector Normalization. The multiplication of these three components yields numerical values that indicate the significance of a word for a text. Nevertheless, the unprocessed state of these values is unsuitable for the semantic analysis of textual material. There are multiple techniques available for this objective, and Topic Analysis, which seeks to identify subjects discussed in extensive text collections, is one of these techniques. The Non-Negative Matrix Factorization (NMF) approach is commonly employed in topic analysis. It involves transforming an input matrix into the product of two or more matrices, using both random and deterministic beginning values. This study involved conducting tests on a dataset of 20,000 articles sourced from Wikipedia, the online encyclopedia, with the aim of investigating the impact of text weighting methods and initial value approaches commonly employed in the literature on the NMF method. The number of clusters to be used in the studies was determined using an analytical procedure, which employed an upper limit. The results indicate that the “lnc” and “nnc” weighting schemes yielded the highest performance in NMF. These findings demonstrate that employing the “lnc” or “nnc” weighting scheme will lead to more favorable outcomes in the domain of topic analysis.

Keywords

• Topic Analysis
• Text-Weighting Schemas
• Non-negative Matrix Factorization
• Performance Comparison

References

1. [1] “Reports & Content — Kepios.” Accessed: Aug. 25, 2023. [Online]. Available: https://kepios.com/reports.
2. [2] Vayansky, I., Kumar, S.A.P., 2020. A review of topic modeling methods. Information Systems, Vol. 94, p. 101582. DOI: 10.1016/J.IS.2020.101582.
3. [3] Blei, D.M., 2012. Probabilistic topic models. Communications of the ACM, Vol. 55, No. 4, pp. 77–84. DOI: 10.1145/2133806.2133826.
4. [4] Schachtner, R., 2010. Extensions of Non-negative Matrix Factorization and Their Application to the Analysis of Wafer Test Data. PhD Thesis, Universität Regensburg, Regensburg.
5. [5] Shen, J., Israël, G.W., 1989. A receptor model using a specific non-negative transformation technique for ambient aerosol. Atmospheric Environment, Vol. 23, No. 10, pp. 2289–2298. DOI: 10.1016/0004-6981(89)90190-X.
6. [6] Boutsidis, C., Gallopoulos, E., 2008. SVD-based initialization: A head start for nonnegative matrix factorization. Pattern Recognition, Vol. 41, No. 4, pp. 1350–1362. DOI: 10.1016/J.PATCOG.2007.09.010.
7. [7] Yamashita, A., Nagata, T., Yagyu, S., Asahi, T., Chikyow, T., 2022. Direct feature extraction from two-dimensional X-ray diffraction images of semiconductor thin films for fabrication analysis. Manufacturing Letters, Vol. 2, No. 1, pp. 23–37. DOI: 10.1080/27660400.2022.2029222.
8. [8] Wang, Z., Yu, Y., 2022. Revealing the spatial and temporal distribution of different chemical states of lithium by EELS analysis using non-negative matrix factorization. Micron, Vol. 154, p. 103213. DOI: 10.1016/J.MICRON.2022.103213.

9. [9] Lu, H., Zhao, Q., Sang, X., Lu, J., 2020. Community Detection in Complex Networks Using Nonnegative Matrix Factorization and Density-Based Clustering Algorithm. Neural Processing Letters, Vol. 51, No. 2, pp. 1731–1748. DOI: 10.1007/S11063-019-10170-1.
10. [10] Wang, J., Zhang, X.L., 2023. Deep NMF topic modeling. Neurocomputing, Vol. 515, pp. 157–173. DOI: 10.1016/J.NEUCOM.2022.10.002.
11. [11] Habbat, N., Anoun, H., Hassouni, L., 2021. Topic Modeling and Sentiment Analysis with LDA and NMF on Moroccan Tweets. Lecture Notes in Networks and Systems, Vol. 183, pp. 147–161. DOI: 10.1007/978-3-030-66840-2_12.
12. [12] Egger, R., Yu, J., 2022. A Topic Modeling Comparison Between LDA, NMF, Top2Vec, and BERTopic to Demystify Twitter Posts. Frontiers in Sociology, Vol. 7, p. 886498. DOI: 10.3389/FSOC.2022.886498.
13. [13] Guo, Y.-T., Li, Q.-Q., Liang, C.-S., 2024. The rise of nonnegative matrix factorization: Algorithms and applications. Information Systems, Vol. 123, p. 102379. DOI: 10.1016/j.is.2024.102379.
14. [14] Takasawa, K., et al., 2024. Advances in cancer DNA methylation analysis with methPLIER: Use of non-negative matrix factorization and knowledge-based constraints to enhance biological interpretability. Experimental & Molecular Medicine. DOI: 10.1038/s12276-024-01173-7.
15. [15] Chen, D., et al., 2024. Comprehensive analyses of solute carrier family members identify SLC12A2 as a novel therapy target for colorectal cancer. Scientific Reports, Vol. 14, No. 1, p. 4459. DOI: 10.1038/s41598-024-55048-y.
16. [16] Dey, A., Das Sharma, K., Bhattacharjee, P., Chatterjee, A., 2024. Identification of disease-related biomarkers in time-varying ‘Omic data: A non-negative matrix factorization aided multi-level self-organizing map based approach. Biomedical Signal Processing and Control, Vol. 90, p. 105860. DOI: 10.1016/j.bspc.2023.105860.
17. [17] Shi, Y., Jin, Z., Deng, J., Zeng, W., Zhou, L., 2024. A novel high-dimensional kernel joint non-negative matrix factorization with multimodal information for lung cancer study. IEEE Journal of Biomedical and Health Informatics, Vol. 28, No. 2, pp. 976–987. DOI: 10.1109/JBHI.2023.3335950.
18. [18] Ramamoorthy, T., Kulothungan, V., Mappillairaju, B., 2024. Topic modeling and social network analysis approach to explore diabetes discourse on Twitter in India. Frontiers in Artificial Intelligence, Vol. 7, p. 1329185. DOI: 10.3389/frai.2024.1329185.
19. [19] Subbarayudu, Y., Sureshbabu, A., 2024. The detection of community health surveillance using distributed semantic-assisted non-negative matrix factorization on topic modeling through sentiment analysis. Multimedia Tools and Applications. DOI: 10.1007/s11042-024-18321-w.
20. [20] Choi, D., et al., 2023. WellFactor: Patient Profiling using Integrative Embedding of Healthcare Data. In Proceedings of the 2023 IEEE International Conference on Big Data (BigData), IEEE, pp. 616–625. DOI: 10.1109/BigData59044.2023.10386138.
21. [21] Ahammad, T., 2024. Identifying hidden patterns of fake COVID-19 news: An in-depth sentiment analysis and topic modeling approach. Natural Language Processing Journal, Vol. 6, p. 100053. DOI: 10.1016/j.nlp.2024.100053.
22. [22] Zong, L., Yang, Y., Xia, H., Yuan, J., Guo, M., 2023. Elucidating the Impacts of Various Atmospheric Ventilation Conditions on Local and Transboundary Ozone Pollution Patterns: A Case Study of Beijing, China. Journal of Geophysical Research: Atmospheres, Vol. 128, No. 20. DOI: 10.1029/2023JD039141.
23. [23] Knobel, P., et al., 2023. Socioeconomic and racial disparities in source-apportioned PM2.5 levels across urban areas in the contiguous US, 2010. Atmospheric Environment, Vol. 303, p. 119753. DOI: 10.1016/j.atmosenv.2023.119753.
24. [24] Westervelt, D.M., et al., 2024. Low-Cost Investigation into Sources of PM2.5 in Kinshasa, Democratic Republic of the Congo. ACS ES&T Air, Vol. 1, No. 1, pp. 43–51. DOI: 10.1021/acsestair.3c00024.
25. [25] Karamouzi, E., Pontiki, M., Krasonikolakis, Y., 2024. Historical Portrayal of Greek Tourism through Topic Modeling on International Newspapers. In Proceedings, pp. 121–132. Accessed: Mar. 30, 2024. [Online]. Available: https://aclanthology.org/2024.latechclfl-1.13.
26. [26] Athurugiriya, A.A.A.G., Sumathipala, P., Hemachandra, K.M.T.A., 2023. Development of an Enhanced Quality Score Calculation Method for Accurate Assessment of Hotel Quality. In Proceedings of the 2023 5th International Conference on Advancements in Computing (ICAC), IEEE, pp. 804–809. DOI: 10.1109/ICAC60630.2023.10417334.
27. [27] Mahmoudi, L., Hossein Shari, M., Bagheri, R., 2024. Exploring Healthcare Research Patterns in Developed and Developing Countries: A Topic Modeling Perspectives. DOI: 10.21203/RS.3.RS-3865906/V1.
28. [28] Hornback, A., et al., 2023. Latent Topic Extraction as a Source of Labeling in Natural Language Processing. In Proceedings of the 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), IEEE, pp. 4312–4319. DOI: 10.1109/BIBM58861.2023.10385618.
29. [29] Sweidan, A.H., El-Bendary, N., Elhariri, E., 2024. Autoregressive Feature Extraction with Topic Modeling for Aspect-based Sentiment Analysis of Arabic as a Low-resource Language. ACM Transactions on Asian and Low-Resource Language Information Processing, Vol. 23, No. 2, pp. 1–18. DOI: 10.1145/3638050.
30. [30] Pallawala, D., Haddela, P.S., 2023. A Comparison of Topic Modeling Techniques for Sinhala. In Proceedings of the 2023 5th International Conference on Advancements in Computing (ICAC), IEEE, pp. 376–381. DOI: 10.1109/ICAC60630.2023.10417327.
31. [31] Nanyonga, A., Wasswa, H., Wild, G., 2023. Topic Modeling Analysis of Aviation Accident Reports: A Comparative Study between LDA and NMF Models. In Proceedings of the 2023 3rd International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON), 2023. DOI: 10.1109/SMARTGENCON60755.2023.10442471.
32. [32] Ghaly, M.Z., Laksito, A.D., 2023. Topic Modeling of Natural Disaster in Indonesia Using NMF. In Proceedings of the 2023 8th International Conference on Informatics and Computing (ICIC), IEEE. DOI: 10.1109/ICIC60109.2023.10382064.
33. [33] Porter, M.F., 1980. An algorithm for suffix stripping. Program, Vol. 14, No. 3, pp. 130–137. DOI: 10.1108/EB046814.
34. [34] Blei, D.M., Ng, A.Y., Jordan, M.I., 2003. Latent Dirichlet Allocation. Journal of Machine Learning Research, Vol. 3, pp. 993–1022.
35. [35] Lee, D.D., Seung, H.S., 1999. Learning the parts of objects by non-negative matrix factorization. Nature, Vol. 401, No. 6755, pp. 788–791. DOI: 10.1038/44565.
36. [36] Paatero, P., 1997. Least squares formulation of robust non-negative factor analysis. Chemometrics and Intelligent Laboratory Systems, Vol. 37, No. 1, pp. 23–35. DOI: 10.1016/S0169-7439(96)00044-5.
37. [37] Gillis, N., 2020. Nonnegative Matrix Factorization. Philadelphia, PA: Society for Industrial and Applied Mathematics. DOI: 10.1137/1.9781611976410.
38. [38] Guillamet, D., Vitrì, J., 2002. Non-negative Matrix Factorization for Face Recognition. In Proceedings of the Fifth Catalonian Conference on Artificial Intelligence, Castellon, Spain, pp. 336–344.
39. [39] Kim, D., Sra, S., Dhillon, I.S., 2007. Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem. In Proceedings of the Sixth SIAM Conference on Data Mining, Minnesota, USA, pp. 343–354.
40. [40] Hofmann, T., 2017. Probabilistic Latent Semantic Indexing. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, pp. 211–218.
41. [41] Kim, H., Park, H., 2008. Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method. SIAM Journal on Matrix Analysis and Applications, Vol. 30, No. 2, pp. 713–730. DOI: 10.1137/07069239X.
42. [42] Belford, M., Mac Namee, B., Greene, D., 2018. Stability of topic modeling via matrix factorization. Expert Systems with Applications, Vol. 91, pp. 159–169. DOI: 10.1016/J.ESWA.2017.08.047.
43. [43] Mimno, D., Wallach, H.M., Talley, E., Leenders, M., McCallum, A., 2011. Optimizing Semantic Coherence in Topic Models. Association for Computational Linguistics, pp. 262–272. Accessed: Nov. 23, 2023. [Online]. Available: https://aclanthology.org/D11-1024.
44. [44] OpenAI, 2024. ChatGPT. Version 3.5. Accessed: Jan. 10, 2024.