MULTI-LABEL TEXT ANALYSIS WITH A CNN AND LSTM BASED HYBRID DEEP LEARNING MODEL

Year 2022, , 447 - 457, 31.08.2022

Halit Çetiner

https://doi.org/10.54365/adyumbd.1106981

Cited By: 3

Abstract

In this article, it is aimed to categorize meaningful content from uncontrolled growing written social sharing data using natural language processing. Uncategorized data can disturb social sharing users with an increasing user network due to deprecating and negative content. For the stated reason, a hybrid model based on CNN and LSTM has been proposed to automatically classify all written social sharing content, both positive and negative, into defined target tags. With the proposed hybrid model, it is aimed at automatically classifying the content of the social sharing system into different categories by using the simplest embedding layer, keras. As a result of the experimental studies carried out, a better result was obtained than in the different studies in the literature using the same data set with the proposed method. The obtained performance results show that the proposed method can be applied to different multilabel text analysis problems.

Keywords

Long Short Term Memory, Convolutional Neural Network, Multi-Label Text Classification, Social Network

CNN VE LSTM TABANLI HİBRİT BİR DERİN ÖĞRENME MODELİ İLE ÇOK ETİKETLİ METİN ANALİZİ

Abstract

Bu makalede doğal dil işleme kullanılarak kontrolsüz olarak büyüyen yazılı sosyal paylaşım verilerinin içerisinden anlamlı içeriklerin kategorize edilmesi amaçlanmıştır. Kategorize edilmeyen verilerin, artan kullanıcı ağına sahip sosyal paylaşım kullanıcılarını olumsuz ve negatif içerikten dolayı rahatsız edebilmektedir. Belirtilen sebepten dolayı olumlu ve olumsuz olmak üzere tüm yazılı sosyal paylaşım içeriklerinin tanımlı hedef etiketlerine otomatik olarak sınıflandırılabilmesi için CNN ve LSTM tabanlı bir hibrit model önerilmiştir. Önerilen hibrit model ile en basit gömme katmanı olan keras kullanılarak farklı kategorilere sahip sosyal paylaşım sistemi içeriklerinin otomatik sınıflandırılması hedeflenmiştir. Gerçekleştirilen deneysel çalışmalar neticesinde önerilen yöntem ile aynı veri setini kullanan literatürdeki farklı çalışmalardan daha iyi bir sonuç elde edilmiştir. Elde edilen performans sonuçları önerilen yöntemin farklı çok etiketli metin analizi problemlerine de uygulanabileceği göstermektedir.

Keywords

Uzun Kısa Süreli Bellek, Konvolüsyon Sinir Ağı, Çok Etiketli Metin Sınıflandırma, Sosyal Ağ

References

• Sahoo SR, Gupta BB. Multiple features based approach for automatic fake news detection on social networks using deep learning. Appl Soft Comput 2021;100:106983. doi:10.1016/j.asoc.2020.106983.
• Horne B, Adali S. This just in: Fake news packs a lot in title, uses simpler, repetitive content in text body, more similar to satire than real news. Proc. Int. AAAI Conf. web Soc. media, 2017; 11: 759–66.
• Balmas M. When fake news becomes real: Combined exposure to multiple news sources and political attitudes of inefficacy, alienation, and cynicism. Communic Res 2014;41:430–54.
• Liu B, Liu X, Ren H, Qian J, Wang Y. Text multi-label learning method based on label-aware attention and semantic dependency. Multimed Tools Appl 2022;81:7219–37. doi:10.1007/s11042-021-11663-9.
• Pavlinek M, Podgorelec V. Text classification method based on self-training and LDA topic models. Expert Syst Appl 2017;80:83–93. doi:10.1016/j.eswa.2017.03.020.
• Feng Y, Wu Z, Zhou Z. Multi-label text categorization using k-Nearest Neighbor approach with M-Similarity. Int. Symp. String Process. Inf. Retr., Springer 2005; 155–60.
• Gong J, Teng Z, Teng Q, Zhang H, Du L, Chen S, Bhuiyan MZA, Li J, Liu M, Ma H. Hierarchical graph transformer-based deep learning model for large-scale multi-label text classification. IEEE Access 2020;8:30885–96.
• Nam J, Kim J, Loza Mencía E, Gurevych I, Fürnkranz J. Large-Scale Multi-label Text Classification — Revisiting Neural Networks BT - Machine Learning and Knowledge Discovery in Databases. In: Calders T, Esposito F, Hüllermeier E, Meo R, editors., Berlin, Heidelberg: Springer Berlin Heidelberg; 2014; 437–52.
• Jayaraman AK, Murugappan A, Trueman TE, Cambria E. Comment toxicity detection via a multichannel convolutional bidirectional gated recurrent unit. Neurocomputing 2021;441:272–8. doi:10.1016/j.neucom.2021.02.023.
• Bahdanau D, Cho K, Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate. ArXiv 2014;1409.
• Yadav V, Bethard S. A Survey on Recent Advances in Named Entity Recognition from Deep Learning models. 2019.
• Lauriola I, Lavelli A, Aiolli F. An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools. Neurocomputing 2022;470:443–56. doi:10.1016/j.neucom.2021.05.103.
• Conneau A, Schwenk H, Barrault L, Lecun Y. Very deep convolutional networks for text classification. ArXiv Prepr ArXiv160601781 2016.
• Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences. ArXiv Prepr ArXiv14042188 2014.
• Kim Y. Convolutional Neural Networks for Sentence Classification 2014.
• Zhou C, Sun C, Liu Z, Lau F. A C-LSTM neural network for text classification. ArXiv Prepr ArXiv151108630 2015.
• Johnson R, Zhang T. Semi-supervised convolutional neural networks for text categorization via region embedding. Adv Neural Inf Process Syst 2015;28:919.
• Rakhlin A. Convolutional neural networks for sentence classification 2016.
• Chen Y. Convolutional neural network for sentence classification 2015.
• Liu G, Guo J. Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 2019;337:325–38. doi:10.1016/j.neucom.2019.01.078.
• Cao J, Zhang Z, Luo Y, Zhang L, Zhang J, Li Z, Tao F. Wheat yield predictions at a county and field scale with deep learning, machine learning, and google earth engine. Eur J Agron 2021;123:126204. doi:https://doi.org/10.1016/j.eja.2020.126204.
• Wulczyn E, Thain N, Dixon L. Ex Machina: Personal Attacks Seen at Scale. Proc. 26th Int. Conf. World Wide Web, Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee 2017; 1391–1399. doi:10.1145/3038912.3052591.
• Guo X. Multi-label Classification and Sentiment Analysis on Textual Records 2019.
• Pang Z, Niu F, O’Neill Z. Solar radiation prediction using recurrent neural network and artificial neural network: A case study with comparisons. Renew Energy 2020;156:279–89. doi:https://doi.org/10.1016/j.renene.2020.04.042.
• Wang JQ, Du Y, Wang J. LSTM based long-term energy consumption prediction with periodicity. Energy 2020;197:117197.
• Srinivasu PN, SivaSai JG, Ijaz MF, Bhoi AK, Kim W, Kang JJ. Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM. Sensors 2021;21. doi:10.3390/s21082852.
• LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436–44. doi:10.1038/nature14539.
• Çetiner H. Classification of Apple Leaf Diseases Using The Proposed Convolution Neural Network Approach. J Eng Sci Des 2021;9:1130–40. doi:10.21923/jesd.980629.
• Langer S. Approximating smooth functions by deep neural networks with sigmoid activation function. J Multivar Anal 2021;182:104696. doi:https://doi.org/10.1016/j.jmva.2020.104696.
• Goutte C, Gaussier E. A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation. Lect. Notes Comput. Sci. 2005; 3408:345–59. doi:10.1007/978-3-540-31865-1_25.
• Kingma D, Ba J. Adam: A Method for Stochastic Optimization. Int Conf Learn Represent 2014.
• Ruder S. An overview of gradient descent optimization algorithms. ArXiv Prepr ArXiv160904747 2016.
• Magalhaes A, Small H. Deep Learning Approaches to Classifying Types of Toxicity in Wikipedia Comments.
• Mohammed HH, Dogdu E, Görür AK, Choupani R. Multi-Label Classification of Text Documents Using Deep Learning. 2020 IEEE Int. Conf. Big Data (Big Data), IEEE 2020; 4681–9.