Transformer Modellerinden Bert ve İki Yönlü LSTM'lerin Hibrit Kullanılması ve Grid Search Hiperparametre Optimizasyonu ile Sahte Haber Tespiti

Year 2025, Volume: 18 Issue: 1, 11 - 28, 31.01.2025

Muhammet Tan , Halit Bakır

https://doi.org/10.17671/gazibtd.1521520

Abstract

Sosyal medyada yayılan sahte haberler ve yanlış bilgiler, toplum algısını ve davranışlarını önemli ölçüde çarpıta-bilir ve ciddi sorunlara yol açabilir. Bu yanıltıcı içerikler, bireylerin yanlış bilgilere dayanarak kararlar almasına neden olarak toplumsal kutuplaşmayı artırma potansiyeline sahiptir. Kriz zamanlarında, sahte haberlerin yayılması halk sağlığını tehlikeye atabilir, ekonomiyi istikrarsızlaştırabilir ve demokratik kurumlara olan güveni zedeleyebil-ir. Bu önemli sorunu ele almak amacıyla, günümüzde birçok çalışma makine öğrenimi ve derin öğrenme modelleri-ni kullanmaktadır. Bu çalışmada, doğal dil işleme alanında yaygın olarak kullanılan transformer mimarisi tercih edilmiştir. Uzun metinlerin daha istikrarlı bir şekilde işlenmesi için modelde Bidirectional LSTM'ler (İki Yönlü Uzun-Kısa Vadeli Bellek) transformer mimarisiyle hibrit hale getirilmiştir. Sahte tweetlerin daha kolay tespit edile-bilmesi amacıyla, veri setindeki hedef kategoriler dengelenmiş ve sınıflama başarımının artırılması için TomekLinks kütüphanesi kullanılmıştır. Model performansını artırmak için bir parametre havuzu oluşturulmuş ve Grid Search metodu ile en başarılı sonuçları veren parametreler belirlenmiştir. Yapılan testlerde, en iyi 10 modelin tamamı %99 doğruluk oranına ulaşmıştır. En yüksek performans gösteren model, %99.908 doğruluk oranı elde etmiştir.

Keywords

sahte haber tespiti, doğal dil işleme, bert, uzun-kısa süreli bellek

Fake News Detection Using BERT and Bi-LSTM with Grid Search Hyperparameter Optimization

Abstract

Fake news and misinformation disseminated on social media can significantly distort public perception and behav-ior, leading to serious issues. These deceptive contents have the potential to increase societal polarization by caus-ing individuals to make decisions based on false information. During crises, the spread of fake news can endanger public health, destabilize the economy, and undermine trust in democratic institutions. To address this critical issue, numerous studies today employ machine learning and deep learning models. In this study, the transformer architec-ture, widely used in natural language processing, was utilized. To process longer texts more reliably, Bidirectional LSTMs were hybridized with the transformer architecture in the model. For easier detection of fake tweets, the target categories in the dataset were balanced, and the TomekLinks algorithm was employed to enhance classification performance. To improve model performance, a parameter pool was established, and Grid Search was used to identi-fy parameters yielding the most successful results. In our tests, all top 10 models achieved an accuracy of 99%. The highest-performing model achieved an impressive accuracy of 99.908%.

Keywords

fake news detection, natural language processing, bert, long-short term memory

References

• Janelle B. Hill, “Gartner insights on spotting and responding to digital disruption Leading Through Digital Disruption EDITED BY Janelle B. Hill, Gart-ner Research Vice President and Distinguished Ana-lyst,” 2017. Accessed: Jan. 04, 2025. [Online]. Available: https://www.gartner.com/imagesrv/books/digital-disruption/pdf/digital_disruption_ebook.pdf
• G. Mavridis, “Fake news and Social Media: How Greek users identify and curb misinformation online,” 2018, Accessed: Jan. 03, 2025. [Online]. Available: https://urn.kb.se/resolve?urn=urn:nbn:se:mau:diva-23196
• S. Dadkhah, X. Zhang, A. G. Weismann, A. Firouzi, and A. A. Ghorbani, “TruthSeeker: The Largest So-cial Media Ground-Truth Dataset for Real/Fake Con-tent.” [Online]. Available: https://www.unb.ca/cic/datasets/truthseeker-2023.html.
• S. Sharma, M. Saraswat, and A. K. Dubey, “Fake news detection on Twitter,” International Journal of Web Information Systems, vol. 18, no. 5–6, pp. 388–412, Dec. 2022, doi: 10.1108/IJWIS-02-2022-0044.
• A. Ali and M. Gulzar, “An Improved FakeBERT for Fake News Detection,” Applied Computer Systems, vol. 28, no. 2, pp. 180–188, Dec. 2023, doi: 10.2478/acss-2023-0018.
• R. K. Kaliyar, A. Goswami, and P. Narang, “FakeBERT: Fake news detection in social media with a BERT-based deep learning approach,” Mul-timed Tools Appl, vol. 80, no. 8, pp. 11765–11788, Mar. 2021, doi: 10.1007/s11042-020-10183-2.
• H. Alsaidi and W. Etaiwi, “Empirical Evaluation of Machine Learning Classification Algorithms for De-tecting COVID-19 Fake News,” International Journal of Advances in Soft Computing and its Applications, vol. 14, no. 1, pp. 49–59, 2022, doi: 10.15849/IJASCA.220328.04.
• A. M. Ali, F. A. Ghaleb, B. A. S. Al-Rimy, F. J. Alsolami, and A. I. Khan, “Deep Ensemble Fake News Detection Model Using Sequential Deep Learn-ing Technique,” Sensors, vol. 22, no. 18, Sep. 2022, doi: 10.3390/s22186970.
• B. Fang and H. Zhou, “Fake news text detection based on convolutional neural network,” Applied and Computational Engineering, vol. 41, no. 1, pp. 202–209, Feb. 2024, doi: 10.54254/2755-2721/41/20230744.
• “(PDF) ULMFiT for Twitter Fake News Spreader Profiling Notebook for PAN at CLEF 2020.” Ac-cessed: Jan. 04, 2025. [Online]. Available: https://www.researchgate.net/publication/359024571_ULMFiT_for_Twitter_Fake_News_Spreader_Profiling_Notebook_for_PAN_at_CLEF_2020
• K. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu, “Fake News Detection on Social Media,” ACM SIGKDD Explorations Newsletter, vol. 19, no. 1, pp. 22–36, Sep. 2017, doi: 10.1145/3137597.3137600.
• O. Stitini and S. Kaloun, “An improved self-training model to detect fake news categories using multi-class classification of unlabeled data: fake news clas-sifi-cation with unlabeled data,” Int. J. Systematic In-novation, vol. 8, no. 1, p. 4, 2024, doi: 10.6977/IJoSI.202403_8(1)0002.
• V. Balakrishnan, H. L. Zing, and E. Laporte, “COVID-19 INFODEMIC – UNDERSTANDING CONTENT FEATURES IN DETECTING FAKE NEWS USING A MACHINE LEARNING AP-PROACH,” Malaysian Journal of Computer Science, vol. 36, no. 1, pp. 1–13, 2023, doi: 10.22452/mjcs.vol36no1.1.
• A. K. Yadav et al., “Fake News Detection Using Hybrid Deep Learning Method,” SN Comput Sci, vol. 4, no. 6, pp. 1–15, Nov. 2023, doi: 10.1007/S42979-023-02296-W/METRICS.
• M. Park and S. Chai, “Constructing a User-Centered Fake News Detection Model by Using Classification Algorithms in Machine Learning Techniques,” IEEE Access, vol. 11, pp. 71517–71527, 2023, doi: 10.1109/ACCESS.2023.3294613.
• “(13) (PDF) DISTILBERT FOR WEB SECURITY: ENHANCED DETECTION OF XSS ATTACKS US-ING NLP APPROACH.” Accessed: Jul. 17, 2024. [Online]. Available: https://www.researchgate.net/publication/381659932_DISTILBERT_FOR_WEB_SECURITY_ENHANCED_DETECTION_OF_XSS_ATTACKS_USING_NLP_APPROACH
• R. Ghanem, H. Erbay, and K. Bakour, “Contents-Based Spam Detection on Social Networks Using RoBERTa Embedding and Stacked BLSTM,” SN Comput Sci, vol. 4, no. 4, pp. 1–15, Jul. 2023, doi: 10.1007/S42979-023-01798-X/METRICS.
• R. Ghanem and H. Erbay, “Spam detection on social networks using deep contextualized word representa-tion,” Multimed Tools Appl, vol. 82, no. 3, pp. 3697–3712, Jan. 2023, doi: 10.1007/S11042-022-13397-8/METRICS.
• A. Makalesi, R. Article Rezan BAKIR, H. Erbay, and H. Bakir, “ALBERT4Spam: A Novel Approach for Spam Detection on Social Networks,” no. 2, p. 17, doi: 10.17671/gazibtd.1426230.
• G. K. Koru and C. Uluyol, “Detection of Turkish Fake News from Tweets with BERT Models,” IEEE Access, vol. 12, pp. 14918–14931, 2024, doi: 10.1109/ACCESS.2024.3354165.
• S. G. TAŞKIN, E. U. KÜÇÜKSİLLE, and K. TOPAL, “Twitter üzerinde Türkçe sahte haber tespiti,” Balıkesir Üniversitesi Fen Bilimleri Enstitüsü Dergisi, vol. 23, no. 1, pp. 151–172, Jan. 2021, doi: 10.25092/baunfbed.843909.
• N. A. S. Abdullah, N. I. A. Rusli, and N. S. Yuslee, “Development of a machine learning algorithm for fake news detection,” Indonesian Journal of Electri-cal Engineering and Computer Science, vol. 35, no. 3, pp. 1732–1743, Sep. 2024, doi: 10.11591/ijeecs.v35.i3.pp1732-1743.
• R. Bakır and H. Bakır, “Swift Detection of XSS At-tacks: Enhancing XSS Attack Detection by Leverag-ing Hybrid Semantic Embeddings and AI Tech-niques,” Arab J Sci Eng, pp. 1–17, Jun. 2024, doi: 10.1007/S13369-024-09140-0/TABLES/14.
• H. Bakir and G. Tarihi, “Using Transfer Learning Technique as a Feature Extraction Phase for Diagno-sis of Cataract Disease in the Eye,” USBTU, vol. 1, no. 1, p. 2022.
• H. Bakır and K. Elmabruk, “Deep learning-based approach for detection of turbulence-induced distor-tions in free-space optical communication links,” Phys Scr, vol. 98, no. 6, p. 065521, May 2023, doi: 10.1088/1402-4896/ACD4FA.
• U. Demircioğlu, A. Sayil, and H. Bakır, “Detecting Cutout Shape and Predicting Its Location in Sand-wich Structures Using Free Vibration Analysis and Tuned Machine-Learning Algorithms,” Arab J Sci Eng, vol. 49, no. 2, pp. 1611–1624, Feb. 2024, doi: 10.1007/S13369-023-07917-3/METRICS.
• J. Bergstra, J. B. Ca, and Y. B. Ca, “Random Search for Hyper-Parameter Optimization Yoshua Bengio,” 2012. [Online]. Available: http://scikit-learn.sourceforge.net.
• V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” Oct. 2019, [Online]. Available: http://arxiv.org/abs/1910.01108
• N. Seddari, A. Derhab, M. Belaoued, W. Halboob, J. Al-Muhtadi, and A. Bouras, “A Hybrid Linguistic and Knowledge-Based Analysis Approach for Fake News Detection on Social Media,” IEEE Access, vol. 10, pp. 62097–62109, 2022, doi: 10.1109/ACCESS.2022.3181184.
• S. R. Sahoo and B. B. Gupta, “Multiple features based approach for automatic fake news detection on social networks using deep learning,” Appl Soft Comput, vol. 100, p. 106983, Mar. 2021, doi: 10.1016/J.ASOC.2020.106983.
• A. Jarrahi and L. Safari, “FR-Detect: A Multi-Modal Framework for Early Fake News Detection on Social Media Using Publishers Features,” Sep. 2021, Ac-cessed: Sep. 24, 2024. [Online]. Available: https://arxiv.org/abs/2109.04835v1
• Y. Wang, Y. Zhang, X. Li, and X. Yu, “COVID-19 Fake News Detection Using Bidirectional Encoder Representations from Transformers Based Models,” Sep. 2021, Accessed: Sep. 24, 2024. [Online]. Avail-able: https://arxiv.org/abs/2109.14816v2
• S. Ni, J. Li, and H. Y. Kao, “MVAN: Multi-View Attention Networks for Fake News Detection on So-cial Media,” IEEE Access, vol. 9, pp. 106907–106917, 2021, doi: 10.1109/ACCESS.2021.3100245.
• Y. J. Lu and C. Te Li, “GCAN: Graph-aware co-attention networks for explainable fake news detec-tion on social media,” Proceedings of the Annual Meeting of the Association for Computational Lin-guistics, pp. 505–514, 2020, doi: 10.18653/V1/2020.ACL-MAIN.48.
• “(3) (PDF) Fake News Early Detection: A Theory-driven Model.” Accessed: Sep. 24, 2024. [Online]. Available: https://www.researchgate.net/publication/332726212_Fake_News_Early_Detection_A_Theory-driven_Model
• K. Shu, S. Wang, H. Liu, and D. Mahudeswaran, “Hierarchical Propagation Networks for Fake News Detection: Investigation and Exploitation”, doi: 10.48550/arXiv.1903.09196.
• A. Kesarwani, S. S. Chauhan, and A. R. Nair, “Fake News Detection on Social Media using K-Nearest Neighbor Classifier,” Proceedings of the 2020 Inter-national Conference on Advances in Computing and Communication Engineering, ICACCE 2020, Jun. 2020, doi: 10.1109/ICACCE49060.2020.9154997.
• S. Yang, K. Shu, S. Wang, R. Gu, F. Wu, and H. Liu, “Unsupervised fake news detection on social media: A generative approach,” 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educa-tional Advances in Artificial Intelligence, EAAI 2019, pp. 5644–5651, 2019, doi: 10.1609/AAAI.V33I01.33015644.
• K. Shu, D. Mahudeswaran, and H. Liu, “FakeNew-sTracker: a tool for fake news collection, detection, and visualization,” Comput Math Organ Theory, vol. 25, no. 1, pp. 60–71, Mar. 2019, doi: 10.1007/s10588-018-09280-3.
• T. Traylor, J. Straub, Gurmeet, and N. Snell, “Classi-fying Fake News Articles Using Natural Language Processing to Identify In-Article Attribution as a Su-pervised Learning Estimator,” Proceedings - 13th IEEE International Conference on Semantic Compu-ting, ICSC 2019, pp. 445–449, Mar. 2019, doi: 10.1109/ICOSC.2019.8665593.
• T. Rasool, W. H. Butt, A. Shaukat, and M. U. Akram, “Multi-label fake news detection using multi-layered supervised learning,” ACM International Conference Proceeding Series, pp. 73–77, Feb. 2019, doi: 10.1145/3313991.3314008.
• M. KAYAKUŞ and F. YİĞİT AÇIKGÖZ, “Twitter’da Makine Öğrenmesi Yöntemleriyle Sahte Haber Tespiti,” Abant Sosyal Bilimler Dergisi, vol. 23, no. 2, pp. 1017–1027, Jul. 2023, doi: 10.11616/asbi.1266179.