Analysis of Word Similarities in Tax Laws Using the Word2Vec Method

Year 2025, Volume: 1 Issue: 1, 84 - 109, 30.01.2025

Ali İhsan Özgür Çilingir

Abstract

This paper describes word similarity analysis in tax law using the Word2Vec model. By similarity analysis, we mean identifying relationships between similar terms in tax terminology. The Word2Vec model represents the meanings of words with vectors and identifies the semantic relationships of words through the proximity between these vectors.

This article analyzes the semantic proximity of terms frequently used in tax law and visualises the relationships between these words. For example, the close relationships of the word ‘mükellef’ with words such as ‘kişi’, ‘tam’, ‘dar’, ‘firma’, and ‘imalatçı’ are represented through vectors. The paper also explains the mathematical structure of the models. Then, the features of the NumPy, Gensim, Scikit-learn, and Matplotlib libraries of the Python programming language are explained and used for this paper. For the visualisation of the similarity analysis, the t-SNE algorithm, which allows the visualisation of highdimensional data on a two-dimensional plane, was used.

The main purpose of this paper is to enable AI systems that can be used as tax advisors to better understand tax law by modelling the conceptual relationships between the terms of tax law, thus contributing to the provision of more accurate and consistent information by AI.

Keywords

Word2Vec, tax law, natural language processing (NLP), t-SNE algorithm, Skip-Gram Model, language model visualisation

References

• Mikolov, Tomas, Chen, Kai, Corrado, Greg, and Dean, Jeffrey. Efficient Estimation of Word Representations on Vector Space. arXiv preprint arXiv:1301.3781 (2013). google scholar
• Pervan, Nergis. DERİN ÖĞRENME YAKLAŞIMLARI KULLANARAK TÜRKÇE METİNLERDEN ANLAMSAL ÇIKARIM YAPMA. Ankara, 2019. google scholar
• Onan, Aytuğ. Evrişimli Sinir Ağı Mimarilerine Dayalı Türkçe Duygu Analizi. Avrupa Bilim ve Teknoloji Dergisi (Aug. 31,2020), 374-380. google scholar
• Tezgider, Murat, Yıldız, Beytullah, and Aydın, Galip. Improving Word Representation by Tuning Word2Vec Parameters with Deep Learning Model. İn International Artificial Intelligence and Data Processing Symposium (IDAP) (Malatya 2018), IEEE, 1-7. google scholar
• Arabacı, Mehmet Ali, Esen, Ersin, Atar, Muhammed Selim, Yılmaz, Eyüp, and Kaltalıoğlu, Batuhan. Kelime Gömevi Yöntemi Kullanarak Benzer Cümle Tespiti. In 2018 26th Signal Processing and Communications Applications Conference ( 2018). google scholar
• Aydoğan, Murat and Karcı, Ali. Kelime Temsil Yöntemleri ile Kelime Benzerliklerinin İncelenmesi. Çukurova Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi, 34, 2 (June 2019), 181-195. google scholar
• Acı, Çiğdem İnan and Çırak, Adem. Türkçe Haber Metinlerinin Konvolüsyonel Sinir Ağları ve Word2Vec Kullanılarak Sınıflandırılması. BİLİŞİM TEKNOLOJİLERİ DERGİSİ, 12, 13 (July 31, 2019), 219-228. google scholar
• Xia, Chunyu, He, Tieke, Li, Wenlong, Qin, Zemin, and Zou, Zhipeng. Similarity Analysis of Law Documents Based on Word2vec. In 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C) (Sofia 2019), IEEE, 354-357. google scholar
• Tatchum, Ghislain Wabo, Makembe, Fritz Sosso, Nzeko, Armel Jacques Nzekon, and Djam, Xaviera Youh. Class-Oriented Text Vectorization for Text Classification: Case Study of Job Offer Classification. Journal of Computer Science an Engineering (JCSE), 5, 2 (Aug. 01, 2024), 116-136. google scholar
• Wei, Wei, Liu, Wei, Zhang, Beibei, Scherer, Rafal, and Damasevicius, Robertas. Discovery of New Words in Tax-related Fields Based on Word Vector Representation. Journal of Internet Technology, 24, 4 (July 2023), 923-930. google scholar
• Chalkidis, Ilias and Kampas, Dimitrios. Deep learning in law: early adaptation and legal word embeddings trained on large corpora. Artificial Intelligence and Law ( (Dec. 2019), 171-198. google scholar
• Mandal, Arpan, Ghosh, Kripabandhu, Ghosh, Saptarshi, and Mandal, Sekhar. Unsupervised approaches for measuring textual similarity between legal court case reports. Artificial Intelligence and Law, 29 (2021), 417-451. google scholar
• Saha, Rohan. Influence of various text embeddings on clustering performance in NLP. arXiv, 44 (May 04, 2023), 1-22. google scholar
• Zhong, Ziyuan, Verma, Nakul, and Lia, Vincent. Lecture 8 - t-Distributed Stochastic Neighbor Embedding. New York, 2018. google scholar
• Linderman, George C. and Steinerberger, Stefan. CLUSTERING WITH T-SNE, PROVABLY. arXiv (June 08, 2017), 1-15. google scholar
• Arora, Sanjeev and Hu, Wei. An Analysis of the t-SNE Algorithm for Data Visualization. In Conference on Learning Theory (COLT) 2018 (Stockholm 2018), arXiv, 1-32. google scholar
• Maaten, Laurens van der and Hinton, Geoffrey. Visualizing Data using t-SNE. Journal of Machine Learning Research, 9, 86 (Sep. 2008), 2579-2605. google scholar
• Mueller, John Paul and Massaron, Luca. Deep Learning for Dummies. John Wiley & Sons, Inc., New Jersey, 2019. google scholar
• Nelson, Hala. Essential Math for AI - Next Level Mathematics for Efficient and Succesful AI Systems. O’Reilly Media, Sebastopol, 2023. google scholar
• Kelleher, John D. Deep Learning. The MIT Press, London, 2019. google scholar
• Anonymous. NumPy documentation. 2024. google scholar
• https://scikit-learn.org/stable/">https://scikit-learn.org/stable/. https://scikit-learn.org/stable/. 2024. google scholar
• https://matplotlib.org/">https://matplotlib.org/. https://matplotlib.org/. 2024. google scholar
• Anonim. Osmanlı Türkçesi Sözlüğü. google scholar
• Haider, Mofiz Mojib, Hossin, Arman, Mahi, Hasibur Rashid, and Arif, Hossain. Automatic Text Summarization Using Gensim Word2Vec and K-Means Clustering Algorithm. In 2020 IEEE Region 10 Symposium (TENSYMP) (Dhaka 2020), 283-286. google scholar
• Li, Zhie and Rao, Zhuyi. Text classification model based on Word2vec and SF-HAN. In 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC 2020) (Shenzhen 2020), 978-1-7281-4323-l/20/$31.00 ©2020 IEEE, 1385-1390. google scholar
• Mao, Yushang, Zhang, Guixuan, and Zhang, Shuwu. Word Semantic Similarity Based on CiLin and Word2vec. In 2020 International Conference on Culture-oriented Science & Technology (ICCST) (Beijing), 978-1-7281-8138-7/20/$31.00 ©2020 IEEE, 304 - 307. google scholar
• Bissiri, Pier Giovanni and Walker, Stephen G. Converting Information into probability measures with the Kullback-Leibler diver-gence. Ann Inst Stat Math (2012), 1139-1160. google scholar
• Jaya, Putra Syopiansyah, Nur, Gunawan Muhamad, and Akbar, Hidayat Arief. Feature Engineering with Word2vec on Text Classi-fication Using The K-Nearest Neighbor Algorithm. İn The 1Oth International Conference on Cyber and İT Service Management (CITSM 2022) (Yogyakarta 2022), ©2022 IEEE. google scholar
• Kurian, Jeomoan Francis and Allali, Mohamed. Detecting drifts in data streams using Kullback-Leibler (KL) divergence measure for data engineering Applications. Journal of Data, Information and Management (2024), 207-2016. google scholar
• Polat, Buğra. TÜRKÇE ÜRÜN YORUMLARI VERİSİ İLE DUYGU ANALİZİ. Ankara, 2021. google scholar
• Çalışkan, Sedrettin, Yazıcıoğlu, Selahattin A., Demirci, Ulaş, and Kuş, Zeki. YAPAY SİNİR AĞLARI, KELİME VEKTÖRLERİ VE DERİN ÖĞRENME UYGULAMALARI. İstanbul, 2018. google scholar
• Pirana, Gurur, Sertbaş, Ahmet, and Ensari, Tolga. Sanal Asistan Uygulamaları İçin Derin Öğrenme Yöntemiyle Cümle Sınıflandırma. In 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies, ISMSIT 2019 (Ankara 2019), Institute of Electrical and Electronics Engineers Inc. google scholar
• Kılıç, Berker and Öner, Yüksel. Yargıtay Kararlarının Suç Türlerine Göre Makine Öğrenmesi Yöntemleri İle Sınıflandırılması. VERİ BİLİMİ DERGİSİ (2021), 61-71. google scholar
• Law, Jarvan, Zhuo, Hankui Hankz, He, Junhua, and Rong, Erhu. LTSG: Latent Topical Skip-Gram for Mutually Learning Topic Model and Vector Representations. arXiv preprint arXiv (Feb. 23, 2017). google scholar
• Önal, Zeynep. Derin Öğrenme. Nobel Akademik Yayıncılık, Ankara, 2022. google scholar
• Guthrie, David, Allison, Ben, Liu, Wei, Guthiere, Louise, and Wilks, Yorick. A Closer Look at Skip-gram Modelling. İn Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06) (Genoa 2006), ACL Anthology, 1222-1225. google scholar
• Srivastava, Rajendra P. New Measure of Similarity in Textual Analysis: Vector Similarity Metric versus Cosine Similarity Metric. JOURNAL OF EMERGING TECHNOLOGIES İN ACCOUNTING, 20, 1 (2023), 77-90. google scholar
• Pudaruth, Sameerchand, Soyjaudah, Sunjiv, and Gunputh, Rajendra. Classification of Legislations using Deep Learning. The International Arab Journal of Information Technology, 18, 5 (Sep. 2021), 651-663. google scholar
• Robaldo, Livio, Villiata, Serena, Wyner, Adam, and Grabmair, Matthias. Introduction for artificial intelligence and law: special issue “natural language processing for legal texts”. Artificial Intelligence and Law (Apr. 2019), 113-115. google scholar
• Tagarelli, Andrea and Simeri, Andrea. Unsupervised law article mining based on deep pre-trained language representation models with Application to the Italian civil code. Artificial Intelligence and Law, 30 (Sep. 2022), 417-473. google scholar
• Makawana, Mayur and Mehta, Rupa G. A novel network-based paragraph filtering technique for legal document similarity analysis. Artificial Intelligence and Law (Oct. 2023). google scholar
• Bilgin, Metin. Kelime Vektörü Yöntemlerinin Model Oluşturma Sürelerinin Karşılaştırılması. BİLİŞİM TEKNOLOJİLERİ DERGİSİ, 12, 2 (Apr. 2019), 141-146. google scholar
• Ahmetoğlu, Hüseyin and Daş, Resul. Türkçe Otel Yorumlarıyla E"gitilen Kelime Vektörü Modellerinin Duygu Analizi ile 'Incelenmesi. Fen Bilimleri Enstitüsü Dergisi, 24, 2 (2020), 455-463. google scholar
• Çelik, Özer and Koç, Burak Can. TF-IDF, Word2vec ve Fasttext Vektör Model Yöntemleri ile Türkçe Haber Metinlerinin Sınıflandırılması. Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen ve Mühendislik Dergisi, 23 (2021), 121-127. google scholar
• Kınık, Doğancan and Güran, Aysun. TF-IDF ve Doc2Vec Tabanlı Türkçe Metin Sınıflandırma Sisteminin Başarım Değerinin Ardışık Kelime Grubu Tespiti ile Arttırılması. Avrupa Bilim ve Teknoloji Dergisi (Jan. 2021), 323-332. google scholar
• Hongnan, Tian and Xin, Guo. Research on Improved Sentence Similarity Calculation Method Based on Word2Vec and Synonym Table in Interactive Machine Translation. In 2021 5th International Conference on Robotics and Automation Sciences (Wuhan 2021), IEEE , 255-261. google scholar
• Xiao, Lu, Li, Qiaoxing, Ma, Qian, Shen, Jiasheng, Yang, Yong, and Li, Danyang. Text classification algorithm of tourist attractions subcategories with modified TF-IDF. PLOS ONE (Oct. 2024), 1-34. google scholar
• Gupta, Megha, Dheekonda, Venkatasai, and Masum, Mohammad. Genie: Enhancing information management in the restaurant industry through AI-powered chatbot. International Journal of Information Management Data Insights (May 25, 2024), 1-9. google scholar
• G, Dhamodharan and A, Kaleemullah. An Innovative Algorithm for Enhanced PDF-Based Chatbot in Domain-Specific Question Answering. Library Progress International, 44, 3 (Sep. 01, 2024), 27648-27653. google scholar
• Godghase, Gauri Anil, Agrawal, Rishit, Obili, Tanush, and Stamp, Mark. Distinguishing Chatbot from Human. arXiv:2408.04647v1 [cs.CL] (Aug. 12, 2024), 1-47. google scholar
• Becha, Rahma, Sellami, Asma, Bouassida, Nadia, Idri, Ali, and Abran, Alain. BotCFP: A Machine Learning based Tool for COSMIC Chatbots Sizing. CEUR, 3852 (Apr. 30, 2024), 1-16. google scholar
• https://pypi.org/project/gensim/">https://pypi.org/project/gensim/. https://pypi.org/project/gensim/. 2024. google scholar
• Leshem, Ido. Skip-Gram Word2Vec Algorithm Explained. 2023. google scholar