Natural Language Processing, Deep Learning, and Text Processing: A Translational Perspective

Öz

This study offers a multifaceted examination of Natural Language Processing (NLP) and deep learning models, which are key components of artificial intelligence. In this context, text processing workflows operating on artificial neural networks are discussed with a particular focus on translation. The advancement of artificial neural networks and the widespread adoption of deep learning algorithms have led to significant developments in the field of NLP. Neural machine translation, which has emerged as a transformative development in artificial intelligence, has sparked debates within the translation community. Although it is still under discussion whether neural machine translation can produce translations at a human level of reliability, the speed and convenience it offers undoubtedly contribute to digitalization and automation in the era of Industry 4.0. NLP employs various methods to enable computers to process human language. Among these, deep learning algorithms based on artificial neural networks dominate current applications and neural machine translation systems. Accordingly, diverse text processing workflows are carried out within NLP and deep learning mechanisms. These workflows may vary depending on the architectural characteristics of the language model being used. The study examines recent language model architectures and their properties, with a particular focus on the text processing procedures inherent to these models. These procedures are examined in relation to the language processing mechanisms of the human brain, within the framework of a descriptive analytical approach. The study concludes that NLP and deep learning technologies will play a significant role in the future of the translation profession and emphasizes the necessity for translators to follow technological advancements in response to this transformation.

Anahtar Kelimeler

• natural language processing
• deep learning
• text processing workflows
• translation.

Doğal Dil İşleme, Derin Öğrenme ve Metin İşleme: Çeviri Süreçlerine Yönelik Bir İnceleme

Öz

Bu çalışmada yapay zekanın önemli bileşenleri olan doğal dil işleme (DDİ) ve derin öğrenme modelleri çok yönlü olarak incelenmiş; bu doğrultuda yapay sinir ağları üzerinde gerçekleşen metin işleme süreçleri, çeviri bakımından ele alınmıştır. Yapay sinir ağlarının gelişimi ve dolayısıyla derin öğrenme algoritmalarının yaygınlaşması, doğal dil işleme alanında önemli gelişmelere yol açmıştır. Yapay zekâ alanında büyük bir dönüşümle yerini alan nöral makine çevirisi, çeviri dünyasını tartışmalara açmıştır. Nöral makine çevirisinin henüz insan düzeyinde güvenilir çeviri yapıp yapmadığı tartışılsa da çevirmenlere sağlayacağı hız ve kolaylık, şüphesiz endüstri 4.0 çağında dijitalleşme ve otomasyona katkı sunacaktır. Doğal dil işleme bilgisayarların insan dilini işlemesi amacıyla çeşitli yöntemler kullanmaktadır. Bu yöntemler arasında yapay sinir ağlarından oluşan derin öğrenme algoritmaları, günümüz uygulamalarını ve nöral makine çevirisini domine etmektedir. Bu doğrultuda doğal dil işlemede ve derin öğrenme mekanizmalarında çeşitli metin işleme süreçleri gerçekleşmektedir. Metin işleme süreçleri dil modelinde kullanılan mimarinin niteliğine bağlı olarak değişiklik arz edebilir. Çalışmada en güncel dil mimarileri ve özellikleri incelenerek dil modellerine içkin metin işleme süreçleri, insan beyninin dil işleme süreçleri ile karşılaştırılarak betimleyici ve analitik bir yaklaşım çerçevesinde ele alınmıştır. Çalışma, doğal dil işleme ve derin öğrenme teknolojilerinin gelecekte çevirmenlik mesleğinde önemli bir rol oynayacağı gerçekliğini ve çevirmenlerin bu dönüşüme karşı teknolojik gelişmeleri takip etmeleri gerekliliğini ortaya koymuştur.

Anahtar Kelimeler

• doğal dil işleme
• derin öğrenme
• metin işleme süreçleri
• çeviri.

Kaynakça

1. Adalı, E. (2016). Doğal Dil İşleme. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi. 5(2).
2. Ahammad, S. H., Kalangi, R. R., Nagendram, S., Inthiyaz, S., Priya P.P., Faragallah O.S., Mohammad, A., Eid, M.M.A.,
3. Rashed, A.N.Z. (2023). Improved neural machine translation using Natural Language Processing (NLP). Multimedia Tools and Applications. Springer.
4. Alpaydın, E. (2022). Yapay Öğrenme Yeni: Yapay Zekâ. çev. Aylin Ağar, 3. Baskı, Tellekt, İstanbul. Antoine J. ve P. Tixier (2018). Notes on Deep Learning for NLP. arXiv, preprint arXiv:1808.09772. Erişim adresi https://doi.org/10.48550/arXiv.1808.09772
5. Bahdanau, D., Cho, KH. ve Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. Proceedings of ICLR. arXiv, preprint arXiv:1409.0473. Erişim adresi https://doi.org/10.48550/arXiv.1409.0473
6. Benítez-Burraco, A. ve Boeckx, C. (2015). Possible functional links among brain- and skull-related genes selected in modern humans. Frontiers in Psychology, 6, Article 794.
7. Camacho-Collados J. ve Pilehvar, M.T. (2018). On the Role of Text Preprocessing in Neural Network Architectures: An Evaluation Study on Text Categorization and Sentiment Analysis. arXiv, preprint arXiv:1707.01780. Erişim adresi https://doi.org/10.48550/arXiv.1707.01780
8. Deniz, F., Nunez-Elizalde, A. O., Huth, A. G. ve Gallant, J. L. (2019). The representation of semantic information across human cerebral cortex during listening versus reading is invariant to stimulus modality. Journal of Neuroscience, 39(39), 7722-7736.

9. Deshmukh, R. D. ve Kiwelekar, A. (2020). Deep learning techniques for part of speech tagging by natural language processing. 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA) içinde, 76-81, IEEE.
10. Dohare, S., Fernando Hernandez-Garcia, J., Lan, Q., Rahman, P., Rupam Mahmood, A., S.Sutton, R. (2024). Loss of plasticity in deep continual learning. Nature, Springer Science and Business Media LLC. Erişim adresi https://doi.org/10.1038/s41586-024-07711-7
11. Feldman, R. ve Dagan, I. (1995). Knowledge Discovery in Textual Databases. KDD’ 95 Proceedings of the First International Conference on Knowledge Discovery and Data Mining, 112-117. Erişim adresi https://www.ijcseonline.org/spl_pub_paper/30-ICCIS-18.pdf
12. Friederici, A. D. (2011). The brain basis of language processing: from structure to function. Physiological reviews, 91(4), 1357-1392.
13. Goodfellow, I., Bengio, Y. ve Courville, A. (2016). Deep Learning. MIT Press. Erişim adresi www.deeplearningbook.org
14. Hickok, G., ve Poeppel, D. (2007). The cortical organization of speech processing. Nature reviews neuroscience, 8(5), 393-402.
15. Hinke, R. M., Hu, X., Stillman, A. E., Kim, S. G., Merkle, H., Salmi, R. ve Ugurbil, K. (1993). Functional magnetic resonance imaging of Broca's area during internal speech. Neuroreport, 4(6), 675-678.
16. Hochreiter S. ve Schmidhuber J. (1997). Long Short-Term Memory. Neural Computation, 9(8); 1735–1780. Erişim adresi https://doi.org/10.1162/neco.1997.9.8.1735 (17.03.2025)
17. Huth, A. G., De Heer, W. A., Griffiths, T. L., Theunissen, F. E. ve Gallant, J. L. (2016). Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600), 453-458.
18. Holmes, J. S. (2000). The Name and Nature of Translation Studies. 1972, In: Venuti, L. (eds): The Translation Studies Reader, Chapter 13, 172-185.
19. Holzinger, A. Saranti, A. Molnar, C. Biecek, P. ve Samek, W. (2022). Explainable AI Methods - A Brief Overview. International workshop on extending explainable AI beyond deep models and classifiers içinde, 13-38, Cham: Springer International Publishing. Erişim adresi https://dx.doi.org/10.1007/978-3-031-04083-2_2
20. Hopfield, J. J. (1982). Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences of the United States of America, Vol. 79, No 8, Part 1Biological Sciences, 2554-2558.
21. Jabeen, R., Ericsson, M., ve Nordqvist, J. (2023). Towards Better Product Quality: Identifying Legitimate Quality Issues through NLP & Machine Learning Techniques. Linköping Electronic Conference Proceedings, Linköping University Electronic Press. doi.org/10.3384/ecp199009
22. Jääskeläinen, R. (2016). Translation Psychology. Çev. Eraçıkbaş, A.F., Handbook of Translation Studies, Vol. 3 (2012), 191-197, John Benjamins Publishing Company.
23. Irie, K., Gerstenberger, A., Schlüter, R. ve Ney, H. (2020). How Much Self-Attention Do We Need? Trading Attention for Feed-Forward Layers. ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6154-6158.
24. Kerimoğlu, C. (2022). Dilin Kökeni Arayışları-5: Beyin ve Dil, Dil araştırmaları Journal of Language Studies, Yıl: 16, Dönem: 2022/Bahar, Sayı: 30 ISSN 1307-7821 | e-ISSN 2757-8003
25. Khurana, D., Koli, A., Khatter, K. ve Singh, S. (2022). Natural language processing: State of the art, current trends and challenges. Multimedia Tools and Applications, 82, 3713–3744.
26. Koroteev, M. V. (2021). BERT: a review of applications in natural language processing and understanding. arXiv, preprint arXiv: 2103.11943. Erişim adresi https://arxiv.org/pdf/2103.11943 (17.03.2025)
27. LeCun, Y., Bengio Y. ve Hinton, G. (2015). Deep learning, Nature, 521.7553 (2015): 436-444.
28. Lindborg, A. ve Rabovsky, M. (2021). Meaning in brains and machines: Internal activation update in large-scale language model partially reflects the n400 brain potential. Proceedings of the annual meeting of the cognitive science society, Vol. 43. Erişim adresi https://escholarship.org/uc/item/6d71c9sj (16.03.2025)
29. Liu, F. H., Gu, L., Gao, Y. ve Picheny, M. (2003). Use of statistical N-gram models in natural language generation for machine translation. 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, Proceedings.(ICASSP'03),Vol. 1, pp.I-I). IEEE.
30. Lourdusamy, R. ve Abraham, S. (2018). A Survey on Text Pre-processing Techniques and Tools. International Journal of Computer Sciences and Engineering, Vol. 6, Special Issue-3, 148-157, E-ISSN: 2347-2693.
31. Lu, F., Yuan, Z. (2019). Explore the Brain Activity during Translation and Interpreting Using Functional Near-Infrared Spectroscopy. In: Li, D., Lei, V., He, Y. (eds) Researching Cognitive Processes of Translation. New Frontiers in Translation Studies. Springer, Singapore. Erişim adresi https://doi.org/10.1007/978-981-13-1984-6_5 Luo, S., Ivison, H., Han, S. C. ve Poon, J. (2024). Local interpretations for explainable natural language processing: A survey, ACM Computing Surveys, 56(9), 1-36.
32. M.P. Brasoveanu, A. ve Andonie, R. (2020). Visualizing Transformers for NLP: A Brief Survey. 24th International Conference Information Visualisation (IV). Erişim adresi https://doi.org/10.1109/iv51561.2020.00051 (17.03.2025)
33. Mänttäri, J.H. (1984). Translatorisches Handeln:Theorie und Methode, Suomalainen Tiedeakatemia, Helsinki.
34. Nord, C. (1995). Textanalyse und Übersetzen: theoretische Grundlagen, Methode und didaktische Anwendung einer übersetzungsrelevanten Textanalyse. 3. Auflage, Heidelberg, Groos. ISBN: 3-87276-649-X
35. Özengi, A. (2024). Derin Öğrenme Teknikleri Kullanılarak Türkçe-İngilizce Nöral Makine Çevirisi. Dönem Projesi, İzmir Kâtip Çelebi Üniversitesi Fen Bilimleri Enstitüsü. Erişim adresi https://acikerisim.ikcu.edu.tr/yayin/1742898&dil=3&q=(17.03.2025)
36. Pasquiou, A., Lakretz, Y., Hale, J., Thirion, B. ve Pallier, C. (2022). Neural language models are not born equal to fit brain data, but training helps. arXiv, preprint arXiv:2207.03380. Erişim adresi https://arxiv.org/pdf/2207.03380 (17.03.2025)
37. Patwardhan, N., Marrone, S. ve Sansone, C. (2023). Transformers in the Real World: A Survey on NLP Applications. Information, MDPI AG. Erişim adresi https://doi.org/10.3390/info14040242 (17.03.2025)
38. Pekcoşkun Güner, S. (2023). Çevirmen-Bilgisayar Etkileşiminin Kilit Bileşeni: Doğal Dil İşleme. Karamanoğlu Mehmetbey Üniversitesi Uluslararası Filoloji ve Çeviribilim Dergisi, 5(1), 56-79. Erişim adresi https://doi.org/10.55036/ufced.1306746 (17.03.2025)
39. Poeppel, D., Emmorey, K., Hickok, G. ve Pylkkänen, L. (2012). Towards A New Neurobiology of Language. Journal of Neuroscience, 32/41: 14125–14131. Rahali, A. ve Akhloufi, M. A. (2023). End-to-End Transformer-Based Models in Textual-Based NLP, AI, 4(1), 54-110. MDPI. Erişim adresi https://doi.org/10.3390/ai4010004 (17.03.2025)
40. Reiß, K. ve Vermeer, H. (1984). Grundlegung einer allgemeinen Translationstheorie. De Gruyter. Rumelhart, D. E., Hinton, G. ve Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323.6088: 533-536.
41. Sankar, H. ve Subramaniyaswamy, V. (2017). Investigating sentiment analysis using machine learning approach. 2017 International Conference on Intelligent Sustainable Systems (ICISS). Erişim adresi https://doi.org/10.1109/iss1.2017.8389293 (17.03.2025)
42. Samek W., Wiegand T. ve Müller K.R. (2017). Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models. arXiv, preprint arXiv:1708.08296. Erişim adresi https://doi.org/10.48550/arXiv.1708.08296
43. Schmidhuber, J. (2014). Deep Learning in Neural Networks: An Overview. Neural networks, 61, 85-117. Erişim adresi https://doi.org/10.48550/arXiv.1404.7828
44. Seker, S. E. (2015). Doğal Dil İşleme (Natural Language Processing). YBS Ansiklopedi 2(4), 14-31.
45. Shaw, P., Uszkoreit, J. ve Vaswani, A. (2018). Self-attention with relative position representations. arXiv, preprint arXiv:1803.02155. Erişim adresi https://doi.org/10.48550/arXiv.1803.02155
46. Sharma, A., Amrita, Chakraborty, S. ve Kumar, S. (2022). Named entity recognition in natural language processing: A systematic review. Proceedings of Second Doctoral Symposium on Computational Intelligence: DoSCI 2021 içinde, 817-828. Springer Singapore.
47. Singh, C., Hsu, A. R., Antonello, R., Jain, S., Huth, A. G., Yu, B., Gao, J. (2023). Explaining black box text modules in natural language with language models. arXiv, preprint arXiv:2305.09863. Erişim adresi https://doi.org/10.48550/arXiv.2305.09863
48. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I. (2017). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS), Long Beach, CA, USA, 4–9, 5998–6008.
49. Vigneau, M., Beaucousin, V., Hervé, P. Y., Duffau, H., Crivello, F., Houde, O., ... ve Tzourio-Mazoyer, N. (2006). Meta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing. Neuroimage, 30(4), 1414-1432.
50. Wang, S., Sun, J., Zhang, Y., Lin, N., Moens, M. F. ve Zong, C. (2024). Computational models to study language processing in the human brain: A survey. arXiv, preprint arXiv:2403.13368. Erişim adresi https://arxiv.org/pdf/2403.13368 (17.03.2025)
51. Zhang, X., Wang, S., Lin, N. ve Zong, C. (2022). Is the brain mechanism for hierarchical structure building universal across languages? an fmri study of chinese and english. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing içinde, 7852–7861.