Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI

Ali Deveci; Mehmet Ali Erkan; İhsan Tolga Medeni; Tunç Durmuş Medeni

doi:10.29130/dubited.1770847

TR EN

Hukuki Yapay Zekanın BERT ile Desteklenmesi: Bekenbey AI’nin İkinci İterasyonu

Abstract

Son yıllarda teknolojinin hızlı gelişimi, makine öğrenmesi modelleri ve yapay zekâ sistemlerinin yalnızca mühendislik alanında değil, aynı zamanda sosyal bilimlerdeki karmaşık problemlerin çözümünde de kullanılmasını zorunlu kılmıştır. Bu çalışmada, hukuk alanı ile üretken yapay zekâ arasında bir bağlantı kuran literatürdeki ilk çalışma olma özelliğini taşıyan Bekenbey AI modeli üzerine inşa ediyoruz. Temel model, çeşitli hukuki uygulamalarda dikkate değer bir doğruluk ve sağlamlık sergilemiş olsa da, doğal dil işleme (NLP) alanındaki son gelişmeler—özellikle Transformer tabanlı yaklaşımlar—modelin performansını ve uyarlanabilirliğini geliştirmek için yeni fırsatlar sunmaktadır. Bu sürümdeki temel amacımız, mevcut Bekenbey AI yapısına Bidirectional Encoder Representations from Transformers (BERT) modelini entegre ederek, hukuki metin işlemenin hassasiyetini artırmak, belge sınıflandırmasını geliştirmek ve daha incelikli öngörüsel analizler sağlamaktır. BERT’in öz-dikkat mekanizmalarından ve güçlü bağlamsal gömme yeteneklerinden yararlanarak, hukuki dile özgü karmaşık anlamsal yapıları daha yüksek doğrulukla ele almayı amaçlıyoruz. Bu çalışma, BERT’in benimsenmesinin ardındaki motivasyonu ortaya koymakta, Bekenbey AI çerçevesine getirdiği temel iyileştirmeleri açıklamakta ve bu alana ilişkin bir yol haritası sunmaktadır.

Keywords

Bekenbey Yapay Zeka, BERT, Hukuksal Belge Analizi, Hukuk

Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI

Abstract

In recent years, the rapid advancement of technology has necessitated the use of machine learning models and artificial intelligence systems not only in the field of engineering but also in the resolution of complex problems in social sciences. In this study, we build upon the Bekenbey AI model, which is the first study in the literature to establish a connection between the legal domain and Generative AI. While the foundational model showed significant accuracy and robustness across various legal applications, recent advancements in natural language processing (NLP)—particularly in Transformer-based approaches—present new opportunities to enhance the model’s performance and adaptability. Our primary objective in this version is to integrate Bidirectional Encoder Representations from Transformers (BERT) into the existing Bekenbey AI pipeline to refine legal text processing, improve document classification, and provide more nuanced predictive analytics. By benefiting from BERT’s self-attention mechanisms and its powerful contextual embedding capabilities, this study aims to handle the complex semantic structures inherent in legal language with greater precision. The proposed approach enhances both classification accuracy and text generation quality, demonstrating the effectiveness of integrating domain-specific attention mechanisms with transformer-based models in legal AI applications. Experimental results indicate that the proposed model consistently outperforms the baseline across multiple evaluation metrics.

Keywords

Bekenbey AI, BERT, Legal document analysis, Law

Supporting Institution

This research received no external funding.

Ethical Statement

This study does not involve human or animal participants. All procedures followed scientific and ethical principles, and all referenced studies are appropriately cited.

Thanks

The authors do not wish to acknowledge any individual or institution.

References

Abimbola, B., de La Cal Marin, E., & Tan, Q. (2024). Enhancing legal sentiment analysis: A convolutional neural network–long short-term memory document-level model. Machine Learning and Knowledge Extraction, 6(2), 877–897. https://doi.org/10.3390/make6020041
Aletras, N., Tsarapatsanis, D., Preoţiuc-Pietro, D., & Lampos, V. (2016). Predicting judicial decisions of the European Court of Human Rights: A natural language processing perspective. Peerj Computer Science, 2, Article e93.
Alhalalmeh, A. H., & Al-Tarawneh, A. (2025). Artificial intelligence and the law: The complexities of technology and legalities. In Intelligence-Driven Circular Economy: Regeneration Towards Sustainability and Social Responsibility (pp. 641–649). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-74220-0_50
Ashford, N. A., & Hall, R. P. (2011). Technology, globalization, and sustainable development: Transforming the industrial state. Yale University Press. https://doi.org/10.12987/9780300177602
Barbosa, T. S., de Castro, D., Singh, A. K., & Vitale, S. (2026). An experimental assessment of ai-based legal decision-making systems in contract analysis and risk detection. Qubahan Techno Journal, 5(1), 37–65. https://doi.org/10.48161/qtj.v5n1a81
Barnett, R. E. (2014). The structure of liberty: Justice and the rule of law (2nd ed.). Oxford University Press. https://doi.org/10.1093/0198297297.001.0001
Bhambhoria, R., Dahan, S., Li, J., & Zhu, X. (2026). Evaluating AI for law: Bridging the gap with open-source solutions. In Compliance for Artificial Intelligence Systems: Strategies, Principles and Methods (pp. 59–74). Springer Nature Switzerland. https://doi.org/10.1007/978-3-032-12795-2_5
Binns, R. (2018). Fairness in machine learning: Lessons from political philosophy. In Conference on Fairness, Accountability and Transparency (Vol. 81, pp. 149–159). PMLR. https://proceedings.mlr.press/v81/binns18a.html
Cheong, B. C. (2024). Transparency and accountability in AI systems: safeguarding wellbeing in the age of algorithmic decision-making. Frontiers in Human Dynamics, 6, Article 1421273. https://doi.org/10.3389/fhumd.2024.1421273
Chen, Q., Wu, Q., Chen, J., Wu, Q., van den Hengel, A., & Tan, M. (2020). Scripted video generation with a bottom-up generative adversarial network. IEEE Transactions on Image Processing, 29, 7454–7467. https://doi.org/10.1109/TIP.2020.3003227

Council of Europe. (1950, November 4). European Convention on Human Rights. https://www.echr.coe.int/documents/d/echr/convention_eng
Dalmia, S., Okhonko, D., Lewis, M., Edunov, S., Watanabe, S., Metze, F., Zettlemoyer, L., & Mohamed, A. (2023). Legonn: Building modular encoder-decoder models. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31, 3112–3126. https://doi.org/10.1109/TASLP.2023.3296019
Deveci, A., Erkan, M. A., Medeni, İ. T., & Medeni, T. D. (2026a). A novel framework for next word prediction using long-short term memory networks. Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen ve Mühendislik Dergisi, 28(82), 113–120. https://doi.org/10.21205/deufmd.2026288215
Deveci, A., Erkan, M. A., Medeni, İ. T., & Medeni, T. D. (2026b). Innovative approaches to high-energy gamma particle classification: A coevolutionary artificial neural network for cherenkov telescopes. Sivas Cumhuriyet Üniversitesi Mühendislik Fakültesi Dergisi, 4(1), 8–15. https://doi.org/10.66248/cumfad.1869125
Gul, S., Asif, M., Fazal-e-Amin, Saleem, K., & Imran, M. (2025). Advancing aspect-based sentiment analysis in course evaluation: A multi-task learning framework with selective paraphrasing. IEEE Access, 13, 7764–7779. https://doi.org/10.1109/ACCESS.2025.3527367
Herasymenko, O., Artemov, V., Kravtsev, O., Yunin, O., & Fedorchuk, Y. (2025). AI use by ukrainian law enforcement in combating crime at critical infrastructure facilities. Salud, Ciencia y Tecnología-Serie de Conferencias, 4, Article 1318. https://doi.org/10.56294/sctconf20251318
Ingebrand, T., Thorpe, A. J., Goswami, S., Kumar, K., & Topcu, U. (2025). Basis-to-basis operator learning using function encoders. Computer Methods in Applied Mechanics and Engineering, 435, Article 117646. https://doi.org/10.1016/j.cma.2024.117646
Katz, D. M., Bommarito II, M. J., & Blackman, J. (2017). A general approach for predicting the behavior of the Supreme Court of the United States. PloS One, 12(4), Article e0174698. https://doi.org/10.1371/journal.pone.0174698
Maurya, A. (2025). Scaling legal AI: Benchmarking Mamba and transformers for statutory classification and case law retrieval. arXiv. https://doi.org/10.48550/arXiv.2509.00141
Neelamegam, P., & Nirmala, S. J. (2025). Recent trends in legal AI: A comprehensive review. In 2025 Third International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) (pp. 152–159). IEEE. https://doi.org/10.1109/ICAISS61471.2025.11042154
Ni, J., Ren, S., Tang, G., Cao, W., & Shi, P. (2023). An improved shared encoder-based model for fast panoptic segmentation. IEEE Sensors Journal, 24(14), 22070–22083. https://doi.org/10.1109/JSEN.2023.3332354
Nirmal, B. C., & Singh, R. K. (Eds.). (2018). Contemporary issues in international law: Environment, international trade, information technology and legal education. Springer Nature Singapore. https://doi.org/10.1007/978-981-10-6277-3
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8), Article 9.
Schabas, W. A. (2015). The European convention on human rights: A commentary. Oxford University Press. https://doi.org/10.1093/law/9780199594061.001.0001
Siino, M., Falco, M., Croce, D., & Rosso, P. (2025). Exploring LLMs applications in law: A literature review on current legal NLP approaches. IEEE Access, 13, 18253–18276. https://doi.org/10.1109/ACCESS.2025.3533217
Sun, S., Che, L., Zhao, R., Chen, Y., & Li, M. (2025). Multi-task learning and voltage reconstruction-based battery degradation prediction under variable operating conditions of energy storage applications. Energy, 317(5), Article 134645. https://doi.org/10.1016/j.energy.2025.134645
United Nations General Assembly. (1948, December 10). Universal declaration of human rights. https://www.un.org/en/about-us/universal-declaration-of-human-rights
Ye, H., Sun, Y., Gao, Y., Xu, F., & Qi, J. (2025). Heterogeneous Graph Transformer Auto-Encoder for multivariate time series forecasting. Computers and Electrical Engineering, 122, Article 109927. https://doi.org/10.1016/j.compeleceng.2024.109927
Yücesan, E., Erkan, M. A., Deveci, A., & Medeni, İ. T. (2024). Bekenbey AI: Innovative solutions at the intersection of deep learning and law. Sivas Cumhuriyet Üniversitesi Mühendislik Fakültesi Dergisi, 2(2), 185–192. https://doi.org/10.66248/cumfad.1590764
Zoetmulder, R., Gavves, E., Caan, M., & Marquering, H. (2022). Domain-and task-specific transfer learning for medical segmentation tasks. Computer Methods and Programs in Biomedicine, 214, Article 106539. https://doi.org/10.1016/j.cmpb.2021.106539

Details

Primary Language

English

Subjects

Deep Learning

Journal Section

Research Article

Authors

Ali Deveci ^*
0000-0002-4990-0785
Türkiye

Mehmet Ali Erkan
0009-0007-5760-1914
Türkiye

İhsan Tolga Medeni
0000-0002-0642-7908
United Kingdom

Tunç Durmuş Medeni
0000-0002-2964-3320
Türkiye

Publication Date

July 24, 2026

Submission Date

August 30, 2025

Acceptance Date

May 18, 2026

Published in Issue

Year 2026 Volume: 14 Number: 3

DOI

https://doi.org/10.29130/dubited.1770847

IZ

https://izlik.org/JA47AY97EP

APA

Deveci, A., Erkan, M. A., Medeni, İ. T., & Medeni, T. D. (2026). Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI. Duzce University Journal of Science and Technology, 14(3), 705-718. https://doi.org/10.29130/dubited.1770847

AMA

1.Deveci A, Erkan MA, Medeni İT, Medeni TD. Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI. DUBİTED. 2026;14(3):705-718. doi:10.29130/dubited.1770847

Chicago

Deveci, Ali, Mehmet Ali Erkan, İhsan Tolga Medeni, and Tunç Durmuş Medeni. 2026. “Augmenting Legal Reasoning With BERT: The Second Iteration of Bekenbey AI”. Duzce University Journal of Science and Technology 14 (3): 705-18. https://doi.org/10.29130/dubited.1770847.

EndNote

Deveci A, Erkan MA, Medeni İT, Medeni TD (July 1, 2026) Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI. Duzce University Journal of Science and Technology 14 3 705–718.

IEEE

[1]A. Deveci, M. A. Erkan, İ. T. Medeni, and T. D. Medeni, “Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI”, DUBİTED, vol. 14, no. 3, pp. 705–718, July 2026, doi: 10.29130/dubited.1770847.

ISNAD

Deveci, Ali - Erkan, Mehmet Ali - Medeni, İhsan Tolga - Medeni, Tunç Durmuş. “Augmenting Legal Reasoning With BERT: The Second Iteration of Bekenbey AI”. Duzce University Journal of Science and Technology 14/3 (July 1, 2026): 705-718. https://doi.org/10.29130/dubited.1770847.

JAMA

1.Deveci A, Erkan MA, Medeni İT, Medeni TD. Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI. DUBİTED. 2026;14:705–718.

MLA

Deveci, Ali, et al. “Augmenting Legal Reasoning With BERT: The Second Iteration of Bekenbey AI”. Duzce University Journal of Science and Technology, vol. 14, no. 3, July 2026, pp. 705-18, doi:10.29130/dubited.1770847.

Vancouver

1.Ali Deveci, Mehmet Ali Erkan, İhsan Tolga Medeni, Tunç Durmuş Medeni. Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI. DUBİTED. 2026 Jul. 1;14(3):705-18. doi:10.29130/dubited.1770847

Hukuki Yapay Zekanın BERT ile Desteklenmesi: Bekenbey AI’nin İkinci İterasyonu

Abstract

Keywords

Augmenting Legal Reasoning with BERT: The Second Iteration of Bekenbey AI

Abstract

Keywords

Supporting Institution

Ethical Statement

Thanks

References

Details

Primary Language

Subjects

Journal Section

Authors

Publication Date

Submission Date

Acceptance Date

Published in Issue

DOI

IZ

Cite