Çok Aşamalı LLM Çıkarımı ve Denetlenebilir Operasyon Planlama ile Türkçe Finans Haberlerinden Alana Özgü Bilgi Grafı İnşası

Abstract

Dijital finans haber akışının sürekliliği ve çeşitliliği, metinden güvenilir bilgi çıkararak bunları alana özgü bir bilgi grafında biriktirmeyi kritik, ancak gürültüye açık bir sorun hâline getirmektedir. Önerilen yaklaşım, varlık, ilişki ve öznitelik çıkarımını üç aşamaya ayırmakta ve ardından denetlenebilir bir operasyon planlama katmanıyla her aday bilginin ekleme, güncelleme veya eleme kararını şema ve kanıt temelli politikalara göre belirlemektedir. 200 haberlik bir veri kümesi üzerinde, yöntemimiz ürettiği düğüm sayısını baz çizgiye kıyasla %19,56, ilişki sayısını ise %21,19 oranında azaltarak grafın kompaktlığını korurken hatalı verileri filtrelemiştir. Ayrıca, vektör tabanlı aday arama ile desteklenen bir güvenlik ağı mekanizması, ihtiyaç halinde büyük ya da küçük dil modelini devreye sokarak adlandırma varyantlarını kanonik temsillerde birleştirmektedir.

Keywords

• Bilgi grafı
• çok aşamalı çıkarım
• operasyon planlama
• Türkçe finans haberleri
• varlık çözümleme

Ethical Statement

Bu çalışma, kamuya açık kaynaklardan elde edilen metin verileri üzerinde gerçekleştirilmiştir. Çalışma; insan katılımcı, deney hayvanı veya kişisel veri içeren bir araştırma kapsamında olmadığından etik kurul onayı gerekmemektedir. Çalışmanın tüm süreçlerinin araştırma ve yayın etiğine uygun yürütüldüğü, etik kurallara ve bilimsel atıf gösterme ilkelerine uyulduğu beyan edilir.

Domain-Specific Knowledge Graph Construction from Turkish Financial News with Multi-Stage LLM Extraction and Auditable Operation Planning

Abstract

The continuous flow and diversity of Turkish financial news make the task of extracting reliable facts and maintaining them in a domain-specific knowledge graph both valuable and challenging. We propose a three-stage extraction workflow that separates entities, relations, and attributes, and then applies an auditable operation planning mechanism to decide whether each candidate fact should be inserted, updated, or rejected based on schema- and evidence-based policies. On 200 articles, the approach yields 19.56% fewer nodes and 21.19% fewer relations compared to a baseline, effectively filtering noise while keeping the knowledge graph compact. In addition, we employ a vector-based candidate retrieval with a safety net that can dynamically invoke either a large or a small language model to consolidate naming variants into canonical entities.

Keywords

• Knowledge graph
• multi-stage extraction
• operation planning
• Turkish financial news
• entity resolution

Ethical Statement

This study was conducted using textual data obtained from publicly available sources. Since the study does not involve human participants, experimental animals, or personal data, ethical committee approval was not required. It is declared that all stages of the study were carried out in accordance with the principles of research and publication ethics, and that ethical rules and scientific citation practices were followed.

References

1. A. Hogan et al., “Knowledge Graphs,” ACM Comput. Surv., vol. 54, no. 4, Jul. 2021, doi: 10.1145/3447772.
2. S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives, “DBpedia: A Nucleus for a Web of Open Data,” Springer-Verlag, 2007, pp. 722–735. doi: 10.1007/978-3-540-76298-0_52.
3. F. M. Suchanek, G. Kasneci, and G. Weikum, “Yago: a core of semantic knowledge,” in Proceedings of the 16th international conference on World Wide Web, New York, NY, USA: ACM, May 2007, pp. 697–706. doi: 10.1145/1242572.1242667.
4. D. Vrandečić and M. Krötzsch, “Wikidata: a free collaborative knowledgebase,” Commun ACM, vol. 57, no. 10, pp. 78–85, Sep. 2014, doi: 10.1145/2629489.
5. X. Dong et al., “Knowledge vault: a web-scale approach to probabilistic knowledge fusion,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, New York, NY, USA: ACM, Aug. 2014, pp. 601–610. doi: 10.1145/2623330.2623623.
6. B. Abu-Salih, “Domain-specific knowledge graphs: A survey,” Journal of Network and Computer Applications, vol. 185, p. 103076, Jul. 2021, doi: 10.1016/j.jnca.2021.103076.
7. B. P. Jeyaraman, B. T. Dai, and Y. Fang, “A Comprehensive Review of Financial Knowledge Graphs,” World Scientific Annual Review of Artificial Intelligence, 2025, doi: 10.1142/s2811032325300014.
8. K. Oflazer, “Türkçe ve Doğal Dil İşleme,” Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, vol. 5, no. 2, 2016.

9. Ç. Çöltekin, A. S. Doğruöz, and Ö. Çetinoğlu, “Resources for Turkish natural language processing: A critical survey,” Lang Resour Eval, vol. 57, no. 1, 2023, doi: 10.1007/s10579-022-09605-4.
10. Z. Ji et al., “Survey of Hallucination in Natural Language Generation,” ACM Comput Surv, vol. 55, no. 12, pp. 1–38, Dec. 2023, doi: 10.1145/3571730.
11. A. Piscopo, L. A. Kaffee, C. Phethean, and E. Simperl, “Provenance information in a collaborative knowledge graph: An evaluation of wikidata external references,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2017. doi: 10.1007/978-3-319-68288-4_32.
12. P. Rasmussen, P. Paliychuk, T. Beauvais, J. Ryan, and D. Chalef, “Zep: A Temporal Knowledge Graph Architecture for Agent Memory,” Jan. 2025, doi: https://doi.org/10.48550/arXiv.2501.13956.
13. X. V. Li and F. Sanna Passino, “FinDKG: Dynamic Knowledge Graphs with Large Language Models for Detecting Global Trends in Financial Markets,” in ICAIF 2024 - 5th ACM International Conference on AI in Finance, 2024. doi: 10.1145/3677052.3698603.
14. A. Arun, F. Dimino, T. P. Agarwal, B. Sarmah, and S. Pasquali, “FinReflectKG: Agentic Construction and Evaluation of Financial Knowledge Graphs,” in Proceedings of the 6th ACM International Conference on AI in Finance, New York, NY, USA: ACM, Nov. 2025, pp. 283–290. doi: 10.1145/3768292.3770363.
15. S. Choi and Y. Jung, “Knowledge Graph Construction: Extraction, Learning, and Evaluation,” Applied Sciences, vol. 15, no. 7, p. 3727, Mar. 2025, doi: 10.3390/app15073727.
16. C. Niklaus, M. Cetto, A. Freitas, and S. Handschuh, “A Survey on Open Information Extraction,” in Proceedings of the 27th International Conference on Computational Linguistics, E. M. Bender, L. Derczynski, and P. Isabelle, Eds., Santa Fe, New Mexico, USA: Association for Computational Linguistics, Aug. 2018, pp. 3866–3878. [Online]. Available: https://aclanthology.org/C18-1326/
17. D. C. Wimalasuriya and Dejing Dou, “Ontology-based information extraction: An introduction and a survey of current approaches,” J Inf Sci, vol. 36, no. 3, pp. 306–323, Jun. 2010, doi: 10.1177/0165551509360123.
18. S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu, “Unifying Large Language Models and Knowledge Graphs: A Roadmap,” IEEE Trans Knowl Data Eng, vol. 36, no. 7, pp. 3580–3599, Jul. 2024, doi: 10.1109/TKDE.2024.3352100.
19. B. Zhang and H. Soh, “Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction,” in EMNLP 2024 - 2024 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, 2024. doi: 10.18653/v1/2024.emnlp-main.548.
20. J. Bai et al., “AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora,” 2025. [Online]. Available: https://arxiv.org/abs/2505.23628
21. H. Chen, C. Zhang, J. Li, P. S. Yu, and N. Jing, “KGGen: A Generative Approach for Incipient Knowledge Graph Population,” IEEE Trans Knowl Data Eng, vol. 34, no. 5, 2022, doi: 10.1109/TKDE.2020.3014166.
22. Y. Lairgi, L. Moncla, R. Cazabet, K. Benabdeslem, and P. Cléau, “iText2KG: Incremental Knowledge Graphs Construction Using Large Language Models,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2025. doi: 10.1007/978-981-96-0573-6_16.
23. J. Wei et al., “Chain-of-thought prompting elicits reasoning in large language models,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, in NIPS ’22. Red Hook, NY, USA: Curran Associates Inc., 2022.
24. S. Yao et al., “Tree of Thoughts: Deliberate Problem Solving with Large Language Models,” in Advances in Neural Information Processing Systems, 2023.
25. X. Tian, J. Xu, S. Wang, and Y. Zhou, “Construction and Application of a Multi-Modal Knowledge Graph Integrated with Large Language Models in the Field of Manufacturing Processes,” in 2025 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), IEEE, Feb. 2025, pp. 0288–0292. doi: 10.1109/ICAIIC64266.2025.10920784
26. P. Christen, Data Matching. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. doi: 10.1007/978-3-642-31164-2.
27. [27] D. Rao, P. McNamee, and M. Dredze, “Entity Linking: Finding Extracted Entities in a Knowledge Base,” 2013. doi: 10.1007/978-3-642-28569-1_5.
28. D. Meher, C. Domeniconi, and G. Correa-Cabrera, “CORE-KG: An LLM-Driven Knowledge Graph Construction Framework for Human Smuggling Networks,” Jun. 2025, doi: 10.48550/arXiv.2506.21607.
29. P. Yang, B. Song, and Z. Zhang, “Research on Knowledge Graph Construction Methods for News Domain,” Academic Journal of Science and Technology, vol. 11, no. 1, pp. 58–64, May 2024, doi: 10.54097/dfdnfs56.
30. O. Çiftçi, F. Soygazi, and S. Tekir, “Enrichment of Turkish question answering systems using knowledge graphs,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 32, pp. 516–533, Dec. 2024, doi: 10.55730/1300-0632.4085.
31. U. Qudus, M. Röder, M. Saleem, and A.-C. Ngonga Ngomo, “Fact Checking Knowledge Graphs - A Survey,” ACM Comput Surv, vol. 58, no. 1, pp. 1–36, Jan. 2026, doi: 10.1145/3749838.
32. J. Gao, X. Li, Y. E. Xu, B. Sisman, X. L. Dong, and J. Yang, “Efficient knowledge graph accuracy evaluation,” Proceedings of the VLDB Endowment, vol. 12, no. 11, pp. 1679–1691, Jul. 2019, doi: 10.14778/3342263.3342642.
33. M. Barry et al., “GraphRAG: Leveraging Graph-Based Efficiency to Minimize Hallucinations in LLM-Driven RAG for Finance Data,” in Proceedings of the Workshop on Generative AI and Knowledge Graphs (GenAIK), G. A. Gesese, H. Sack, H. Paulheim, A. Merono-Penuela, and L. Chen, Eds., Abu Dhabi, UAE: International Committee on Computational Linguistics, Jan. 2025, pp. 54–65. [Online]. Available: https://aclanthology.org/2025.genaik-1.6/
34. T. T. Procko and O. Ochoa, “Graph Retrieval-Augmented Generation for Large Language Models: A Survey,” in 2024 Conference on AI, Science, Engineering, and Technology (AIxSET), IEEE, Sep. 2024, pp. 166–169. doi: 10.1109/AIxSET62544.2024.00030.
35. Y. Bang et al., “A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity,” in Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Stroudsburg, PA, USA: Association for Computational Linguistics, 2023, pp. 675–718. doi: 10.18653/v1/2023.ijcnlp-main.45.