Müze Sesli Betimlemelerinde İnsan ve Yapay Zekâ Üretimlerinin Karşılaştırılması: Derlem Temelli Bir Çalışma

Öz

Kültürel miras alanlarında sesli betimleme (SB), görme engelli ve az gören ziyaretçilerin müzelerdeki ve galerilerdeki çeşitli görsel içerikle etkileşim kurmasını sağlayan bir erişilebilirlik hizmetidir. Görsel-işitsel medyada daha geniş bir kitleye yönelik etkinleştirici hizmetler sunma ihtiyacının artmasıyla birlikte, yapay zekâ (YZ) araçlarının uygulanması kültürel miras bağlamlarında da giderek artmaktadır. Bu bağlamda, mevcut çalışma, seçilen durağan sanat eserlerinin sesli betimleme metinlerini oluşturarak YZ modellerinin ve araçlarının uygulanabilirliğini değerlendirmektedir. Bunu, üç yöntemle üretilen SB metninin derleme tabanlı dilbilimsel bir analizini uygulayarak yapmaktadır: (1) Uzmanlar tarafından yazılan SB metinlerinden oluşan İnsan Yazarlı SB Derlemi, (2) Eğitimsiz bir YZ modelinin oluşturduğu metinlerin bir derlemesi olan Temel YZ SB Derlemi ve (3) Etki alanına özel veri kümeleriyle eğitilmiş bir YZ modeli aracılığıyla oluşturulan SB metinlerinin bulunduğu Eğitimli YZ SB Derlemi. Derlem analizinin sonuçları, müze SB'sinin özelliklerine uygunluk çerçevesinde sunulmakta ve tartışılmaktadır. Bir doktora araştırmasının bir parçası olan bu çalışma, Türkçe müze SB'sinin dilbilimsel özelliklerini araştırmaya ve özellikle durağan sanat eserleri için SB metinleri üreten bir YZ modelini eğitmeye yönelik ilk girişimdir. Analizin sonuçları, insan tarafından yazılan metinler şimdilik altın standart olmaya devam etse de özelleştirilmiş veri setleriyle eğitilen YZ tabanlı otomatik betimleme sistemlerinin erişilebilirlik standartlarına uyum ve kapsayıcılık açısından güçlü betimleme araçlarının geliştirilmesini kolaylaştırma konusunda önemli bir potansiyele sahip olduğunu göstermektedir.

Anahtar Kelimeler

• sesli betimleme
• müze erişilebilirliği
• üretken yapay zeka
• kör
• az gören

Comparing Human and AI-Generated Museum Audio Descriptions: A Corpus-Based Linguistic Analysis

Öz

Audio description (AD) in cultural heritage settings serves as an accessibility service that enables blind and partially-sighted visitors to interact with the diverse visual content in museums and galleries. With the growing need to provide enabling services for a broader audience in audio-visual media, the implementation of artificial intelligence (AI) tools has become significant in cultural heritage contexts as well. In this context, the current study evaluates the applicability of Large Language Models (LLMs) and artificial intelligence (AI) tools through generating ADs of selected static artworks. It does so by implementing a corpus based linguistic analysis of AD texts that are produced by three modalities: (1) a Human-Authored AD Corpus; which consists of AD texts written by experts, (2) a Baseline AI AD Corpus which is a compilation of texts that an untrained AI model generates, and (3) a Trained AI AD Corpus in which AD texts are generated through an AI model that is specifically trained with domain-specific datasets. The results of the corpus analysis are presented and discussed within the framework of adherence to the characteristics of museum AD. As part of a vast doctoral research project, this study is the first attempt to investigate the linguistic attributes of Turkish museum AD and to train an AI model to generate AD texts for static artworks. The results of the analysis indicate that, while human-authored texts remain the gold standard for the time being, AI-based automatic description systems have significant potential to meet accessibility standards and facilitate the development of inclusive, semantically robust automatic description tools.

Anahtar Kelimeler

• audio-description
• museum accessibility
• generative artificial intelligence
• blind
• partially-sighted

Kaynakça

1. ACB (American Council of Blind) (2018). The audio description project. http://www.acb.org/adp/guidelines.html
2. Adams, E. (Ed.). (2021). Disability Studies and the classical body: The forgotten other (1st ed.). Routledge. https://doi.org/10.4324/9780429273711
3. ADP (The Audio Description Project). (2017). Audio description at museums, parks, and exhibits. http://www.acb.org/adp/museums.html
4. Ateşman, E. (1997). Türkçede okunabilirliğin ölçülmesi. Dil Dergisi, 58(71- 74). https://doi.org/10.54979/turkegitimdergisi.1176597
5. Balcı, M. (2022) Museums! How accessible for individuals with disabilities?. İDEALKENT, 16(45), 1493-1513. https://doi.org/10.31198/idealkent.1453559
6. Braun, S., & Starr, K. (Eds.). (2022). Innovation in audio description research. Routledge. https://doi.org/10.4324/9781003052968
7. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., & Amodei, D. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901. https://doi.org/10.48550/arXiv.2005.14165
8. Caramiaux, B. (2023). AI with museums and cultural heritage. AI in Museums, 117-130. https://doi.org/10.14361/9783839467107-011

9. COME-IN!. (2017). COME-IN! Guidelines for accessible museums. Description Guidelines. Edizioni Università di Trieste
10. Erkut, B. (2025). Uluslararası müzelerde yapay zekâ uygulamaları: ziyaretçi deneyimi ve koleksiyon yönetimi üzerine bir derleme. UNIMUSEUM, 8(1), 18-37.
11. Gao, Y., Fischer, L., Lintner, A., & Ebling, S. (2024). Audio description generation in the era of LLMs and VLMs: A Review of Transferable Generative AI Technologies. arXiv preprint arXiv:2410.08860. https://doi.org/10.48550/arXiv.2410.08860
12. Güngör, T. (1995). Computer processing of Turkish: Morphological and lexical investigation [Unpublished doctoral dissertation]. Boğaziçi University.
13. Hutchinson, R. S., & Eardley, A. F. (2019). Museum audio description: The problem of textual fidelity. Perspectives, 27(1), 42-57. https://doi.org/10.1080/0907676X.2018.1473451
14. Izsak, K., Terrier, A., Kreutzer, S., Strähle, T., Roche, C., Moretto, M., & Tomchak, D. (2022). Opportunities and challenges of artificial intelligence technologies for the cultural and creative sectors. Publications Office of the European Union.
15. Kilgarriff, A., Baisa, V., Bušta, J., Jakubíček, M., Kovář, V., Michelfeit, J., & Suchomel, V. (2014). The Sketch Engine: ten years on. Lexicography, 1(1), 7–36. https://doi.org/10.1007/s40607-014-0009-9
16. Kleymann, R., Niekler, A., & Burghardt, M. (2022). Conceptual forays: A corpus-based study of “theory” in digital humanities journals. Journal of Cultural Analytics, 7(4).
17. Maszerowska, A., Matamala, A., & Orero, P. (Eds.). (2014). Audio description: New perspectives illustrated (Vol. 112). John Benjamins Publishing Company.
18. Mazur, I. (2020). Audio description: Concepts, theories and research approaches. In Ł Bogucki & M. Deckert (Eds.), The Palgrave handbook of audio-visual translation and media accessibility (pp. 227-247). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030- 42105-2_12
19. Neves, J. (2012). Multi-sensory approaches to (audio) describing the visual arts. MonTI. Monografías de Traducción e Interpretación, (4), 277-293. http://dx.doi.org/10.6035/MonTI. 2012.4.12
20. Önenç Şahin,T. (forthcoming) The applicability of AI in a museum audio description context: A study on accessible artworks [Unpublished doctoral dissertation]. Hacettepe University, Graduate School of Social Sciences.
21. OpenAI. (2025). ChatGPT (April 14 version) [Large language model]. https://chat.openai.com/chat
22. Perego, E. (2019). Into the language of museum audio descriptions: a corpus-based study. Perspectives, 27(3), 333-349. https://doi.org/10.1080/0907676X.2018.1544648
23. Pisoni, G., Díaz-Rodríguez, N., Gijlers, H., & Tonolli, L. (2021). Human- centered artificial intelligence for designing accessible cultural heritage. Applied Sciences, 11(2), 870. https://doi.org/10.3390/app11020870
24. Saldanha, G., & O'Brien, S. (2014). Research methodologies in translation studies. Routledge.
25. Snyder, J. (2008). Audio description: The visual made verbal. International Journal of the Arts inSociety, 2(2), 99–104. https://doi.org/10.1016/j.ics.2005.05.215
26. Torné, A. F., & Matamala, A. (2015). Text-to-speech vs. human voiced audio descriptions: a reception study in films dubbed into Catalan. The Journal of Specialised Translation, (24), 61-88. https://doi.org/10.26034/cm.jostrans.2015.323
27. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M. A., Lacroix, T., & Lample, G. (2023). Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. https://doi.org/10.48550/arXiv.2302.13971
28. Voichur, O., Hovorushchenko, O., Boyarchuk, A., Voichur, Y., & Nester, A. (2025). Method of preprocessing information for preparing a description of art objects using artificial intelligence. Brain, 14, 15.
29. Wang, X., Crookes, D., Harding, S. A., & Johnston, D. (2022). Evaluating audio description and emotional engagement for BPS visitors in a museum context: An experimental perspective. Translation Spaces, 11(1), 134-156. https://doi.org/10.1075/ts.21019.wan
30. Weil, S. E. (2004). Rethinking the museum: An emerging new paradigm. In G. Anderson (Ed.). reinventing the Museum: Historical and contemporary perspectives on the paradigm shift (pp. 74-79). AltaMira Press, Lanhman.
31. Xie, J., Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2024). Autoad-zero: A training-free framework for zero-shot audio description. In Proceedings of the Asian Conference on Computer Vision (pp. 2265-2281). https://doi.org/10.48550/arXiv.2407.15850