Current Technologies for Information Retrieval of Documents in Digital Libraries: A Survey

Abstract

In recent years, the number of digital information sources available for different topics has grown enormously. Many of the systems that support access to these digital information resources are focused on scanning, searching and information retrieval tools. Digital libraries, electronic libraries and Web pages bring many new initiatives to improve information access, create and organize document collections hierarchically according to different key criteria. Different search tools; by using software-based services to organize, index and summarize documents that can be accessed using information retrieval techniques, a more comprehensive document coverage can be provided. In digital libraries; the techniques applied to search mechanisms have made it necessary to use different methods and technologies to manage document collections, to extract meaningful data and to determine document relationships. In particular, the relationship between documents cannot be clearly defined neither by their forms nor by their types. This study provides a comprehensive study of methods and techniques used for extracting metadata, named entity recognition, keyword extraction and obtaining document similarities from the content of
the documents for digital libraries.

Keywords

• Document processing
• Metadata extraction
• Name entity recognition
• Keyword extraction
• Document similarity

Dijital Kütüphanelerde Dokümanlardan Bilgi Geri Kazanımı için Kullanılan Güncel Teknolojiler: Derleme Çalışması

Abstract

Son yıllarda, farklı konular için sunulan dijital bilgi kaynaklarının sayısı aşırı miktarda artmaktadır. Bu dijital bilgi kaynaklarına erişim desteği sunan sistemlerin birçoğu tarama, arama ve bilgi geri kazanımı araçlarına odaklanmıştır. Sayısal kütüphaneler, elektronik kitaplıklar ve Web sayfaları, bilgi erişimini iyileştirmek, belge koleksiyonlarını farklı anahtar kriterlere göre hiyerarşik olarak oluşturmak ve düzenlemek için yeni birçok açılım sunmaktadır. Farklı arama araçları, bilgi erişim teknikleri kullanılarak erişilebilen belgeleri düzenlemek, endekslemek ve özetlemek için yazılım tabanlı hizmetleri kullanarak daha kapsamlı bir doküman kapsamı sunulabilmektedir. Dijital kütüphanelerdeki arama mekanizmalarına uygulanan teknolojiler, doküman koleksiyonlarını yönetmek, anlamlı veri çıkarmak ve doküman ilişkilerinin belirlenmesi için farklı yöntem ve teknolojilerin kullanımını zorunlu kılmıştır. Özellikle belgeler arasındaki ilişki ne biçimleri ne de türleri ile açıkça tanımlanamamaktadır. Bu çalışma, sayısal kütüphaneler için belgelerin içeriğinden üst-veri çıkarımı, varlık isimlerinin elde edilmesi, anahtar kelimelerin elde erilmesi ve doküman benzerliklerinin oluşturulması için kullanılan yöntem ve teknikler için kapsamlı bir çalışma sunmaktadır.

Keywords

• Doküman işleme
• Üst veri çıkarımı
• Varlık ismi tanıma
• Anahtar kelime çıkarımı
• Doküman benzerliği

References

1. [1] M. Afzali, “Karma Kütüphane: Dijital ve Geleneksel Kütüphanelerin Odak Noktası,” Türk Kütüphaneciliği, c. 22, s. 3, ss. 266-278, 2008.
2. [2] L. Masinter (1995). Document management, digital libraries and the Web [Online]. Available: http://www. cernet. edu. cn/HMP/PAPER/243/html/paper.htm
3. [3] V. Yadav, S. Bethard, “A Survey On Recent Advances In Named Entity Recognition From Deep Learning Models,” The 27th International Conference on Computational Linguistics (COLING), 2018, ss. 1-14.
4. [4] S. Beliga, “Keyword extraction: a review of methods and approaches,” University of Rijeka, Department of Informatics, Rijeka, 2014, ss. 1-9.
5. [5] S. Beliga, A. Meštrović and S. Martinčić-Ipšić, “An overview of graph-based keyword extraction methods and approaches,” Journal of information and organizational sciences, c. 39 s. 1, ss. 1-20, 2015.
6. [6] S. Chatvichienchai, “SEMEXSS - A Rule-Based Semantic Metadata Extraction System for Spreadsheets,” International Journal of Computer Theory and Engineering, c. 8, s. 2, ss. 102–108, 2016.
7. [7] K. Hamad and M. Kaya, “A Detailed Analysis of Optical Character Recognition Technology,” International Journal of Applied Mathematics, Electronics and Computers, c. 4, s. Special Issue-1, ss. 244–249, Dec. 2016.
8. [8] N. Sahu and M. Sonkusare, “A Study on Optical Character Recognition Techniques,” The International Journal of Computational Science, Information Technology and Control Engineering, c. 4, s. 1, ss. 01–15, Jan. 2017.

9. [9] A. Chaudhuri, K. Mandaviya, P. Badelia and S. K. Ghosh, “Optical Character Recognition Systems,” In Optical Character Recognition Systems for Different Languages with Soft Computing, c. 352, ss. 9–41, 2017.
10. [10] I. G. Councill, C. L. Giles, E. Di Iorio, M. Gori, M. Maggini, A. Pucci, “Towards Next Generation CiteSeer: A Flexible Architecture for Digital Library Deployment,” International Conference on Theory and Practice of Digital Libraries, 2006, ss. 111–122.
11. [11] J. Zhao and H. Liu, “Metadata Extraction Approach of PDF Documents Based on Measurement Fusion,” Journal of Multimedia, c. 8, s. 6, Nov. 2013.
12. [12] P. Flynn, L. Zhou, K. Maly, S. Zeil, M. Zubair, “Automated template-based metadata extraction architecture,” International Conference on Asian Digital Libraries, 2007, ss. 327-336.
13. [13] L. Kovriguina, A. Shipilo, F. Kozlov, M. Kolchin, E. Cherny, “Metadata extraction from conference proceedings using template-based approach”, Semantic Web Evaluation Challenges, 2015, ss. 153-164.
14. [14] Z. Huang, H. Jin, P. Yuan, Z. Han, “Header metadata extraction from semi-structured documents using template matching,” International Conferences On the Move to Meaningful Internet Systems, 2006, ss. 1776-1785.
15. [15] H. Han, C. L. Giles, E. Manavoglu, H. Zha, Z. Zhang, E. A. Fox, “Automatic Document Metadata Extraction using Support Vector Machines,” Joint Conference on Digital Libraries (JCDL03), 2003, ss. 37-49.
16. [16] L. Shi, R. Khushaba, S. Kodagoda, G. Dissanayake, “Application of CRF and SVM based semi-supervised learning for semantic labeling of environments,” 12th International Conference on Control Automation Robotics & Vision (ICARCV), 2012, ss. 835-840.
17. [17] H. Han, E. Manavoglu, H. Zha, K. Tsioutsiouliklis, C. L. Giles, X. Zhang, “Rule-based word clustering for document metadata extraction,” ACM symposium on Applied computing (SAC ’05), 2005, ss. 1049-1053.
18. [18] M. Granitzer, M. Hristakeva, K. Jack, R. Knight, “A comparison of metadata extraction techniques for crowdsourced bibliographic metadata management,” 27th Annual ACM Symposium on Applied Computing (SAC ’12), 2012, ss. 962-964.
19. [19] D. Misra, S. Chen, G. R. Thoma, “A System for Automated Extraction of Metadata from Scanned Documents using Layout Recognition and String Pattern Search Models,” Archiving, 2019, ss. 107–112.
20. [20] J. Azimjonov, J. Alikhanov, “Rule Based Metadata Extraction Framework from Academic Articles,” ArXiv, 2018, ss. 1-10.
21. [21] L. Runtao, L. Gao, D. An, Z. Jiang, Z. Tang, “Automatic document metadata extraction based on deep networks,” National CCF Conference on Natural Language Processing and Chinese Computing, 2017, ss. 305-317.
22. [22] I. Safder, S. Hassan, A. Visvizi, T. Noraset and R. Nawaz, “Deep Learning-based Extraction of Algorithmic Metadata in Full-Text Scholarly Documents,” Information Processing & Management, c. 57, s. 6, 102269, 2020.
23. [23] J. Greenberg, W. Klas, Metadata for Semantic and Social Applications, Dublin Core Metadata Initative and Universitätsverlag, Göttingen, 2008.
24. [24] M. Lipinski, K. Yao, C. Breitinger, J. Beel, B. Gipp, “Evaluation of header metadata extraction approaches and tools for scientific PDF documents,” 13th ACM/IEEE-CS joint conference on Digital libraries (JCDL ’13), 2013, ss. 385-386.
25. [25] E. Mannens, R. Verborgh, S. Hooland, L. Hauttekeete, T. Evens, S. Coppens and R. Walle, “On the Origin of Metadata,” Information, c. 3, s. 4, ss. 790–808, 2012.
26. [26] L. Kovriguina, A. Shipilo, F. Kozlov, M. Kolchin, and E. Cherny, Metadata Extraction from Conference Proceedings Using Template-Based Approach, in Semantic Web Evaluation Challenges, Springer International Publishing, 2015, ss. 153–164.
27. [27] Lisa F. Rau, “Extracting company names from text,” The Seventh IEEE Conference on Artificial Intelligence Application, 1991, ss. 29-32.
28. [28] Ö. Uzuner, et al. “2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text,” Journal of the American Medical Informatics Association, c.18, s. 5, ss. 552-556, 2011.
29. [29] I. Segura-Bedmar, P. Martínez, M. Herrero Zazo, “Semeval-2013 task 9: Extraction of drug-drug interactions from biomedical texts (ddiextraction 2013),” Second Joint Conference on Lexical and Computational Semantics (SEM), 2013, ss. 341-350.
30. [30] Piskorski, Jakub, et al. “The first cross-lingual challenge on recognition, normalization and matching of named entities in Slavic languages,” 6th Workshop on Balto-Slavic Natural Language Processing, 2017, ss. 76–85.
31. [31] D. Farmakiotou, et al. “Rule-based named entity recognition for Greek financial texts,” The Workshop on Computational lexicography and Multimedia Dictionaries (COMLEX 2000), 2000, ss. 75-78.
32. [32] Sang, Erik F., and Sabine Buchholz. “Introduction to the CoNLL-2000 shared task: Chunking,” 2nd workshop on Learning language in logic and the 4th conference on Computational natural language learning, 2000, ss. 127-132.
33. [33] L. Ratinov, D. Roth. “Design challenges and misconceptions in named entity recognition,” Thirteenth Conference on Computational Natural Language Learning (CoNLL '09), 2009, ss. 147-155.
34. [34] E. F. Sang, F. D. Meulder. “Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition,” The Seventh Conference on Natural Language Learning at HLT-NAACL, 2003, ss. 142-147.
35. [35] Z. Ju, J. Wang, F. Zhu, “Named Entity Recognition from Biomedical Text Using SVM,” 5th International Conference on Bioinformatics and Biomedical Engineering, 2011, ss. 1-4.
36. [36] A. Ekbal, R. Haque, S. Bandyopadhyay, “Named entity recognition in Bengali: A conditional random field approach,” Third International Joint Conference on Natural Language Processing, 2008.
37. [37] D. Zeng, C. Sun, L. Lin, and B. Liu, “LSTM-CRF for drug-named entity recognition,” Entropy, c. 19, s. 6, ss. 283, 2017.
38. [38] S. Morwal, N. Jahan, and D. Chopra, “Named entity recognition using hidden Markov model (HMM),” International Journal on Natural Language Computing, c. 4, ss. 15-23, 2012.
39. [39] G. T. Ngompé, S. Harispe, G. Zambrano, J. Montmain, and S. Mussard, “Detecting sections and entities in court decisions using HMM and CRF graphical models.” Advances in Knowledge Discovery and Management, ss. 61-86, 2019.
40. [40] J. Li, A. Sun, J. Han, and C. Li, “A survey on deep learning for named entity recognition.”, IEEE Transactions on Knowledge and Data Engineering, Early Access, 2021.
41. [41] C. Zhang, H. Xu, “Using Citation-KNN for automatic keyword assignment.” International Conference on Electronic Commerce and Business Intelligence, 2009, ss. 131-134.
42. [42] Witten, I. H., Paynter, G. W., Frank, E., Gutwin, C. and Nevill-Manning, C. G. “Kea: Practical automated keyphrase extraction,” Fourth ACM conference on Digital Libraries, 1999, ss. 129-152.
43. [43] K. Zhang, H. Xu, J. Tang, J. Li, “Keyword extraction using support vector machine,” International conference on web-age information management, 2016, ss. 85-96.
44. [44] A. K. John, L. Di Caro, G. Boella, “A supervised keyphrase extraction system,” 12th International Conference on Semantic Systems, 2016, ss. 57-62.
45. [45] M. R. Murty, J. V. R. Murthy, P. P. Reddy, S. C. Satapathy, “Statistical approach based keyword extraction aid dimensionality reduction,” International Conference on Information Systems Design and Intelligent Applications (INDIA), 2012, ss. 445-452.
46. [46] S. Beliga, A. Meštrović, and S. Martinčić-Ipšić, “An overview of graph-based keyword extraction methods and approaches,” Journal of information and organizational sciences, c. 39, s. 1, ss. 1-20, 2015.
47. [47] M. Shishigan, C. Ridings, “PageRank Uncovered,” Technical report, 2002, ss. 1-55.
48. [48] C. Florescu, C. Caragea, “An unsupervised approach to keyphrase extraction from scholarly documents,” 55th Annual Meeting of the Association for Computational Linguistics, 2017, ss. 1105-1115.
49. [49] R. Mihalcea, P. Tarau, “Bringing order into text”, Conference on Empirical Methods in Natural Language Processing, 2004, ss. 404-411.
50. [50] R. Campos, V. Mangaravite, A. Pasquali, A. Jorge, C. Nunes, and A. Jatowt, “YAKE! Keyword extraction from single documents using multiple local features”, Information Sciences, c. 509, ss. 257-289, 2020.
51. [51] D. A.Vega-Oliveros, P. S. Gomes, E. E. Milios, L. Berton, “A multi-centrality index for graph-based keyword extraction,” Information Processing & Management, c. 56, s. 6,102063, 2019.
52. [52] A. Tixier, F. Malliaros, M. Vazirgiannis, “A graph degeneracy-based approach to keyword extraction,” Conference on Empirical Methods in Natural Language Processing, 2016, ss. 1860-1870.
53. [53] F. C. Jonathan,O. Karnalim, “Semi-supervised keyphrase extraction on scientific article using fact-based sentiment,”Telkomnika, c. 16, s. 4, ss.1771-1778, 2018.
54. [54] H. M. Lynn, C. Choi, J. Choi, J. Shin, P. Kim, “The method of semi-supervised automatic keyword extraction for web documents using transition probability distribution generator,” International Conference on Research in Adaptive and Convergent Systems, 2016, ss. 1-6.
55. [55] Z. Runqiang, “Text Similarity Calculation Method Based on Levenshtein and TFRSF,” Computer and Modernization, c. 4, 2018.
56. [56] N. Gali, R. Mariescu-Istodor, D. Hostettler, P. Fränti, “Framework for syntactic string similarity measures,” Expert Systems with Applications, c. 129, ss. 169-185, 2019.
57. [57] S. Temma, M. Sugii, H. Matsuno, “The document similarity index based on the Jaccard distance for mail filtering,” 34th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), 2019, ss. 1-4.
58. [58] M. Alewiwi, C. Orencik, E. Savaş, “Efficient top-k similarity document search utilizing distributed file systems and cosine similarity,” Cluster Computing, c. 19, s. 1, ss. 109-126, 2016.
59. [59] N. Niraula, R. Banjade, D. Ştefănescu, V. Rus, “Experiments with semantic similarity measures based on lda and lsa,” International conference on statistical language and speech processing, 2013, ss. 188-199.
60. [60] M. Farouk, “Measuring Sentences Similarity: A Survey,” Indian Journal of Science. And Technology, c. 12, s. 25, ss. 1–11, Jul. 2019.
61. [61] C. Fellbaum, P. Vossen, P., “The Challenge of Multilingual WordNets,” Lexical Resources and Evaluation, c. 46, ss. 313-326, 2012.
62. [62] P. Resnik. (1995, Kasım). Using Information Content to Evaluate Semantic Similarity in a Taxonomy [Çevrimiçi]. Erişim: http://arxiv.org/abs/cmp-lg/9511007. Erişim Tarihi: 11 Eylül 2020.
63. [63] D. Lin, “Extracting collocations from text corpora”, First workshop on computational terminology, 1998, ss. 57-63.
64. [64] Z. Wu and M. Palmer. (1994, Haziran). Verb Semantics and Lexical Selection [Çevrimiçi]. Erişim: http://arxiv.org/abs/cmp-lg/9406033. Erişim Tarihi: 11 Eylül 2020.