A Systematic Mapping Review of Image Annotation Studies for Obtaining Information Retrieval from Images

Arda Sezen; Çigdem Turhan

Research Article

A Systematic Mapping Review of Image Annotation Studies for Obtaining Information Retrieval from Images

Year 2020, Volume: 13 Issue: 4, 423 - 434, 30.10.2020

Arda Sezen , Çigdem Turhan

Abstract

Image annotation concept which aims to obtain and process image metadata from various kinds of images, to achieve meaningful results has become more and more critical in the information society in retrieving information from images. There are many approaches proposed in the literature, however, choosing the most appropriate ones is not an easy task. In this study, the key approaches on image annotation have been investigated through a systematic mapping with an extensive literature review. The novelty of our study is that it represents the first attempt to explore, investigate, map and analyze image annotation studies in literature and to help classify the studies, reveal research gaps, and prepare a visual summary for the research questions presented in this paper. The literature recommends the systematic mapping approach to investigate an area of interest from different perspectives. For this purpose, 95 studies were selected from a total of 404 studies identified. The examination of the literature on the domain shows that the available methods/techniques, tools, metrics, processes, or other technical approaches are not enough to produce a complete solution on their own. The necessity of generating new solutions by combining the proposed techniques and considering interdisciplinary approaches are also suggested.

Keywords

semantic image annotation, image processing, picture interpretation

References

V. Lavrenko, R. Manmatha, J. Jeon, “A Model for Learning the Semantics of Pictures”, Advances in Neural Information Processing Systems 17, Vancouver, Canada, 553-560, 13-18 December, 2004.
Internet: Halaschek, C. Wiener, Image Annotation on the Semantic Web, https://www.w3.org/2005/Incubator/mmsem/XGR-image-annotation/#annot_intro, 03.05.2019.
B. Kitchenham, S. Charters, Guidelines for Performing Systematic Literature Reviews in Software Engineering, Keele University, UK, 2007.
B. Kitchenham, et al., “Systematic literature reviews in software engineering – A systematic literature review”, Information and Software Technology, 51(1), 7-15, 2009.
K. Petersen, et al., “Systematic Mapping Studies in Software Engineering”, 12th International Conference on Evaluation and Assessment in Software Engineering, Bari, Italy, 68-77, 26-27 June, 2008.
J. Jill, L. Matheson, F. M. Lacey, Doing Your Literature Review: traditional and systematic techniques, SAGE Publishing, London, UK, 2011.
Internet: A. Sezen, Ç. Turhan, SM – Classification Scheme, https://drive.google.com/open?id=1cnItNure_P7oolIcoVmDqP5EXJMim00-HPOweH_qPDA, 10.10.2019.
X. Wang, et al., “High-level semantic image annotation based on hot Internet topics”, Multimedia Tools and Applications, 74(6), 2055-2084, 2015.
H. Bannour, C. Hudelot, “Building and using fuzzy multimedia ontologies for semantic image annotation”, Multimedia Tools and Applications, 72(3), 2107-2141, 2014.
A. Fakhari, A. M. E. Moghadam, “Combination of classification and regression in decision tree for multi-labeling image annotation and retrieval”, Applied Soft Computing, 13(2), 1292-1302, 2013.
F. Kınlı, F. Kıraç, “FashionCapsNet: Clothing Classification with Capsule Networks”, International Journal of Informatics Technologies, 13(1), 87-96, 2020.
Internet: Y. Gong, et al., Deep Convolutional Ranking for Multilabel Image Annotation, https://arxiv.org/abs/1312.4894, 03.05.2019.
R. Hong, et al., “Image Annotation by Multiple-Instance Learning With Discriminative Feature Mapping and Selection”, IEEE Transactions on Cybernetics, 44(5), 669-680, 2014.
S. Zhang, et al., “ObjectPatchNet: Towards scalable and semantic image annotation and retrieval”, Computer Vision and Image Understanding, 118(1), 16-29, 2014.
D-H. Im, G-D. Park, “STAG: Semantic Image Annotation Using Relationships between Tags”, International Conference on Information Science and Applications, Suwon, South Korea, 1-2, 24-26 June, 2013.
L. Seneviratne, E. Izquierdo, “A mathematical approach towards semi-automatic image annotation”, 19th European Signal Processing Conference, Barcelona, Spain, 559-563, 29 August-2 September, 2011.
A. Ç. Seçkin, “Labeling System with EEG, EMG, and IMU for Visual Training of Autonomous Vehicles”, International Journal of Informatics Technologies, 12(4), 299-305, 2019.
S-B. Chan, H. Yamana, S. Satoh, “A Negative Sample Image Selection Method Referring to Semantic Hierarchical Structure for Image Annotation”, International Conference on Signal-Image Technology & Internet-Based Systems, Kyoto, Japan, 162-167, 2-5 December, 2013.
X. Liu, H. Yao, R. Ji, “Exploring statistical properties for semantic annotation: sparse distributed and convergent assumptions for keywords”, IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, USA, 802-805, 14-19 March, 2010.
Z. Mehmood, T. Mahmood, M. A. Javid, “Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine”, Applied Intelligence, 48(1), 166-181, 2018.
F. Bulut, “Performance Analysis of Ensemble Methods on Imbalanced Datasets”, International Journal of Informatics Technologies, 9(2), 153-159, 2016.
S. Ali, et al., “A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation”, IEEE Transactions on Software Engineering, 36(6), 742-762, 2010.
F. Paz, J. A. Pow-Sang, “Systematic Mapping Review of Usability Evaluation Methods for Software Development Process”. International Journal of Software Engineering and Its Applications, 10(1), 165-178, 2016.
V. R. Basili, Software Modeling and Measurement: The Goal/Question/Metric Paradigm, University of Maryland, USA, 1992.
F. A. Silva, et al., “Benchmark applications used in mobile cloud computing research: a systematic mapping study”, The Journal of Supercomputing, 72(4), 1431-1452, 2016.
Internet: Y. Gong, et al., Deep Convolutional Ranking for Multilabel Image Annotation, https://arxiv.org/abs/1312.4894, 03.05.2019.
I. Banerjee, et al., “Graphical user interface (GUI) testing: Systematic mapping and repository”, Information and Software Technology, 55(10), 1679-1694, 2013.
N. Bencomo, S. Hallsteinsen, E. S. de Almeida, “A View of the Dynamic Software Product Line Landscape”, IEEE Computer, 45(10), 36-41, 2012.
S. Easterbrook, et al., "Selecting Empirical Methods for Software Engineering Research", Guide to Advanced Empirical Software Engineering, Editor: F. Shull, J. Singer, D. I. K. Sjøberg, Springer, London, UK, 285-311, 2008.
T. Kanimozhi, T., A. Christy, “Incorporating ontology and SPARQL for semantic image annotation”, IEEE Conference on Information & Communication Technologies, Thuckalay, Tamil Nadu, India, 26-31, 11-12 April, 2013.
D-H. Im, G-D Park, “Linked tag: image annotation using semantic relationships between image tags”, Multimedia Tools and Applications, 74(7), 2273-2287, 2015.
D. Wang, et al., “Mining Weakly Labeled Web Facial Images for Search-Based Face Annotation”, IEEE Transactions on Knowledge and Data Engineering, 26(1), 166-179, 2014.
T. Sjekavica, G. Gledec, M. Horvat, “Multimedia Annotation using Semantic Web Technologies”, 7th European Computing Conference, Dubrovnik, Croatia, 228-233, 25-27 June, 2013.
Internet: G. Ciocca, P. Napoletano, R. Schettini, IAT - Image Annotation Tool: Manual, https://arxiv.org/abs/1502.05212, 17.05.2019.
C. Kurtz, et al., “On combining image-based and ontological semantic dissimilarities for medical image retrieval applications”, Medical Image Analysis, 18(7), 1082-1100, 2014.
K. C. Wang, et al., “Ontology-based Image Navigation: Exploring 3.0-T MR Neurography of the Brachial Plexus Using AIM and RadLex”, RadioGraphics, 35(1), 142-151, 2015.
M. Grassi, et al., “Pundit: Creating, Exploring and Consuming Semantic Annotations”, 3rd International Workshop on Semantic Digital Archives, Valetta, Malta, 65-72, 26 September, 2013.
D. K. Iakovidis, et al., “Ratsnake: A Versatile Image Annotation Tool with Application to Computer-Aided Diagnosis”, The Scientific World Journal, 2014(286856), 1-12, 2014.
A. C. Bukhari, M. Krauthammer, C. J. O. Baker, “SEBI: An Architecture for Biomedical Image Discovery, Interoperability and Reusability based on Semantic Enrichment”, International SWAT4LS Workshop, Berlin, Germany, 1-6, 9-11 December, 2014.
H. Chuanping, et al., “Semantic Link Network-Based Model for Organizing Multimedia Big Data”, IEEE Transactions on Emerging Topics in Computing, 2(3), 376-387, 2014.
S. Alcic, S. Conrad, “2-DOM: A 2-Dimensional Object Model towards Web Image Annotation”, Third International Workshop on Semantic Media Adaptation and Personalization, Prague, Czech Republic, 3-8, 15-16 December, 2008.
L. Nixon, R. Troncy, “Survey of Semantic Media Annotation Tools for the Web: Towards New Media Applications with Linked Media”, European Semantic Web Conference, Anissaras, Crete, 110-114, 16 October, 2014.
Internet: R. Kiros, R. Salakhutdinov, R. S. Zemel, Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models, https://arxiv.org/abs/1411.2539, 17.05.2019.
M. Bakalem, N. Benblidia, S. Ait-Aoudia, “A comparative image auto-annotation”, IEEE International Symposium on Signal Processing and Information Technology, Athens, Greece, 000086-000091, 12-15 December, 2013.
D. Zhao, W. Zou, G. Sun, “A fast image classification algorithm using Support Vector Machine”, 2nd International Conference on Computer Technology and Development, Cairo, Egypt, 385-388, 2-4 November, 2010.
Irfanullah, et al., “A framework for high level semantic annotation using trusted object annotated dataset”, The 10th IEEE International Symposium on Signal Processing and Information Technology, Luxor, Egypt, 491-495, 15-18 December, 2010.
J. M. Almendros-Jiménez, L. Domene, J. A. Piedra-Fernández, “A Framework for Ocean Satellite Image Classification Based on Ontologies”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(2), 1048-1063, 2013.
X. Li, et al., “A Locality Sensitive Low-Rank Model for Image Tag Completion”, IEEE Transactions on Multimedia, 18(3), 474-483, 2016.
D. A. Moreira, et al., “3D Markup of Radiological Images in ePAD, a Web-Based Image Annotation Tool”, IEEE 28th International Symposium on Computer-Based Medical Systems, Sao Carlos, Brazil, 97-102, 22-25 June, 2015.
N. T. Lingutla, et al., “AISO: Annotation of Image Segments with Ontologies”, Journal of Biomedical Semantics, 5(50), 2014.
S. R. Kakade, N. R. Kakade, “A novel approach to link semantic gap between images and tags via probabilistic ranking”, IEEE International Conference on Computational Intelligence and Computing Research, Enathi, India, 1-6, 26-28 December, 2013.
C. Wang, et al., “A novel image tag saliency ranking algorithm based on sparse representation”, Visual Communications and Image Processing, Kuching, Malaysia, 1-5, 17-20 November, 2013.
H-K. Hong, K-W. Park, D-H. Lee, “A Novel Semantic Tagging Technique Exploiting Wikipedia-Based Associated Words”, IEEE 39th Annual Computer Software and Applications Conference, Taichung, Taiwan, Vol. 3, 648-649, 1-5 July, 2015.
J. Xu, S. Okada, K. Nitta, “A Semantic-Similarity-Based Method for Object Description and Clustering”, IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK, 3669-3674, 13-16 October, 2013.
G. K. Kyriazos, et al., “A semantically-aided approach for online annotation and retrieval of medical images”, International Conference of the IEEE Engineering in Medicine and Biology, Boston, USA, 2372-2375, 30 August-3 September, 2011.
P. Symeonidis, A. Nanopoulos, Y. Manolopoulos, “A Unified Framework for Providing Recommendations in Social Tagging Systems Based on Ternary Semantic Analysis”, IEEE Transactions on Knowledge and Data Engineering, 22(2), 179-192, 2010.
J-Y. Shih, I. Huang, S-W. Chan, “A Web-Based Annotation System to Conducting Learning Activities for Mammography”, International Conference on Educational Innovation through Technology, Tainan, Taiwan, 218-221, 22-24 September, 2016.
M. C. You, et al., “A web-based semi-automated method for semantic annotation of high schools in remote sensing images”, Third International Conference on Agro-Geoinformatics, Beijing, China, 1-4, 11-14 August, 2014.
H. Zarzour, M. Sellami, “Achieving consistency in collaborative image annotation systems”, 5th International Conference on Information and Communication Systems, Irbid, Jordan, 1-7, 1-3 April, 2014.
M. Shi, H. Cai, L. Jiang, “An approach to semi-automatic semantic annotation on Web3D scenes based on an ontology framework”, 12th International Conference on Intelligent Systems Design and Applications, Kochi, India, 574-579, 27-29 November, 2012.
Z. Kastrati, A. S. Imran, S. Y. Yayilgan, “An Improved Concept Vector Space Model for Ontology Based Classification”, 11th International Conference on Signal-Image Technology & Internet-Based Systems, Bangkok, Thailand, 240-245, 23-27 November 2015.
Q. Wu, P. Boulanger, “An Unified Image Tagging System Driven by Image-Click-Ads Framework”, IEEE International Symposium on Multimedia, Miami, USA, 369-372, 14-16 December, 2015.
X-J. Wang, et al., “ARISTA - image search to annotation on billions of web photos”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, USA, 2987-2994, 13-18 June, 2010.
A. S. Reddy, et al., “Automatic caption generation for annotated images by using clustering algorithm”, International Conference on Innovations in Information, Embedded and Communication Systems, Coimbatore, India, 1-5, 19-20 March, 2015.
Y. Feng, M. Lapata, “Automatic Caption Generation for News Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(4), 797-812, 2013.
E. F. Luque, D. L. Rubin, D. A. Moreira, “Automatic Classification of Cancer Tumors Using Image Annotations and Ontologies”, IEEE 28th International Symposium on Computer-Based Medical Systems, Sao Carlos, Brazil, 368-369, 22-25 June, 2015.
M. Zachariasova, et al., “Automatic extraction of non-textual information in web document and their classification”, 35th International Conference on Telecommunications and Signal Processing, Prague, Czech Republic, 753-757, 3-4 July, 2012.
F. Tian, et al., “Automatic Image Annotation Based on Sparse Representation and Multiple Label Learning”, International Conference on Virtual Reality and Visualization, Qinhuangdao, China, 27-32, 14-15 September, 2012.
G. Chu, K. Niu, B. Tian, “Automatic image annotation combining SVMS and KNN algorithm”, IEEE 3rd International Conference on Cloud Computing and Intelligence Systems, Shenzhen, China, 13-17, 27-29 November, 2014.
Y-F. Huang, H-Y. Lu, “Automatic Image Annotation Using Multi-object Identification”, Fourth Pacific-Rim Symposium on Image and Video Technology, Singapore, Singapore, 386-392, 14-17 November, 2010.
G. Ding, N. Xu, “Automatic semantic annotation of images based on Web data”, Sixth International Conference on Information Assurance and Security, Atlanta, USA, 317-322, 23-25 August, 2010.
J. Söderberg, E. Kakogianni, “Automatic tag generation for photos using contextual information and description logics”, International Workshop on Content Based Multimedia Indexing, Grenoble, France, 1-7, 23-25 June, 2010.
Y-Y. Chen, W. H. Hsu, H-Y. M. Liao, “Automatic Training Image Acquisition and Effective Feature Selection From Community-Contributed Photos for Facial Attribute Detection”, IEEE Transactions on Multimedia, 15(6), 1388-1399, 2013.
A. A. Olaode, G. Naghdy, C. A. Todd, “Bag-of-Visual Words Codebook Development for the Semantic Content Based Annotation of Images”, 11th International Conference on Signal-Image Technology & Internet-Based Systems, Bangkok, Thailand, 7-14, 23-27 November, 2015.
H. Ma, et al. “Bridging the Semantic Gap Between Image Contents and Tags”, IEEE Transactions on Multimedia, 12(5), 462-473, 2010.
C. Hongjun, Z. Junfeng, “Classification of image based on semantic features and Bayesian networks”, International Conference on Consumer Electronics, Communications and Networks, XianNing, China, 4858-4861, 16-18 April, 2011.
J. Jing, et al., “Cognition-Based Semantic Annotation for Web Images”, Fourth International Conference on Big Data and Cloud Computing, Sydney, Australia, 540-546, 3-5 December, 2014.
M. Oujaoura, B. Minaoui, M. Fakir, “Combined descriptors and classifiers for automatic image annotation”, International Conference on Multimedia Computing and Systems, Marrakech, Morocco, 270-276, 14-16 April, 2014.
Y. Gong, S. Lazebnik, “Comparing data-dependent and data-independent embeddings for classification and ranking of internet images”, Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, 2633-2640, 20-25 June, 2011.
Y. Lu, et al., “Constructing Concept Lexica With Small Semantic Gaps”, IEEE Transactions on Multimedia, 12(4), 288-299, 2010.
V. Franzoni, et al., “Context-based image semantic similarity”, 12th International Conference on Fuzzy Systems and Knowledge Discovery, Zhangjiajie, China, 1280-1284, 15-17 August, 2015.
M. Shiyamala, G. Kalaiarasi, “Contextual image search with keyword and image input”, International Conference on Information Communication and Embedded Systems, Chennai, India, 1-5, 27-28 February, 2014.
D. T. Parry, T-C. Tsai, “Crowdsourcing techniques to create a fuzzy subset of SNOMED CT for semantic tagging of medical documents”, Soft Computing, 16(7), 1119-1127, 2012.
Z. Ji, “Decoupling Sparse Coding with Fusion of Fisher Vectors and Scalable SVMs for Large-Scale Visual Recognition”, IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, 450-457, 23-28 June, 2013.
M. Volpi, D. Tuia, “Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks”, IEEE Transactions on Geoscience and Remote Sensing, 55(2), 881-893, 2017.
N. C. Kuicheu, et al., “Description logic based icons semantics: An Ontology for Icons”, IEEE 11th International Conference on Signal Processing, Beijing, China, 1260-1263, 21-25 October, 2012.
S. P. Malewar, H. Dakhore, “Design approach of visual image detection in rescue robot system of urban search using Bayesian's logical algorithm”, International Conference on Innovations in Information, Embedded and Communication Systems, Coimbatore, India, 1-5, 19-20 March, 2015.
S. Zhuo, et al., “Distributed File System and Classification for Small Images”, IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, Beijing, China, 2231-2234, 20-23 August, 2013.
X-J. Wang, L. Zhang, W-Y. Ma, “Duplicate-Search-Based Image Annotation Using Web-Scale Data”, Proceedings of the IEEE, 100(9), 2705-2721, 2012.
D. K. Iakovidis, C. V. Smailis, “Efficient semantically-aware annotation of images”, IEEE International Conference on Imaging Systems and Techniques, Penang, Malaysia, 146-149, 17-18 May, 2011.
L. Dai, et al., “Efficient Tag Mining via Mixture Modeling for Real-Time Search-Based Image Annotation”, IEEE International Conference on Multimedia and Expo, Melbourne, Australia, 134-139, 9-13 July, 2012.
Z. Akata, et al., “Evaluation of Output Embeddings for Fine-Grained Image Classification”, IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2927-2936, 7-12 June, 2015.
M. Morchid, R. Dufour, G. Linares, “Event detection from image hosting services by slightly-supervised multi-span context models”, 11th International Workshop on Content-Based Multimedia Indexing, Veszprem, Hungary, 103-107, 17-19 June, 2013.
M. Katsurai, M. Haseyama, “Exploring and visualizing tag relationships in photo sharing websites based on distributional representations”, IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, Canada, 3617-3621, 26-31 May, 2013.
V. Garousi, “Classification and trend analysis of UML books (1997-2009)”, Software & System Modeling (SoSyM), 11(2), 273-285, 2012.
R. Gaur, D. K. Sharma, “Focused crawling with ontology using semi-automatic tagging for relevancy”, Seventh International Conference on Contemporary Computing, Noida, India, 501-506, 7-9 August, 2014.
O. Inel, et al., “CrowdTruth: Machine-Human Computation Framework for Harnessing Disagreement in Gathering Annotated Data”, International Semantic Web Conference, Riva Del Garda, Italy, 486-504, 19-23 October, 2014.
V. de Boer, et al., “DIVE into the event-based browsing of linked historical media”, Journal of Web Semantics, 35(3), 152-158, 2015.
L. Wang, L. Khan, “Automatic image annotation and retrieval using weighted feature selection”, Multimedia Tools and Applications, 29(1), 55-71, 2006.
Y. Chabane, et al., “Medical data management in the SYSEO project”, ACM SIGMOD Record, 42(3), 48-53, 2013.
R. Fujimoto, et al., “Generation of Stratified Image Database with Web Image Sharing Service and Ontology”, 3rd International Conference on Advanced Applied Informatics, Kitakyushu, Japan, 966-971, 31 August-4 September, 2014.
T. Helmy, “A Generic Framework for Semantic Annotation of Images”, International Journal of Image and Graphics, 18(03), 1850013, 2018.
H. Gu, et al., “An Object-Based Semantic Classification Method for High Resolution Remote Sensing Imagery Using Ontology”, Remote Sensing, 9(4), 329, 2017.
D. Goel, et al., “An ontology-driven context aware framework for smart traffic monitoring”, IEEE Region 10 Symposium, Cochin, India, 1-5, 14-16 July, 2017.
J. Yu, Y. Rui, D. Tao, “Click Prediction for Web Image Reranking Using Multimodal Sparse Coding”, IEEE Transactions On Image Processing, 23(5), 2019-2032., 2014.
J. Wang, et al., “Image Semantic Segmentation Algorithm Based on Self-learning Super-Pixel Feature Extraction”, International Conference on Emerging Internetworking, Data & Web Technologies, Tirana, Albania, 773-781, 15-17 March, 2018.
S. Baier, Y. Ma, V. Tresp, “Improving Visual Relationship Detection Using Semantic Modeling of Scene Descriptions”, International Semantic Web Conference, Vienna, Austria, 53-68, 21-25 October, 2017.
C. Huang, et al., “Large-scale semantic web image retrieval using bimodal deep learning techniques”, Information Sciences, 430(2018), 331-348, 2018.
M. Hu, et al., “Robust Web Image Annotation via Exploring Multi-Facet and Structural Knowledge”, IEEE Transactions on Image Processing, 26(10), 4871-4884, 2017.
X. Wang, et al., “Semantic annotation for complex video street views based on 2D–3D multi-feature fusion and aggregated boosting decision forests”, Pattern Recognition, 62(2017), 189-201, 2017.
P. Phursutkar, K. Wanjale, “Social re-ranking of image based on visual and semantic information”, 8th International Conference on Computing, Communication and Networking Technologies, Delhi, India, 1-6, 3-5 July, 2017.
Y. Wei, et al., “STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(11), 2314-2320, 2017.
P. Wang, et al., “Training-free indexing refinement for visual media via multi-semantics”, Neurocomputing, 236(2017), 39-47, 2017.
L. F. Sikos, “RDF-powered semantic video annotation tools with concept mapping to Linked Data for next-generation video indexing: a comprehensive review”, Multimedia Tools and Applications, 76(12), 14437-14460, 2017.
R. Krishna, et al., “Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations”, International Journal of Computer Vision, 123(1), 32-73, 2017.
Internet: P-M. Cheung, et al., Image annotation using aggregated page information from active and inactive indices, https://patents.google.com/patent/US9697236B2/en, 29.08.2019.
L. Zheng, Z. Caiming, C. Caixian, “MMDF-LDA: An improved Multi-Modal Latent Dirichlet Allocation model for social image annotation”, Expert Systems with Applications, 104(2018), 168-184, 2018.
A. Karpathy, L. Fei-Fei, “Deep Visual-Semantic Alignments for Generating Image Descriptions”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 664-676, 2017.
R. Socher, et al., “Grounded Compositional Semantics for Finding and Describing Images with Sentences”, Transactions of the Association for Computational Linguistics, 2(2014), 207-218, 2014.
G. Yunchao, et al., “A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics”, International Journal of Computer Vision, 106(2), 210-233, 2014.

Görüntülerden Bilgi Elde Etmek İçin Görüntü Açıklama Çalışmalarının Sistematik Haritalandırma İncelemesi

Year 2020, Volume: 13 Issue: 4, 423 - 434, 30.10.2020

Arda Sezen , Çigdem Turhan

Abstract

Çeşitli görüntü türlerinden görüntü meta verilerini elde etmeyi, işlemeyi ve anlamlı sonuçlar çıkarmayı amaçlayan görüntü açıklaması kavramı, bilgi toplumunda görüntülerden bilgi edinme konusunda daha kritik hale gelmiştir. Bu konuda literatürde önerilen birçok yaklaşım vardır, ancak en uygun olanı seçmek kolay değildir. Bu çalışmada, görüntü açıklamalarındaki kilit yaklaşımlar, kapsamlı bir literatür taraması ve sistematik haritalama yoluyla incelenmiştir. Çalışmamızın yeniliği, literatürdeki görüntü açıklama çalışmalarını araştırmak, haritalamak ve analiz etmek ve seçilen çalışmaları sınıflandırmak, araştırma boşluklarını ortaya çıkarmak ve bu makalede sunulan araştırma soruları için görsel özet hazırlamak adına ilk denemeyi temsil etmesidir. Literatürde, ilgi alanını farklı açılardan araştırmak için sistematik haritalama yaklaşımı önerilmektedir. Bu amaçla, tespit edilen toplam 404 çalışma içerisinden 95 çalışma seçilmiştir. Alandaki literatürün incelenmesi, mevcut yöntemlerin / tekniklerin, araçların, metriklerin, süreçlerin veya diğer teknik yaklaşımların kendi başlarına eksiksiz bir çözüm üretmek için yeterli olmadığını göstermektedir. Önerilen teknikleri birleştirerek ve disiplinlerarası yaklaşımları dikkate alarak yeni çözümler üretmenin gerekliliği de önerilmiştir.

Keywords

anlamsal görüntü açıklaması, görüntü işleme, resim yorumlama

References

V. Lavrenko, R. Manmatha, J. Jeon, “A Model for Learning the Semantics of Pictures”, Advances in Neural Information Processing Systems 17, Vancouver, Canada, 553-560, 13-18 December, 2004.
Internet: Halaschek, C. Wiener, Image Annotation on the Semantic Web, https://www.w3.org/2005/Incubator/mmsem/XGR-image-annotation/#annot_intro, 03.05.2019.
B. Kitchenham, S. Charters, Guidelines for Performing Systematic Literature Reviews in Software Engineering, Keele University, UK, 2007.
B. Kitchenham, et al., “Systematic literature reviews in software engineering – A systematic literature review”, Information and Software Technology, 51(1), 7-15, 2009.
K. Petersen, et al., “Systematic Mapping Studies in Software Engineering”, 12th International Conference on Evaluation and Assessment in Software Engineering, Bari, Italy, 68-77, 26-27 June, 2008.
J. Jill, L. Matheson, F. M. Lacey, Doing Your Literature Review: traditional and systematic techniques, SAGE Publishing, London, UK, 2011.
Internet: A. Sezen, Ç. Turhan, SM – Classification Scheme, https://drive.google.com/open?id=1cnItNure_P7oolIcoVmDqP5EXJMim00-HPOweH_qPDA, 10.10.2019.
X. Wang, et al., “High-level semantic image annotation based on hot Internet topics”, Multimedia Tools and Applications, 74(6), 2055-2084, 2015.
H. Bannour, C. Hudelot, “Building and using fuzzy multimedia ontologies for semantic image annotation”, Multimedia Tools and Applications, 72(3), 2107-2141, 2014.
A. Fakhari, A. M. E. Moghadam, “Combination of classification and regression in decision tree for multi-labeling image annotation and retrieval”, Applied Soft Computing, 13(2), 1292-1302, 2013.
F. Kınlı, F. Kıraç, “FashionCapsNet: Clothing Classification with Capsule Networks”, International Journal of Informatics Technologies, 13(1), 87-96, 2020.
Internet: Y. Gong, et al., Deep Convolutional Ranking for Multilabel Image Annotation, https://arxiv.org/abs/1312.4894, 03.05.2019.
R. Hong, et al., “Image Annotation by Multiple-Instance Learning With Discriminative Feature Mapping and Selection”, IEEE Transactions on Cybernetics, 44(5), 669-680, 2014.
S. Zhang, et al., “ObjectPatchNet: Towards scalable and semantic image annotation and retrieval”, Computer Vision and Image Understanding, 118(1), 16-29, 2014.
D-H. Im, G-D. Park, “STAG: Semantic Image Annotation Using Relationships between Tags”, International Conference on Information Science and Applications, Suwon, South Korea, 1-2, 24-26 June, 2013.
L. Seneviratne, E. Izquierdo, “A mathematical approach towards semi-automatic image annotation”, 19th European Signal Processing Conference, Barcelona, Spain, 559-563, 29 August-2 September, 2011.
A. Ç. Seçkin, “Labeling System with EEG, EMG, and IMU for Visual Training of Autonomous Vehicles”, International Journal of Informatics Technologies, 12(4), 299-305, 2019.
S-B. Chan, H. Yamana, S. Satoh, “A Negative Sample Image Selection Method Referring to Semantic Hierarchical Structure for Image Annotation”, International Conference on Signal-Image Technology & Internet-Based Systems, Kyoto, Japan, 162-167, 2-5 December, 2013.
X. Liu, H. Yao, R. Ji, “Exploring statistical properties for semantic annotation: sparse distributed and convergent assumptions for keywords”, IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, USA, 802-805, 14-19 March, 2010.
Z. Mehmood, T. Mahmood, M. A. Javid, “Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine”, Applied Intelligence, 48(1), 166-181, 2018.
F. Bulut, “Performance Analysis of Ensemble Methods on Imbalanced Datasets”, International Journal of Informatics Technologies, 9(2), 153-159, 2016.
S. Ali, et al., “A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation”, IEEE Transactions on Software Engineering, 36(6), 742-762, 2010.
F. Paz, J. A. Pow-Sang, “Systematic Mapping Review of Usability Evaluation Methods for Software Development Process”. International Journal of Software Engineering and Its Applications, 10(1), 165-178, 2016.
V. R. Basili, Software Modeling and Measurement: The Goal/Question/Metric Paradigm, University of Maryland, USA, 1992.
F. A. Silva, et al., “Benchmark applications used in mobile cloud computing research: a systematic mapping study”, The Journal of Supercomputing, 72(4), 1431-1452, 2016.
Internet: Y. Gong, et al., Deep Convolutional Ranking for Multilabel Image Annotation, https://arxiv.org/abs/1312.4894, 03.05.2019.
I. Banerjee, et al., “Graphical user interface (GUI) testing: Systematic mapping and repository”, Information and Software Technology, 55(10), 1679-1694, 2013.
N. Bencomo, S. Hallsteinsen, E. S. de Almeida, “A View of the Dynamic Software Product Line Landscape”, IEEE Computer, 45(10), 36-41, 2012.
S. Easterbrook, et al., "Selecting Empirical Methods for Software Engineering Research", Guide to Advanced Empirical Software Engineering, Editor: F. Shull, J. Singer, D. I. K. Sjøberg, Springer, London, UK, 285-311, 2008.
T. Kanimozhi, T., A. Christy, “Incorporating ontology and SPARQL for semantic image annotation”, IEEE Conference on Information & Communication Technologies, Thuckalay, Tamil Nadu, India, 26-31, 11-12 April, 2013.
D-H. Im, G-D Park, “Linked tag: image annotation using semantic relationships between image tags”, Multimedia Tools and Applications, 74(7), 2273-2287, 2015.
D. Wang, et al., “Mining Weakly Labeled Web Facial Images for Search-Based Face Annotation”, IEEE Transactions on Knowledge and Data Engineering, 26(1), 166-179, 2014.
T. Sjekavica, G. Gledec, M. Horvat, “Multimedia Annotation using Semantic Web Technologies”, 7th European Computing Conference, Dubrovnik, Croatia, 228-233, 25-27 June, 2013.
Internet: G. Ciocca, P. Napoletano, R. Schettini, IAT - Image Annotation Tool: Manual, https://arxiv.org/abs/1502.05212, 17.05.2019.
C. Kurtz, et al., “On combining image-based and ontological semantic dissimilarities for medical image retrieval applications”, Medical Image Analysis, 18(7), 1082-1100, 2014.
K. C. Wang, et al., “Ontology-based Image Navigation: Exploring 3.0-T MR Neurography of the Brachial Plexus Using AIM and RadLex”, RadioGraphics, 35(1), 142-151, 2015.
M. Grassi, et al., “Pundit: Creating, Exploring and Consuming Semantic Annotations”, 3rd International Workshop on Semantic Digital Archives, Valetta, Malta, 65-72, 26 September, 2013.
D. K. Iakovidis, et al., “Ratsnake: A Versatile Image Annotation Tool with Application to Computer-Aided Diagnosis”, The Scientific World Journal, 2014(286856), 1-12, 2014.
A. C. Bukhari, M. Krauthammer, C. J. O. Baker, “SEBI: An Architecture for Biomedical Image Discovery, Interoperability and Reusability based on Semantic Enrichment”, International SWAT4LS Workshop, Berlin, Germany, 1-6, 9-11 December, 2014.
H. Chuanping, et al., “Semantic Link Network-Based Model for Organizing Multimedia Big Data”, IEEE Transactions on Emerging Topics in Computing, 2(3), 376-387, 2014.
S. Alcic, S. Conrad, “2-DOM: A 2-Dimensional Object Model towards Web Image Annotation”, Third International Workshop on Semantic Media Adaptation and Personalization, Prague, Czech Republic, 3-8, 15-16 December, 2008.
L. Nixon, R. Troncy, “Survey of Semantic Media Annotation Tools for the Web: Towards New Media Applications with Linked Media”, European Semantic Web Conference, Anissaras, Crete, 110-114, 16 October, 2014.
Internet: R. Kiros, R. Salakhutdinov, R. S. Zemel, Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models, https://arxiv.org/abs/1411.2539, 17.05.2019.
M. Bakalem, N. Benblidia, S. Ait-Aoudia, “A comparative image auto-annotation”, IEEE International Symposium on Signal Processing and Information Technology, Athens, Greece, 000086-000091, 12-15 December, 2013.
D. Zhao, W. Zou, G. Sun, “A fast image classification algorithm using Support Vector Machine”, 2nd International Conference on Computer Technology and Development, Cairo, Egypt, 385-388, 2-4 November, 2010.
Irfanullah, et al., “A framework for high level semantic annotation using trusted object annotated dataset”, The 10th IEEE International Symposium on Signal Processing and Information Technology, Luxor, Egypt, 491-495, 15-18 December, 2010.
J. M. Almendros-Jiménez, L. Domene, J. A. Piedra-Fernández, “A Framework for Ocean Satellite Image Classification Based on Ontologies”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(2), 1048-1063, 2013.
X. Li, et al., “A Locality Sensitive Low-Rank Model for Image Tag Completion”, IEEE Transactions on Multimedia, 18(3), 474-483, 2016.
D. A. Moreira, et al., “3D Markup of Radiological Images in ePAD, a Web-Based Image Annotation Tool”, IEEE 28th International Symposium on Computer-Based Medical Systems, Sao Carlos, Brazil, 97-102, 22-25 June, 2015.
N. T. Lingutla, et al., “AISO: Annotation of Image Segments with Ontologies”, Journal of Biomedical Semantics, 5(50), 2014.
S. R. Kakade, N. R. Kakade, “A novel approach to link semantic gap between images and tags via probabilistic ranking”, IEEE International Conference on Computational Intelligence and Computing Research, Enathi, India, 1-6, 26-28 December, 2013.
C. Wang, et al., “A novel image tag saliency ranking algorithm based on sparse representation”, Visual Communications and Image Processing, Kuching, Malaysia, 1-5, 17-20 November, 2013.
H-K. Hong, K-W. Park, D-H. Lee, “A Novel Semantic Tagging Technique Exploiting Wikipedia-Based Associated Words”, IEEE 39th Annual Computer Software and Applications Conference, Taichung, Taiwan, Vol. 3, 648-649, 1-5 July, 2015.
J. Xu, S. Okada, K. Nitta, “A Semantic-Similarity-Based Method for Object Description and Clustering”, IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK, 3669-3674, 13-16 October, 2013.
G. K. Kyriazos, et al., “A semantically-aided approach for online annotation and retrieval of medical images”, International Conference of the IEEE Engineering in Medicine and Biology, Boston, USA, 2372-2375, 30 August-3 September, 2011.
P. Symeonidis, A. Nanopoulos, Y. Manolopoulos, “A Unified Framework for Providing Recommendations in Social Tagging Systems Based on Ternary Semantic Analysis”, IEEE Transactions on Knowledge and Data Engineering, 22(2), 179-192, 2010.
J-Y. Shih, I. Huang, S-W. Chan, “A Web-Based Annotation System to Conducting Learning Activities for Mammography”, International Conference on Educational Innovation through Technology, Tainan, Taiwan, 218-221, 22-24 September, 2016.
M. C. You, et al., “A web-based semi-automated method for semantic annotation of high schools in remote sensing images”, Third International Conference on Agro-Geoinformatics, Beijing, China, 1-4, 11-14 August, 2014.
H. Zarzour, M. Sellami, “Achieving consistency in collaborative image annotation systems”, 5th International Conference on Information and Communication Systems, Irbid, Jordan, 1-7, 1-3 April, 2014.
M. Shi, H. Cai, L. Jiang, “An approach to semi-automatic semantic annotation on Web3D scenes based on an ontology framework”, 12th International Conference on Intelligent Systems Design and Applications, Kochi, India, 574-579, 27-29 November, 2012.
Z. Kastrati, A. S. Imran, S. Y. Yayilgan, “An Improved Concept Vector Space Model for Ontology Based Classification”, 11th International Conference on Signal-Image Technology & Internet-Based Systems, Bangkok, Thailand, 240-245, 23-27 November 2015.
Q. Wu, P. Boulanger, “An Unified Image Tagging System Driven by Image-Click-Ads Framework”, IEEE International Symposium on Multimedia, Miami, USA, 369-372, 14-16 December, 2015.
X-J. Wang, et al., “ARISTA - image search to annotation on billions of web photos”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, USA, 2987-2994, 13-18 June, 2010.
A. S. Reddy, et al., “Automatic caption generation for annotated images by using clustering algorithm”, International Conference on Innovations in Information, Embedded and Communication Systems, Coimbatore, India, 1-5, 19-20 March, 2015.
Y. Feng, M. Lapata, “Automatic Caption Generation for News Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(4), 797-812, 2013.
E. F. Luque, D. L. Rubin, D. A. Moreira, “Automatic Classification of Cancer Tumors Using Image Annotations and Ontologies”, IEEE 28th International Symposium on Computer-Based Medical Systems, Sao Carlos, Brazil, 368-369, 22-25 June, 2015.
M. Zachariasova, et al., “Automatic extraction of non-textual information in web document and their classification”, 35th International Conference on Telecommunications and Signal Processing, Prague, Czech Republic, 753-757, 3-4 July, 2012.
F. Tian, et al., “Automatic Image Annotation Based on Sparse Representation and Multiple Label Learning”, International Conference on Virtual Reality and Visualization, Qinhuangdao, China, 27-32, 14-15 September, 2012.
G. Chu, K. Niu, B. Tian, “Automatic image annotation combining SVMS and KNN algorithm”, IEEE 3rd International Conference on Cloud Computing and Intelligence Systems, Shenzhen, China, 13-17, 27-29 November, 2014.
Y-F. Huang, H-Y. Lu, “Automatic Image Annotation Using Multi-object Identification”, Fourth Pacific-Rim Symposium on Image and Video Technology, Singapore, Singapore, 386-392, 14-17 November, 2010.
G. Ding, N. Xu, “Automatic semantic annotation of images based on Web data”, Sixth International Conference on Information Assurance and Security, Atlanta, USA, 317-322, 23-25 August, 2010.
J. Söderberg, E. Kakogianni, “Automatic tag generation for photos using contextual information and description logics”, International Workshop on Content Based Multimedia Indexing, Grenoble, France, 1-7, 23-25 June, 2010.
Y-Y. Chen, W. H. Hsu, H-Y. M. Liao, “Automatic Training Image Acquisition and Effective Feature Selection From Community-Contributed Photos for Facial Attribute Detection”, IEEE Transactions on Multimedia, 15(6), 1388-1399, 2013.
A. A. Olaode, G. Naghdy, C. A. Todd, “Bag-of-Visual Words Codebook Development for the Semantic Content Based Annotation of Images”, 11th International Conference on Signal-Image Technology & Internet-Based Systems, Bangkok, Thailand, 7-14, 23-27 November, 2015.
H. Ma, et al. “Bridging the Semantic Gap Between Image Contents and Tags”, IEEE Transactions on Multimedia, 12(5), 462-473, 2010.
C. Hongjun, Z. Junfeng, “Classification of image based on semantic features and Bayesian networks”, International Conference on Consumer Electronics, Communications and Networks, XianNing, China, 4858-4861, 16-18 April, 2011.
J. Jing, et al., “Cognition-Based Semantic Annotation for Web Images”, Fourth International Conference on Big Data and Cloud Computing, Sydney, Australia, 540-546, 3-5 December, 2014.
M. Oujaoura, B. Minaoui, M. Fakir, “Combined descriptors and classifiers for automatic image annotation”, International Conference on Multimedia Computing and Systems, Marrakech, Morocco, 270-276, 14-16 April, 2014.
Y. Gong, S. Lazebnik, “Comparing data-dependent and data-independent embeddings for classification and ranking of internet images”, Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, 2633-2640, 20-25 June, 2011.
Y. Lu, et al., “Constructing Concept Lexica With Small Semantic Gaps”, IEEE Transactions on Multimedia, 12(4), 288-299, 2010.
V. Franzoni, et al., “Context-based image semantic similarity”, 12th International Conference on Fuzzy Systems and Knowledge Discovery, Zhangjiajie, China, 1280-1284, 15-17 August, 2015.
M. Shiyamala, G. Kalaiarasi, “Contextual image search with keyword and image input”, International Conference on Information Communication and Embedded Systems, Chennai, India, 1-5, 27-28 February, 2014.
D. T. Parry, T-C. Tsai, “Crowdsourcing techniques to create a fuzzy subset of SNOMED CT for semantic tagging of medical documents”, Soft Computing, 16(7), 1119-1127, 2012.
Z. Ji, “Decoupling Sparse Coding with Fusion of Fisher Vectors and Scalable SVMs for Large-Scale Visual Recognition”, IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, 450-457, 23-28 June, 2013.
M. Volpi, D. Tuia, “Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks”, IEEE Transactions on Geoscience and Remote Sensing, 55(2), 881-893, 2017.
N. C. Kuicheu, et al., “Description logic based icons semantics: An Ontology for Icons”, IEEE 11th International Conference on Signal Processing, Beijing, China, 1260-1263, 21-25 October, 2012.
S. P. Malewar, H. Dakhore, “Design approach of visual image detection in rescue robot system of urban search using Bayesian's logical algorithm”, International Conference on Innovations in Information, Embedded and Communication Systems, Coimbatore, India, 1-5, 19-20 March, 2015.
S. Zhuo, et al., “Distributed File System and Classification for Small Images”, IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, Beijing, China, 2231-2234, 20-23 August, 2013.
X-J. Wang, L. Zhang, W-Y. Ma, “Duplicate-Search-Based Image Annotation Using Web-Scale Data”, Proceedings of the IEEE, 100(9), 2705-2721, 2012.
D. K. Iakovidis, C. V. Smailis, “Efficient semantically-aware annotation of images”, IEEE International Conference on Imaging Systems and Techniques, Penang, Malaysia, 146-149, 17-18 May, 2011.
L. Dai, et al., “Efficient Tag Mining via Mixture Modeling for Real-Time Search-Based Image Annotation”, IEEE International Conference on Multimedia and Expo, Melbourne, Australia, 134-139, 9-13 July, 2012.
Z. Akata, et al., “Evaluation of Output Embeddings for Fine-Grained Image Classification”, IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2927-2936, 7-12 June, 2015.
M. Morchid, R. Dufour, G. Linares, “Event detection from image hosting services by slightly-supervised multi-span context models”, 11th International Workshop on Content-Based Multimedia Indexing, Veszprem, Hungary, 103-107, 17-19 June, 2013.
M. Katsurai, M. Haseyama, “Exploring and visualizing tag relationships in photo sharing websites based on distributional representations”, IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, Canada, 3617-3621, 26-31 May, 2013.
V. Garousi, “Classification and trend analysis of UML books (1997-2009)”, Software & System Modeling (SoSyM), 11(2), 273-285, 2012.
R. Gaur, D. K. Sharma, “Focused crawling with ontology using semi-automatic tagging for relevancy”, Seventh International Conference on Contemporary Computing, Noida, India, 501-506, 7-9 August, 2014.
O. Inel, et al., “CrowdTruth: Machine-Human Computation Framework for Harnessing Disagreement in Gathering Annotated Data”, International Semantic Web Conference, Riva Del Garda, Italy, 486-504, 19-23 October, 2014.
V. de Boer, et al., “DIVE into the event-based browsing of linked historical media”, Journal of Web Semantics, 35(3), 152-158, 2015.
L. Wang, L. Khan, “Automatic image annotation and retrieval using weighted feature selection”, Multimedia Tools and Applications, 29(1), 55-71, 2006.
Y. Chabane, et al., “Medical data management in the SYSEO project”, ACM SIGMOD Record, 42(3), 48-53, 2013.
R. Fujimoto, et al., “Generation of Stratified Image Database with Web Image Sharing Service and Ontology”, 3rd International Conference on Advanced Applied Informatics, Kitakyushu, Japan, 966-971, 31 August-4 September, 2014.
T. Helmy, “A Generic Framework for Semantic Annotation of Images”, International Journal of Image and Graphics, 18(03), 1850013, 2018.
H. Gu, et al., “An Object-Based Semantic Classification Method for High Resolution Remote Sensing Imagery Using Ontology”, Remote Sensing, 9(4), 329, 2017.
D. Goel, et al., “An ontology-driven context aware framework for smart traffic monitoring”, IEEE Region 10 Symposium, Cochin, India, 1-5, 14-16 July, 2017.
J. Yu, Y. Rui, D. Tao, “Click Prediction for Web Image Reranking Using Multimodal Sparse Coding”, IEEE Transactions On Image Processing, 23(5), 2019-2032., 2014.
J. Wang, et al., “Image Semantic Segmentation Algorithm Based on Self-learning Super-Pixel Feature Extraction”, International Conference on Emerging Internetworking, Data & Web Technologies, Tirana, Albania, 773-781, 15-17 March, 2018.
S. Baier, Y. Ma, V. Tresp, “Improving Visual Relationship Detection Using Semantic Modeling of Scene Descriptions”, International Semantic Web Conference, Vienna, Austria, 53-68, 21-25 October, 2017.
C. Huang, et al., “Large-scale semantic web image retrieval using bimodal deep learning techniques”, Information Sciences, 430(2018), 331-348, 2018.
M. Hu, et al., “Robust Web Image Annotation via Exploring Multi-Facet and Structural Knowledge”, IEEE Transactions on Image Processing, 26(10), 4871-4884, 2017.
X. Wang, et al., “Semantic annotation for complex video street views based on 2D–3D multi-feature fusion and aggregated boosting decision forests”, Pattern Recognition, 62(2017), 189-201, 2017.
P. Phursutkar, K. Wanjale, “Social re-ranking of image based on visual and semantic information”, 8th International Conference on Computing, Communication and Networking Technologies, Delhi, India, 1-6, 3-5 July, 2017.
Y. Wei, et al., “STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(11), 2314-2320, 2017.
P. Wang, et al., “Training-free indexing refinement for visual media via multi-semantics”, Neurocomputing, 236(2017), 39-47, 2017.
L. F. Sikos, “RDF-powered semantic video annotation tools with concept mapping to Linked Data for next-generation video indexing: a comprehensive review”, Multimedia Tools and Applications, 76(12), 14437-14460, 2017.
R. Krishna, et al., “Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations”, International Journal of Computer Vision, 123(1), 32-73, 2017.
Internet: P-M. Cheung, et al., Image annotation using aggregated page information from active and inactive indices, https://patents.google.com/patent/US9697236B2/en, 29.08.2019.
L. Zheng, Z. Caiming, C. Caixian, “MMDF-LDA: An improved Multi-Modal Latent Dirichlet Allocation model for social image annotation”, Expert Systems with Applications, 104(2018), 168-184, 2018.
A. Karpathy, L. Fei-Fei, “Deep Visual-Semantic Alignments for Generating Image Descriptions”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 664-676, 2017.
R. Socher, et al., “Grounded Compositional Semantics for Finding and Describing Images with Sentences”, Transactions of the Association for Computational Linguistics, 2(2014), 207-218, 2014.
G. Yunchao, et al., “A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics”, International Journal of Computer Vision, 106(2), 210-233, 2014.

There are 120 citations in total.

Details

Primary Language	English
Subjects	Computer Software
Journal Section	Articles
Authors	Arda Sezen Çigdem Turhan 0000-0002-6595-7095
Publication Date	October 30, 2020
Submission Date	October 24, 2019
Published in Issue	Year 2020 Volume: 13 Issue: 4

Cite

APA	Sezen, A., & Turhan, Ç. (2020). A Systematic Mapping Review of Image Annotation Studies for Obtaining Information Retrieval from Images. Bilişim Teknolojileri Dergisi, 13(4), 423-434.

Download Cover Image

Article Files

Full Text