SNN tabanlı çok seviyeli eşikleme ile görüntü erişimi

Year 2022, IOCENS’21 Konferansı Ek Sayısı, 98 - 108, 30.09.2022

Mürsel Ozan İncetaş Mahmut Kılıçaslan Taymaz Rahkar Farshi

https://doi.org/10.17714/gumusfenbil.1002577

Abstract

Görüntü erişimi, dijital bir görüntü veri tabanından benzer veya özdeş görüntülerin indekslenmesi olarak tanımlanır. Benzer bir dijital görüntü aranırken görüntülerden elde edilen çeşitli öznitelik vektörleri kullanılır. Çünkü görüntülerin pikselleri üzerinde işlem yapmak maliyetli algoritmalar gerektirir. Ayrıca, erişim yaklaşımlarında kullanılan görüntülerin farklı boyutlarda olması olası bir problemdir. Bu nedenle, görüntüleri karşılaştırırken piksel düzeyindeki işlemler yetersiz kalmaktadır. Görüntüleri temsil eden vektörel yapılar gereklilik olarak karşımıza çıkmaktadır. Bu vektörel yapıları elde etme sürecine özellik çıkarımı denir ve içerik tabanlı görüntü erişiminin en önemli aşamalarından biridir. Histogram ise görüntünün boyutlarından bağımsız ve kolaylıkla hesaplanabilen en temel öznitelik vektörüdür. Gri seviyeli görüntülerde histogramın boyutu öznitelik vektörü olarak kullanıma uygundur. Ancak, renkli görüntülerdeki üç farklı kanal, özellik vektörleri olarak kullanılmak için çok fazla veri içerir. Bu nedenle vektör boyutunu küçültmek kaçınılmaz bir işlemdir. Bu çalışmada, insan görsel sisteminden esinlenerek İğnecikli Sinir Ağı modeline dayalı yeni bir çok-seviyeli eşikleme yöntemi önerilmiştir. Önerilen model ile RGB renk kanallarının her biri için 3 ayrı eşik değeri belirlenmiş ve her bir renk kanalı 4 parçaya bölünmüştür. Böylece elde edilen renk paleti ile renk uzayı 64 farklı renge indirgenir. Önerilen yöntem, görüntü erişimi için yaygın olarak kullanılan çok seviyeli eşikleme yöntemleri ile karşılaştırılmıştır. Elde edilen sonuçlar önerilen yöntemin başarısını açıkça göstermektedir.

Keywords

Renk niceleme, İçerik tabanlı görüntü erişimi, Çok seviyeli eşikleme, İğnecikli sinir ağı

Image retrieval with SNN-based multi-level thresholding

Abstract

Image retrieval is defined as indexing similar or identical images in a digital image database. Various feature vectors obtained from the images are used while searching for a similar digital image. However, processing all pixels of the images requires costly algorithms. In addition, it is a possible issue that the images used in retrieval approaches are of different sizes. For this reason, pixel-level operations are insufficient when comparing images. Therefore, it requires vectorial structures that represent images. The process of obtaining these vectorial structures is called feature extraction, and it is one of the most important stages of content-based image retrieval. On the other hand, the histogram is the most basic feature vector that is independent of the dimensions of the image and can be easily calculated. In gray-level images, the size of the histogram is suitable for use as a feature vector. However, three different channels in color images contain too much data to be used as feature vectors. The data of 3 separate histograms are reduced using various thresholding processes and feature vectors are extracted. Therefore, reducing the vector size is an inevitable operation. In this study, a new multi-thresholding method based on the Spiking Neural Network model, inspired by the human visual system, is proposed. With the proposed model, 3 threshold values are determined for each of the RGB color channels, and each color channel is divided into 4 parts. Thus, the color palette of the image is quantized to 64 different colors and a feature vector with 64 elements is obtained. The proposed method was compared with the commonly used multilevel thresholding methods. The results obtained showed that the proposed method is quite successful.

Keywords

Color quantization, Content-based image retrieval, Multilevel thresholding, Spiking neural network

References

• Alamdar, F. & Keyvanpour, M. (2011). A new color feature extraction method based on QuadHistogram. Procedia Environmental Sciences, 10, 777-783. https://doi.org/10.1016/j.proenv.2011.09.126
• Barber, R., Flickner, M., Hafner, J., Niblack, W., Petkovic, D., Equitz, W. & Faloutsos, C. (1994). Efficient and effective querying by image content. Journal of Intelligent Information Systems, 3(3-4), 231-262. https://doi.org/10.1007/BF00962238
• Cambronero, J., Stanley-Marbell, P. & Rinard, M. (2018). Incremental color quantization for color-vision-deficient observers using mobile gaming data. arXiv preprint arXiv:1803.08420. https://doi.org/10.48550/arXiv.1803.08420
• Clogenson, M., Kerr, D., McGinnity, T. M., Coleman, S. A. & Wu, Q. (2011). Biologically inspired edge detection using spiking neural networks and hexagonal images. In International Conference on Neural Computation Theory and Applications (pp. 381-384). SciTePress. https://doi.org/ 10.5220/0003682103810384
• Chen, Y. H., Chang, C. C., & Hsu, C. Y. (2020). Content-based image retrieval using block truncation coding based on edge quantization. Connection Science, 32(4), 431-448. https://doi.org/10.1080/09540091.2020.1753174
• Deselaers, T. (2003). Features for image retrieval. Master's thesis. University of Rhine-Westphalia Alsiche Technical University of Aachen.
• Demirci, R., & Ümit, O. (2019). Renkli Görüntülerin Ortalama Tabanlı Çok Seviyeli Eşiklenmesi. Düzce Üniversitesi Bilim ve Teknoloji Dergisi,7(1), 664-676. https://doi.org/10.29130/dubited.471040
• Devaraj, A. F. S., Murugaboopathi, G., Elhoseny, M., Shankar, K., Min, K., Moon, H. & Joshi, G. P. (2020). An Efficient Framework for Secure Image Archival and Retrieval System Using Multiple Secret Share Creation Scheme. IEEE Access, 8, 144310-144320. https://doi.org/10.1109/ACCESS.2020.3014346
• FitzHugh, R. (1969). Mathematical models of excitation and propagation in nerve. Biological engineering, 1-85.
• Gerstner, W. & Kistler, W. M. (2002). Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press.
• Gervautz, M. & Purgathofer, W. (1988). A simple method for color quantization: Octree quantization. In New trends in computer graphics (pp. 219-231). Springer, Berlin, Heidelberg.
• Ghosh-Dastidar, S. & Adeli, H. (2009). Spiking neural networks. International journal of neural systems, 19(04), 295-308. https://doi.org/10.1142/S0129065709002002
• Gupta, A. & Jain, R. (1997). Visual information retrieval. Communications of the ACM, 40(5), 70-79. https://doi.org/10.1145/253769.253798
• Heckbert, P. (1982). Color Image Quantization for Frame Buffer Display. Computer Graphics, 16(2):297-307. https://doi.org/10.1145/965145.801294
• Hildreth, E. C. (1983). The detection of intensity changes by computer and biological vision systems. Computer Vision, Graphics and Image Processing, 22(1), 1-27. https://doi.org/10.1016/0734-189X(83)90093-2
• Hodgkin, A. L. & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of physiology, 117(4), 500-544.
• Huang, C. Li X. & Wen, Y. (2021). AN OTSU image segmentation based on fruitfly optimization algorithm. Alexandria Engineering Journal, 60(1), 183-188. https://doi.org/10.1016/j.aej.2020.06.054
• Islam, S.M., Joardar, S., Dogra, D.P. (2021) Ornament Image Retrieval Using Multimodal Fusion. SN COMPUT. SCI.2,336. https://doi.org/10.1007/s42979-021-00734-1
• Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on neural networks, 14(6), 1569-1572. https://doi.org/10.1109/TNN.2003.820440
• Kayhan, N., & Fekri-Ershad, S. (2021). Content based image retrieval based on weighted fusion of texture and color features derived from modified local binary patterns and local neighborhood difference patterns. Multimedia Tools and Applications, 80(21), 32763-32790. https://doi.org/10.1007/s11042-021-11217-z
• Kerr, D., Coleman, S., McGinnity, M., Wu, Q. & Clogenson, M. (2011a, November). Biologically inspired edge detection. In 2011 11th International Conference on Intelligent Systems Design and Applications (pp. 802-807). IEEE.
• Kerr, D., McGinnity, M., Coleman, S., Wu, Q. & Clogenson, M., (2011b, January) Spiking hierarchical neural network for corner detection, NCTA 2011 - Proceedings of the International Conference on Neural Computation Theory and Applications, pp. 230-235.
• Kilicaslan, M., Tanyeri, U. & Demirci, R. (2020). Image Retrieval using One-Dimensional Color Histogram Created with Entropy. Advances in Electrical and Computer Engineering, 20(2), 79-88. https://doi.org/10.4316/AECE.2020.02010
• Kılıçaslan, M., Tanyeri, U., & Demirci, R. (2020). Tekrarlı Ortalama Yardımıyla Renk İndirgeme ve Görüntü Erişimi. Düzce Üniversitesi Bilim ve Teknoloji Dergisi,8(1), 1042-1057. https://doi.org/10.29130/dubited.643351
• Kılıçaslan, M., Tanyeri, U., & Demirci, R. (2018). Renkli Görüntüler İçin Tek Boyutlu Histogram. Düzce Üniversitesi Bilim ve Teknoloji Dergisi,6(4), 1094-1107. https://doi.org/10.29130/dubited.413822
• Konstantinidis, K., Gasteratos, A. & Andreadis, I. (2005). Image retrieval based on fuzzy color histogram processing. Optics Communications, 248(4-6), 375-386. https://doi.org/10.1016/j.optcom.2004.12.029
• Kucuktunc, O. & Zamalieva, D. (2009, March). Fuzzy color histogram-based CBIR system. In Proceedings of 1st International Fuzzy Systems Symposium.
• Kunkle, D. R. & Merrigan C, (2002). Pulsed Neural Networks and Their Application. Computer Science Dept., College of Computing and Information Sciences, Rochester Institute of Technology.
• Lai, C. C. & Chen, Y. C. (2011). A user-oriented image retrieval system based on interactive genetic algorithm. IEEE transactions on instrumentation and measurement, 60(10), 3318-3325. https://doi.org/10.1109/TIM.2011.2135010
• Linde, Y., Buzo, A. & Gray, R. (1980). An algorithm for vector quantizer design. IEEE Transactions on communications, 28(1), 84-95. https://doi.org/10.1109/TCOM.1980.1094577
• Liu, G. H. & Yang J. Y. (2013) Content-based image retrieval using color difference histogram. Pattern Recognition 46(1), 188–198. https://doi.org/10.1016/j.patcog.2012.06.001
• Liu, S., Wei, G., Song, Y. & Ding, D. (2017). Set-membership state estimation subject to uniform quantization effects and communication constraints. Journal of the Franklin Institute, 354(15), 7012-7027. https://doi.org/10.1016/j.jfranklin.2017.08.012
• Long, F., Zhang, H. & Feng, D. D. (2003). Fundamentals of content-based image retrieval. In Multimedia information retrieval and management (pp. 1-26). Springer, Berlin, Heidelberg.
• Manjunath, B. S. & Chellappa, R. (1993). A unified approach to boundary perception: edges, textures and illusory contours. IEEE Transactions on neural networks, 4(1), 96-108. https://doi.org/10.1109/72.182699
• Márquez-de-Silva, S., Felipe-Riverón, E. & Fernández, L. P. S. (2008, September). A simple and effective method of color image quantization. In Iberoamerican Congress on Pattern Recognition (pp. 749-757). Springer, Berlin, Heidelberg.
• Messing, D. S., Van Beek, P. & Errico, J. H. (2001, October). The mpeg-7 colour structure descriptor: Image description using colour and local spatial information. In Proceedings 2001 International Conference on Image Processing (Cat. No. 01CH37205) (Vol. 1, pp. 670-673). IEEE.
• Mojsilovic, A. & Rogowitz, B. (2001, October). Capturing image semantics with low-level descriptors. In Proceedings 2001 International Conference on Image Processing (Cat. No. 01CH37205) (Vol. 1, pp. 18-21). IEEE.
• Nagumo, J., Arimoto, S. & Yoshizawa, S. (1962). An active pulse transmission line simulating nerve axon. Proceedings of the IRE, 50(10), 2061-2070.
• Nelson, M. E. (2004). Electrophysiological models. Databasing the brain: from data to knowledge, 285-301.
• Patanè, G. & Russo, M. (2001). The enhanced LBG algorithm. Neural networks, 14(9), 1219-1237. https://doi.org/10.1016/S0893-6080(01)00104-6
• Rahkar Farshi, T., Demirci R. & Mohammad-Reza F. (2018). Image clustering with optimization algorithms and color space. Entropy 20(4) 296-314. https://doi.org/10.3390/e20040296
• Sathya, P.D., Kalyani, R., & Sakthivel, V.P. (2021). Color image segmentation using Kapur, Otsu and minimum cross entropy functions based on exchange market algorithm. Expert Systems with Applications, 172, 114636. https://doi.org/10.1016/j.eswa.2021.114636
• Singhal, A., Agarwal, M., & Pachori, R. B. (2021). Directional local ternary co-occurrence pattern for natural image retrieval. Multimedia Tools and Applications, 80(10), 15901-15920. https://doi.org/10.1007/s11042-020-10319-4
• Smith, J. R. & Chang, S. F. (1996, March). Tools and techniques for color image retrieval. In Storage and retrieval for still image and video databases iv (Vol. 2670, pp. 426-437). International Society for Optics and Photonics.
• Smith, J. R. & Chang, S. F. (1997, February). VisualSEEk: a fully automated content-based image query system. In Proceedings of the fourth ACM international conference on Multimedia (pp. 87-98).
• Vemuru, K. V. (2020). Image Edge Detector with Gabor Type Filters Using a Spiking Neural Network of Biologically Inspired Neurons. Algorithms, 13(7), 165. https://doi.org/10.3390/a13070165
• Yuan, B. H., & Liu, G. H. (2020). Image retrieval based on gradient-structures histogram. Neural Computing and Applications, 32(15), 11717-11727. https://doi.org/10.1007/s00521-019-04657-0
• Wu, Q., McGinnity, M., Maguire, L., Belatreche, A. & Glackin, B. (2007, August). Edge detection based on spiking neural network model. In International Conference on Intelligent Computing (pp. 26-34). Springer, Berlin, Heidelberg.