Mısır Tohumu Embriyolarının Bölütlenmesinde Tam Evrişimsel Ağ Tabanlı Mimarilerin Tam Bağlı Şartlı Rastgele Alanlar ile Entegrasyonu

Year 2022, Volume: 22 Issue: 1, 100 - 111, 28.02.2022

Serdar Alasu , Muhammed Fatih Talu

https://doi.org/10.35414/akufemubid.1013047

Abstract

Haploid ve diploid mısır tohumlarının sınıflandırılması mısır ıslahında önemli bir konudur. R1-nj renk markörü sayesinde haploid ve diploid mısır tohumları embriyolarındaki renklenme farklılıkları dikkate alınarak görsel olarak ayırt edilebilmektedir. Bu nedenle, mısır tohumu embriyolarının bölütlenmesi haploid ve diploid mısır tohumlarının sınıflandırılması için önemli bir ön-işlemdir. Bu çalışmada, mısır tohumu görüntülerinin otomatik embriyo bölütlemesinde tam evrişim ağ tabanlı derin öğrenme mimarilerinin (FCN8s, SegNet ve U-Net) bölütleme performansları değerlendirilmektedir ve bölütleme çıktılarının tam bağlı Şartlı Rastgele Alanlar yöntemiyle düzgünleştirilmesi incelenmektedir. Böylece tam bağlı Şartlı Rastgele Alanların bölütleme sonucuna etkisi araştırılmıştır Ayrıca bu çalışma için mısır tohumu görüntüleri piksel seviyesinde etiketlenerek referans görüntüler elde edilmiş ve haploid ve diploid mısır tohumu görüntüleri için yeni bir semantik görüntü bölütleme veri seti oluşturulmuştur. Çalışma sonuçları göstermiştir ki, tam evrişim ağ tabanlı derin öğrenme mimarileri ile tam bağlı Şartlı Rastgele Alanlar’ın birlikte kullanımı, görüntü bölütleme sonucunu ortalama IoU performans değerlendirme metriğinde FCN8s, SegNet ve U-Net derin öğrenme mimarileri için sırasıyla 0.0139, 0.0076, 0.0024 iyileştirdiği görülmüştür.

Keywords

Mısır Tohumu Embriyo Bölütleme, Semantik Bölütleme, Tam Evrişimsel Ağlar, SegNet, U-Net, Tam Bağlı Şartlı Rastgele Alanlar

Integration of Fully Convolutional Network Based Architectures with Fully Connected Conditional Random Fields in Maize Seed Embryos Segmentation

Abstract

Classification of haploid and diploid maize seeds is an important issue in maize breeding. Thanks to the R1-nj color marker, haploid and diploid maize seeds can be visually distinguished by considering the coloration differences in embryos. Therefore, segmentation of maize seed embryos is an important pre-processing for the classification of haploid and diploid maize seeds. In this study, the segmentation performances of fully convolution network-based deep learning architectures (FCN8s, SegNet and U-Net) in automatic embryo segmentation of maize seed images are evaluated and the smoothing of segmentation outputs with the fully connected Conditional Random Fields method is examined. Thus, the effect of fully connected Conditional Random Fields on the segmentation result was studied. In addition, for this study ground truths were obtained by labeling the maize seed images at the pixel level, and a new semantic image segmentation dataset was created for the haploid and diploid maize seed images. The study results showed that the combined use of full convolution network-based deep learning architectures and fully connected Conditional Random Fields improved the image segmentation result in the mean IoU performance evaluation metric for FCN8s, SegNet and U-Net deep learning architectures by 0.0139, 0.0076, 0.0024, respectively.

Keywords

Maize Seed Embryo Segmentation, Semantic Segmentation, Fully Convolutional Networks, SegNet, U-Net, Fully Connected Conditional Random Fields

References

• Altuntaş, Y., Kocamaz, A.F., Cengiz, R. ve Esmeray M., 2018a. Haploid ve Diploid Mısır Tohumlarının Görüntü İşleme Teknikleri ve Destek Vektör Makineleri Kullanılarak Sınıflandırılması. 26th Signal Processing and Communications Applications Conference (SIU), 1-4.
• Altuntaş, Y., Kocamaz, A.F., Cömert, Z., Cengiz, R. and Esmeray, M., 2018b. Identification of haploid maize seeds using gray level co-occurrence matrix and machine learning techniques. In 2018 International Conference on Artificial Intelligence and Data Processing (IDAP), 1-5.
• Altuntaş, Y., Cömert, Z. and Kocamaz, A.F., 2019a. Identificaton of haploid and diploid maize seeds using convolutional neural networks and a transfer learning approach. Computers and Electronics in Agriculture, 163, 104874.
• Altuntaş, Y. ve Kocamaz, A.F., 2019b. Renk Momentleri ve Destek Vektör Makineleri Kullanarak Haploid Mısır Tohumlarının Tanımlanmasında Renk Uzaylarının Sınıflandırma Performansına Etkisinin Karşılaştırılması. Fırat Üniversitesi Mühendislik Bilim Dergisi, 31(2), 551–560.
• Arani, M.N. and Zhang, X.P., 2014. Generalized Gaussian mixture Conditional Random Field model for image labeling. 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 1068-1072.
• Arnab, A., Zheng, S., Jayasumana, S., Romera-Paredes, B., Larsson, M. and Kirillov, A., 2018. Conditional Random Fields Meet Deep Neural Networks for Semantic Segmentation: Combining Probabilistic Graphical Models with Deep Learning for Structured Prediction. IEEE Signal Processing Magazine, 35(1), 37-52.
• Badrinarayanan, V., Kendall, A. and Cipolla, R., 2017. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 39(12), 2481–2495.
• Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K. and Yuille, A.L., 2017. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834-848.
• Chu, X., Tao, Y., Wang, W., Yuan, Y. and Xi, M., 2014. Rapid detection method of moldy maize kernels based on color feature. Advances in Mechanical Engineering, 6, 625090.
• De La Fuente, G.N., Carstensen, J.M., Edberg, M.A. and Lübberstedt T., 2017. Discrimination of haploid and diploid maize kernels via multispectral imaging. Plant Breed, 136(1), 50–60.
• Deng, J., Dong W., Socher, R., Li, L.J., Li K. and Fei-Fei, L., 2009 ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248-255.
• Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V. and Garcia-Rodriguez, J., 2017. A Review on Deep Learning Techniques Applied to Semantic Segmentation. arXiv preprint, 1-23.
• He, K., Zhang, X., Ren, S. and Sun J., 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. IEEE International Conference on Computer Vision, 1026-1034.
• He, X., Zemel, R.S. and Carreira-Perpinán, M.A., 2004. Multiscale Conditional Random Fields for Image Labeling. 2004 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 2, pp. II-II.
• Koller, D. and Nir, F., 2009. Probabilistic Graphical Models: Principles and Techniques, MIT press, 3-5.
• Krähenbühl, P. and Koltun, V., 2011. Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials. 24th International Conference on Neural Information Processing Systems (NIPS), 109-117.
• Krizhevsky, A., Sutskever and I., Hinton, G.E., 2012. ImageNet classification with deep convolutional neural networks. 25th International Conference on Neural Information Processing Systems (NIPS), 1097-1105.
• Long, J., Shelhamer, E. and Darrell T., 2015. Fully convolutional networks for semantic segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3431-3440.
• Ma, D., Cheng, H. and Zhang, W., 2013. Maize embryo image acquisition and variety identification based on OTSU and K-means clustering algorithm. 2013 International Conference on Information Science and Cloud Computing Companion, 835-840.
• Minaee, S., Boykov, Y.Y., Porikli, F., Plaza, A.J., Kehtarnavaz, N. and Terzopoulos, D., 2021. Image Segmentation Using Deep Learning: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1-20. https://doi.org/10.1109/TPAMI.2021.3059968
• Murphy, K.P., 2012. Machine Learning: A Probabilistic Perspective, MIT press, 91-93.
• Nanda, D.K. and Chase, S.S., 1966. An Embryo Marker for Detecting Monoploids Of Maize (Zea Mays L.). Crop Science, 6(2), 213–215, 1966.
• Otsu, N., 1979. A threshold selection method from gray level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62–66, 1979.
• Ronneberger, O., Fischer, P. and Brox, T., 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. International Conference on Medical Image Computing & Computer Assisted Intervention (MICCAI), 234-241.
• Simonyan, K. and Zisserman, A., 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. 3rd International Conference on Learning Representations (ICLR).
• Sultana, F., Sufian, A. and Dutta, P., 2020. Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey. Knowledge-Based Systems, 201, 106062.
• Toyoda, T. and Hasegawa, O., 2008. Random field model for integration of local information and global information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(8), 1483-1489.
• Wang, X.Y., Liao, W.X., An, D. and Wei, Y.G., 2018. Maize Haploid Identification via LSTM-CNN and Hyperspectral Imaging Technology. arXiv preprint, 1-14 .
• Wang, X., Zhang, X.P., Clarke, I. and Yakubovich, Y., 2009. A new Gaussian mixture conditional random field model for indoor image labeling. 1st International Workshop on Interactive Multimedia for Consumer Electronics, 51-56.