Development of Autoencoder Based Models for High Accuracy Nuclear Segmentation in Fluorescent Microscope Systems

Year 2023, , 801 - 813, 15.09.2023

Sümeyye Nur Emir , Sibel Danışmaz , Hülya Doğan , Ramazan Özgür Doğan

https://doi.org/10.31466/kfbd.1222686

Abstract

Hematoxylin and eosin (H&E) histological staining, immunohistochemical (IHC) and immunofluorescence (IF) staining approaches have been developed to visualize sections such as nuclear morphology or biomarkers in tissue or cell samples in microscopic systems. Compared to H&E or IHC staining, digitizing fluorescent staining is more challenging and time consuming for experts. However, more cellular or subcellular markers can be visualized in IF staining approaches. Automated nuclear segmentation obtained from fluorescent microscopes with high accuracy provides more information about cells in IF staining approaches. In the literature, many studies have been developed for cell or tissue segmentation in images obtained from microscopic systems and high accuracy results have been obtained. However, this success in other fields has not been achieved for nuclear segmentation in images obtained from fluorescent microscopes. In this context, high-accuracy autoencoder models are developed for nuclear segmentation in fluorescent microscope systems. The analysis of the developed autoencoder models is carried out using a dataset of fluorescent microscope images marked by experts. In terms of the performance evaluation procedures used in the study, it is clearly seen that the success of the autoencoder models performed is satisfactory for automatic nuclear segmentation.

Keywords

Fluorescent microscope, Nuclear segmentation, Deep learning, Autoencoder, Transfer Learning, U-Net

Floresan Mikroskop Sistemlerinde Yüksek Doğruluklu Nükleer Segmentasyonu için Otomatik Kodlayıcı Tabanlı Modellerin Geliştirilmesi

Abstract

Mikroskobik sistemlerde doku veya hücre numunelerinde nükleer morfoloji veya biyolojik belirteçler gibi bölümleri görselleştirmek için hematoksilen ve eozin (Hematoxylin and eosin - H&E) histolojik boyamalar, immünohistokimyasal (immunohistovhemical - IHC) ve immünofloresan (immunofluorescence - IF) boyama yaklaşımları geliştirilmiştir. H&E veya IHC boyamalar ile karşılaştırıldığında, IF boyamaların sayısala aktarılması uzmanlar için daha zorlu ve zaman alıcı olmaktadır. Fakat, IF boyama yaklaşımlarında daha fazla hücresel veya hücre altı belirteç görüntülenebilmektedir. Floresan mikroskoplardan elde edilmiş nükleer segmentasyonunun yüksek doğrulukla otomatik gerçekleştirilmesi IF boyama yaklaşımlarındaki hücreler hakkında daha fazla bilgi elde edilmesini sağlamaktadır. Literatürde diğer mikroskobik sistemlerden elde edilmiş görüntülerde hücre veya doku segmentasyonu için birçok çalışma geliştirilmiş ve yüksek doğruluklu sonuçlar elde edilmiştir. Fakat diğer alanlarda gerçekleştirilen bu başarı, floresan mikroskoplardan elde edilmiş görüntülerdeki nükleer segmentasyonu için elde edilmemiştir. Bu kapsamda, çalışmada floresan mikroskop sistemlerinde nükleer segmentasyonu için yüksek doğruluklu otomatik kodlayıcı modelleri geliştirilmektedir. Geliştirilen otomatik kodlayıcı modellerinin analizi uzman kişiler tarafından işaretlenmiş, floresan mikroskop görüntülerinden oluşan veri seti kullanılarak gerçekleştirilmektedir. Çalışmada kullanılan performans değerlendirme prosedürleri açısından, gerçekleştirilen otomatik kodlayıcı modellerinin başarılarının otomatik nükleer segmentasyon için tatmin edici olduğu açıkça görülmektedir.

Keywords

Floresan mikroskop, Nükleer segmentasyon, Derin öğrenme, Otomatik kodlayıcı, Transfer öğrenme, U-Net

References

• Amiri, M., Brooks, R., and Rivaz, H., (2020). Fine-tuning U-Net for ultrasound image segmentation: different layers, different outcomes. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 67(12), 2510-2518.
• Araújo, F. H., Silva, R. R., Ushizima, D. M., Rezende, M. T., Carneiro, C. M., Bianchi, A. G. C., and Medeiros, F. N., (2019). Deep learning for cell image segmentation and ranking. Computerized Medical Imaging and Graphics, 72, 13-21.
• Caicedo, J. C., Roth, J., Goodman, A., Becker, T., Karhohs, K. W., Broisin, M., ... and Carpenter, A. E., (2019). Evaluation of deep learning strategies for nucleus segmentation in fluorescence images. Cytometry Part A, 95(9), 952-965.
• Cheng, D., and Lam, E. Y., (2021). Transfer learning U-Net deep learning for lung ultrasound segmentation. arXiv preprint arXiv:2110.02196.
• Daniel, J., Rose, J. T., Vinnarasi, F., and Rajinikanth, V., (2022). VGG-UNet/VGG-SegNet Supported Automatic Segmentation of Endoplasmic Reticulum Network in Fluorescence Microscopy Images. Scanning, 2022.
• Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., and Fei-Fei, L., (2009). Imagenet: A large-scale hierarchical image database. IEEE Conference on Computer Vision and Pattern Recognition, 248-255.
• Du, X., and Dua, S., (2010). Segmentation of fluorescence microscopy cell images using unsupervised mining. The open medical informatics journal, 4, 41.
• Durkee, M. S., Abraham, R., Clark, M. R., and Giger, M. L., (2021). Artificial intelligence and cellular segmentation in tissue microscopy images. The American Journal of Pathology, 191(10), 1693-1701.
• He, K., Zhang, X., Ren, S., and Sun, J., (2016). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, 770-778.
• Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... and Adam, H., (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
• Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K. Q., (2017). Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, 4700-4708.
• Ikeda, J., Ohe, C., Yoshida, T., Kuroda, N., Saito, R., Kinoshita, H., ... and Matsuda, T., (2021). Comprehensive pathological assessment of histological subtypes, molecular subtypes based on immunohistochemistry, and tumor‐associated immune cell status in muscle‐invasive bladder cancer. Pathology International, 71(3), 173-182.
• Jeevitha, K., Iyswariya, A., RamKumar, V., Basha, S. M., and Kumar, V. P., (2020). A review on various segmentation techniques in image processsing. European Journal of Molecular & Clinical Medicine, 7(4), 1342-1348.
• Jose, L., Liu, S., Russo, C., Nadort, A., and Di Ieva, A., (2021). Generative adversarial networks in digital pathology and histopathological image processing: A review. Journal of Pathology Informatics, 12(1), 43.
• Kromp, F., Bozsaky, E., Rifatbegovic, F., Fischer, L., Ambros, M., Berneder, M., ... and Taschner-Mandl, S., (2020). An annotated fluorescence image dataset for training nuclear segmentation methods. Scientific Data, 7(1), 1-8.
• Kromp, F., Fischer, L., Bozsaky, E., Ambros, I., Doerr, W., Taschner-Mandl, S., ... and Hanbury, A., (2019). Deep Learning architectures for generalized immunofluorescence based nuclear image segmentation. arXiv preprint arXiv:1907.12975.
• Liu, Z., Jin, L., Chen, J., Fang, Q., Ablameyko, S., Yin, Z., and Xu, Y., (2021). A survey on applications of deep learning in microscopy image analysis. Computers in Biology and Medicine, 134, 104523.
• Masubuchi, S., Watanabe, E., Seo, Y., Okazaki, S., Sasagawa, T., Watanabe, K., ... and Machida, T., (2020). Deep-learning-based image segmentation integrated with optical microscopy for automatically searching for two-dimensional materials. NPJ 2D Materials and Applications, 4(1), 1-9.
• Niazi, M. K. K., Parwani, A. V., and Gurcan, M. N., (2019). Digital pathology and artificial intelligence. The lancet oncology, 20(5), e253-e261.
• Pan, W., Liu, Z., Song, W., Zhen, X., Yuan, K., Xu, F., and Lin, G. N., (2022). An Integrative Segmentation Framework for Cell Nucleus of Fluorescence Microscopy. Genes, 13(3), 431.
• Pare, S., Kumar, A., Singh, G. K., and Bajaj, V., (2020). Image segmentation using multilevel thresholding: a research review. Iranian Journal of Science and Technology, Transactions of Electrical Engineering, 44(1), 1-29.
• Raj, R., Londhe, N. D., and Sonawane, R., (2021). Automated psoriasis lesion segmentation from unconstrained environment using residual U-Net with transfer learning. Computer Methods and Programs in Biomedicine, 206, 106123.
• Ramesh, K. K. D., Kumar, G. K., Swapna, K., Datta, D., and Rajest, S. S., (2021). A review of medical image segmentation algorithms. EAI Endorsed Transactions on Pervasive Health and Technology, 7(27), e6-e6.
• Rizk, A., Paul, G., Incardona, P., Bugarski, M., Mansouri, M., Niemann, A., ... and Sbalzarini, I. F., (2014). Segmentation and quantification of subcellular structures in fluorescence microscopy images using Squassh. Nature protocols, 9(3), 586-596.
• Schubert, W., Bonnekoh, B., Pommer, A. J., Philipsen, L., Böckelmann, R., Malykh, Y., ... and Dress, A. W., (2006). Analyzing proteome topology and function by automated multidimensional fluorescence microscopy. Nature biotechnology, 24(10), 1270-1278.
• Wang, L., Guo, S., Huang, W., and Qiao, Y., (2015). Places205-vggnet models for scene recognition. arXiv preprint arXiv:1508.01667.
• Wang, Q., Niemi, J., Tan, C. M., You, L., and West, M., (2010). Image segmentation and dynamic lineage analysis in single‐cell fluorescence microscopy. Cytometry Part A: The Journal of the International Society for Advancement of Cytometry, 77(1), 101-110.
• Zhang, Y., Jiang, H., Ye, T., and Juhas, M., (2021). Deep learning for imaging and detection of microorganisms. Trends in Microbiology, 29(7), 569-572.