ODAKLAMA DERİNLİĞİNİN ARTIRILMASINDA DERİN ÖZELLİKLERİN ODAKLAMA DEĞERLERİNİN ÇIKARILMASINDAKİ ETKİLERİNİN İNCELENMESİ

Year 2023, Volume: 31 Issue: 3, 917 - 930, 16.12.2023

Sibel Danışmaz , Sümeyye Nur Emir , Hülya Doğan , Ramazan Özgür Doğan

https://doi.org/10.31796/ogummf.1299670

Abstract

Mikroskobik sistemlerde var olan odaklama derinliğinden dolayı numunenin tüm alanının odaklandığı görüntü elde etmek imkânsız olabilmektedir. Bu durum, mikroskobik sistemlerde görüntü işleme ve yapay zekâ algoritmaları kullanılarak gerçekleştirilen sınıflandırma, bölütleme, hizalama (registration), panoramik birleştirme (stitching) gibi uygulamalarının başarılarını olumsuz yönde etkilemektedir. Literatürde numunenin tüm alanının odaklandığı görüntü elde etmek için odaklama derinliğinin artırılması yaklaşımları geliştirilmektedir. Literatür çalışmaları, bu yaklaşımların, görüntülerdeki eğrilerin ve kenarların düşük kesinlikte karakterizasyonu, daha yüksek koşma süresi ve incelenen numuneye ve kullanılan mikroskoba göre performans değişimi gibi çeşitli kısıtlamalara sahip olduklarını ortaya koymaktadır. Ek olarak, bu yaklaşımlar odaklama bilgilerini genelde görüntülerin gri seviye değerlerini kullanarak hesaplamaktadırlar. Bu çalışmada bu kısıtlamaları minimize etmek için yeni bir odaklama derinliğinin artırılması yaklaşımı geliştirilmekte ve odaklama derinliğinin artırılmasında derin özelliklerin odaklama değerlerinin çıkarılmasındaki etkileri incelenmektedir. Çalışmada elde edilen sonuçlar derin özelliklerin piksellerin odaklama değerlerini hesaplamada gri seviye değerlerine göre daha etkin olduğunu göstermektedir.

Keywords

Odaklama derinliği, Odaklama derinliğinin artırılması, Derin öğrenme, Odaklama ölçüm operatörü

INVESTIGATION OF EFFECTS OF DEEP FEATURES ON FOCUS VALUES EXTRACTION IN EXTENDED DEPTH OF FOCUS

Abstract

Due to the focusing depth in microscopic systems, it may be impossible to obtain an image in which the entire area of the sample is focused. This situation negatively affects the success of applications such as classification, segmentation, registration, panoramic stitching, which are performed using image processing and artificial intelligence algorithms in microscopic systems. In the literature, approaches are developed to increase the focusing depth to obtain an image in which the entire area of the sample is focused. Literature studies reveal that these approaches have several limitations, such as low-precision characterization of curves and edges in images, higher running time, and performance variation according to the sample examined and the microscope used. In addition, these approaches often calculate focusing information using the gray level values of the images. In this study, a new approach to increasing the focusing depth is developed in order to minimize these limitations and the effects of deep features on the extraction of focusing values in increasing the focusing depth are examined. The results obtained in the study show that the deep features are more effective in calculating the focusing values of the pixels than the gray level values.

Keywords

Depth of focus, Extended depth of focus, Deep learning, Focus measurement operator

References

• Aguet, F., Van De Ville, D. ve Unser, M. (2008). Model-based 2.5-D deconvolution for extended depth of field in brightfield microscopy. IEEE Transactions on Image Processing, 17(7), 1144-1153. doi: https://doi.org/10.1109/TIP.2008.924393
• Akpinar, U., Sahin, E., Meem, M., Menon, R. ve Gotchev, A. (2021). Learning wavefront coding for extended depth of field imaging. IEEE transactions on image processing, 30, 3307-3320. doi: https://doi.org/10.1109/TIP.2021.3060166
• Ambikumar, A. S., Bailey, D. G. ve Gupta, G. S. (2016). Extending the depth of field in microscopy: A review. 2016 International Conference on Image and Vision Computing New Zealand (IVCNZ), 1-6.
• Cao, Z., Zhai, C., Li, J., Xian, F. ve Pei, S. (2017). Combination of color coding and wavefront coding for extended depth of field. Optics Communications, 392, 252-257. doi: https://doi.org/10.1016/j.optcom.2017.02.016
• Chen, J., Li, X., Luo, L., Mei, X. ve Ma, J. (2020). Infrared and visible image fusion based on target-enhanced multiscale transform decomposition. Information Sciences, 508, 64-78. doi: https://doi.org/10.1016/j.ins.2019.08.066
• Cohen, N., Yang, S., Andalman, A., Broxton, M., Grosenick, L., Deisseroth, K., Horowitz, M. ve Levoy, M. (2014). Enhancing the performance of the light field microscope using wavefront coding. Optics express, 22(20), 24817-24839. doi: https://doi.org/10.1364/OE.22.024817
• Costa, M. G. F., Pinto, K. M. B., Fujimoto, L. B., Ogusku, M. M. ve Costa Filho, C. F. (2019). Multi-focus image fusion for bacilli images in conventional sputum smear microscopy for tuberculosis. Biomedical Signal Processing and Control, 49, 289-297. doi: https://doi.org/10.1016/j.bspc.2018.12.018
• Crete, F., Dolmiere, T., Ladret, P. ve Nicolas, M. (2007). The blur effect: perception and estimation with a new no-reference perceptual blur metric. In Human vision and electronic imaging XII, 6492, 196-206.
• Dogan, H., Baykal, E., Ekinci, M., Ercin, M. E. ve Ersoz, S. (2018). A novel extended depth of field process based on nonsubsampled shearlet transform by estimating optimal range in microscopic systems. Optics Communications, 429, 88-99. doi: https://doi.org/10.1016/j.optcom.2018.08.006
• Dowski, E. R. ve Cathey, W. T. (1995). Extended depth of field through wave-front coding. Applied optics, 34(11), 1859-1866. doi: https://doi.org/10.1364/AO.34.001859
• Du, H., Dong, L., Liu, M., Zhao, Y., Wu, Y., Li, X., Jia, W., Liu X., Hui, M. ve Kong, L. (2019). Increasing aperture and depth of field simultaneously with wavefront coding technology. Applied Optics, 58(17), 4746-4752. doi: https://doi.org/10.1364/AO.58.004746
• Elmalem, S., Giryes, R. ve Marom, E. (2018). Learned phase coded aperture for the benefit of depth of field extension. Optics express, 26(12), 15316-15331. doi: https://doi.org/10.1364/OE.26.015316
• Forster, B., Van De Ville, D., Berent, J., Sage, D. ve Unser, M. (2004). Complex wavelets for extended depth‐of‐field: A new method for the fusion of multichannel microscopy images. Microscopy research and technique, 65(1‐2), 33-42. doi: https://doi.org/10.1002/jemt.20092
• Gierlak, M., Albrecht, S., Kauer, J., Leverenz, E. ve Beckers, I. E. (2013). Wavefront coding using a spatial light modulator for extended depth of field microscopy. In European Conference on Biomedical Optics, p. 879803.
• Hermessi, H., Mourali, O. ve Zagrouba, E. (2021). Multimodal medical image fusion review: Theoretical background and recent advances. Signal Processing, 183, 108036. doi: https://doi.org/10.1016/j.sigpro.2021.108036
• Huang, M., Liu, S., Li, Z., Feng, S., Wu, D., Wu, Y. ve Shu, F. (2022). Remote sensing image fusion algorithm based on two-stream fusion network and residual channel attention mechanism. Wireless Communications and Mobile Computing, 2022, 1-14. doi: https://doi.org/10.1155/2022/8476000
• Jose, J., Gautam, N., Tiwari, M., Tiwari, T., Suresh, A., Sundararaj, V. ve Rejeesh, M. R. (2021). An image quality enhancement scheme employing adolescent identity search algorithm in the NSST domain for multimodal medical image fusion. Biomedical Signal Processing and Control, 66, 102480. doi: https://doi.org/10.1016/j.bspc.2021.102480
• Li, L., Si, Y., Wang, L., Jia, Z. ve Ma, H. (2020). A novel approach for multi-focus image fusion based on SF-PAPCNN and ISML in NSST domain. Multimedia Tools and Applications, 79, 24303-24328. doi: https://doi.org/10.1007/s11042-020-09154-4
• Li, Y., Wang, J., Zhang, X., Hu, K., Ye, L., Gao, M., Cao, Y. ve Xu, M. (2022). Extended depth-of-field infrared imaging with deeply learned wavefront coding. Optics Express, 30(22), 40018-40031. doi: https://doi.org/10.1364/OE.471443
• Liu, S., Wang, M., Yin, L., Sun, X., Zhang, Y. D. ve Zhao, J. (2022). Two-scale multimodal medical image fusion based on structure preservation. Frontiers in Computational Neuroscience, 15, 133. doi: https://doi.org/10.3389/fncom.2021.803724
• Liu, Y., Wang, L., Cheng, J., Li, C. ve Chen, X. (2020). Multi-focus image fusion: A survey of the state of the art. Information Fusion, 64, 71-91. doi: https://doi.org/10.1016/j.inffus.2020.06.013
• Mo, X., Zhang, T., Wang, B., Huang, X., Kuang, C. ve Liu, X. (2019). Alleviating image artifacts in wavefront coding extended depth of field imaging system. Optics Communications, 436, 232-238. doi: https://doi.org/10.1016/j.optcom.2018.12.006
• Pan, C., Chen, J., Zhang, R. ve Zhuang, S. (2008). Extension ratio of depth of field by wavefront coding method. Optics express, 16(17), 13364-13371. doi: https://doi.org/10.1364/OE.16.013364
• Pertuz, S., Puig, D. ve Garcia, M. A. (2013). Analysis of focus measure operators for shape-from-focus. Pattern Recognition, 46(5), 1415-1432. doi: https://doi.org/10.1016/j.patcog.2012.11.011
• Piccinini, F., Tesei, A., Zoli, W. ve Bevilacqua, A. (2012). Extended depth of focus in optical microscopy: Assessment of existing methods and a new proposal. Microscopy research and technique, 75(11), 1582-1592. doi: https://doi.org/10.1002/jemt.22104
• Ramlal, S. D., Sachdeva, J., Ahuja, C. K. ve Khandelwal, N. (2019). An improved multimodal medical image fusion scheme based on hybrid combination of nonsubsampled contourlet transform and stationary wavelet transform. International Journal of Imaging Systems and Technology, 29(2), 146-160. doi: https://doi.org/10.1002/ima.22310
• Tan, W., Tiwari, P., Pandey, H. M., Moreira, C. ve Jaiswal, A. K. (2020). Multimodal medical image fusion algorithm in the era of big data. Neural Computing and Applications, 1-21. doi: https://doi.org/10.1007/s00521-020-05173-2
• Tessens, L., Ledda, A., Pizurica, A. ve Philips, W. (2007). Extending the depth of field in microscopy through curvelet-based frequency-adaptive image fusion. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP'07, 1, I-861.
• Valdecasas, A. G., Marshall, D., Becerra, J. M. ve Terrero, J. J. (2001). On the extended depth of focus algorithms for bright field microscopy. Micron, 32(6), 559-569. doi: https://doi.org/10.1016/S0968-4328(00)00061-5
• Wang, K., Zheng, M., Wei, H., Qi, G. ve Li, Y. (2020). Multi-modality medical image fusion using convolutional neural network and contrast pyramid. Sensors, 20(8), 2169. doi: https://doi.org/10.3390/s20082169
• Wei, X., Han, J., Xie, S., Yang, B., Wan, X. ve Zhang, W. (2019). Experimental analysis of a wavefront coding system with a phase plate in different surfaces. Applied Optics, 58(33), 9195-9200. doi: https://doi.org/10.1364/AO.58.009195
• Ye, F., Li, X. ve Zhang, X. (2019). FusionCNN: a remote sensing image fusion algorithm based on deep convolutional neural networks. Multimedia Tools and Applications, 78, 14683-14703. doi: https://doi.org/10.1007/s11042-018-6850-3
• Zhao, T., Mauger, T. ve Li, G. (2013). Optimization of wavefront-coded infinity-corrected microscope systems with extended depth of field. Biomedical optics express, 4(8), 1464-1471. doi: https://doi.org/10.1364/BOE.4.001464