Deep learning microscopy image enhancement for a line scanning confocal microscope

Amirmohammad Ketabchi; Nima Bavili; Berna Morova

doi:10.33769/aupse.1621490

Deep learning microscopy image enhancement for a line scanning confocal microscope

Abstract

Confocal microscopy provides images with a higher resolution and signal-to-noise ratio in comparison with wide-field illumination microscopy by eliminating the out-of-focus background. Besides the advances in confocal microscopy, new techniques like light-sheet microscopy, structured illumination microscopy, line-scanning confocal microscopy, or deep learning enhanced microscopy draw attention due to their advantages including higher image acquisition speed, higher contrast, and better resolution, and being more cost- effective. In this work, we first demonstrate the operation of a line-scanning confocal microscope developed using a digital light projector (DLP) and a CMOS camera with a rolling shutter. In this method, a series of illumination lines are projected on a sample by a DLP and focusing objective (50X, NA=0.55). The reflected light is imaged with a rolling shutter CMOS camera. The line-scanning confocal imaging is achieved by overlapping the illumination lines and the rolling shutter of the sensor. Significant improvement of ∼52% in image contrast is obtained using this technique. Then by using this setup, a dataset including 500 pairs of images of fibers of a tissue paper is prepared. This dataset is employed for training a Convolutional Neural Network (CNN) and a conditional Generative Adversarial Network (cGAN) for deep learning enhanced microscopy. 95% of the dataset is used for the training of the net- work, while the remaining 5% is used for the validation. ∼45% and ∼47% contrast improvement was measured in the test images for CNN and cGAN, respectively, comparable to the ground-truth images.

Keywords

Project Number

118F529

References

Jonkman J, et al. Tutorial: guidance for quantitative confocal microscopy. Nature Protocols 2020; 15(5): p. 1585-1611. https://doi.org/10.1038/s41596-020-0313-9.
Paddock SW. Principles and practices of laser scanning confocal microscopy. Molecular Biotechnology 2000; 16(2): p. 127-149. https://doi.org/10.1385/MB:16:2:127.
Oreopoulos J, Berman R, and Browne M. Spinning-disk confocal microscopy: present technology and future trends. Methods in Cell Biology 2014; 123: p. 153-175. https://doi.org/10.1016/B978-0-12-420138-5.00009-4.
Gareau DS , Abeytunge S, and Rajadhyaksha M, Line-scanning reflectance confocal microscopy of human skin: comparison of full-pupil and divided-pupil configurations. Optics Letters 2009; 34(20): p. 3235-3237. https://doi.org/10.1364/OL.34.003235.
Muller MS, Elsner AE, Ozawa GY. Non-Mydriatic Confocal Retinal Imaging Using a Digital Light Projector. Proceedings of SPIE-The International Society for Optical Engineering 2013 Mar 26;8567. https://doi.org/10.1117/12.2001305.
Deng L, Hinton G, Kingsbury B. New types of deep neural network learning for speech recognition and related applications: an overview. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada; 2013. pp. 8599-8603, https://doi.org/10.1109/ICASSP.2013.6639344.
Zeiler MD, Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012. https://doi.org/10.48550/arXiv.1212.5701.
Renuka DK, Hamsapriya T, Chakkaravarthi MR, Surya PL. Spam Classification Based on Supervised Learning Using Machine Learning Techniques. 2011 International Conference on Process Automation, Control and Computing, Coimbatore, India; 2011. pp. 1-7, https://doi.org/10.1109/PACC.2011.5979035.

Gardner-Medwin AR, Barlow HB. The limits of counting accuracy in distributed neural representations. Neural Computation 2001 Mar; 13(3):477-504. https://doi.org/10.1162/089976601300014420.
Kaelbling LP, Littman ML, Moore AW. Reinforcement learning: A survey. Journal of Artificial Intelligence Research 1996; 4: pp.237-285. https://doi.org/10.1613/jair.301.
Lee WH, Ozger M, Challita U, Sung KW. Noise Learning-Based Denoising Autoencoder. IEEE Communications Letters 2021; 25(9): pp. 2983-2987. https://doi.org/10.1109/LCOMM.2021.3091800.
Vincent P, Larochelle H, Bengio Y, and Manzagol PA. Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th International Conference on Machine Learning (ICML) 2008; pp. 1096-1103. https://doi.org/10.1145/1390156.1390294.
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, and Bengio Y. Generative adversarial networks. Communications of the ACM 2016; 63(11), 139- 144. https://doi.org/10.1145/3422622.
Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA. Generative Adversarial Networks: An Overview, in IEEE Signal Processing Magazine 2018; 35(1), pp. 53-65. https://doi.org/10.1109/MSP.2017.2765202.
Li S, Li W, Cook C, Zhu C, Gao Y. Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2018; pp. 5457-5466. https://doi.org/10.1109/CVPR.2018.00572.
Yu Y, Si X, Hu C, Zhang J. A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures. Neural Computation 2019; 31(7):1235-1270. https://doi.org/10.1162/neco a 01199.
De K, Masilamani V. Image Sharpness Measure for Blurred Images in Frequency Domain. Procedia Engineering 2013; 64: pp. 149-158. https://doi.org/10.1016/j.proeng.2013.09.086. DEEP LEARNING MICROSCOPY IMAGE ENHANCEMENT 151
Sultana F, Sufian A, Dutta P. Advancements in Image Classification using Convolutional Neural Network. 2018 Fourth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), Kolkata, India; 2018. pp. 122-129, https://doi.org/10.1109/ICRCICN.2018.8718718.
Ghiasi-Shirazi K. Competitive Cross-Entropy Loss: A Study on Training Single-Layer Neural Networks for Solving Nonlinearly Separable Classification Problems. Neural Processing Letters 2019; 50: p.1115–1122. https://doi.org/10.1007/s11063-018-9906-5.
Bera S, Shrivastava VK. Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remote sensing image classification. International Journal of Remote Sensing 2019; 41(7): pp.2664–2683. https://doi.org/10.1080/01431161.2019.1694725.
Webb S. Deep learning for biology. Nature 2018; 554(7690): p. 555-558. https://doi.org/10.1038/d41586-018-02174-z.
Agrawal A, Choudhary A. Deep materials informatics: Applications of deep learning in materials science. MRS Communications 2019; 9:779–792. https://doi.org/10.1557/mrc.2019.73.
Wang F, Casalino LP, Khullar D. Deep Learning in Medicine—Promise, Progress, and Challenges. JAMA Intern Medicine 2019; 179(3):293–294. https://doi.org/10.1001/jamainternmed.2018.7117.
Li J, Tong G, Pan Y, Yu Y. Spatial and temporal super-resolution for fluorescence microscopy by a recurrent neural network. Optics Express 2021; 29: 15747-15763. https://doi.org/10.1364/OE.423892.
Wang H, Rivenson Y, Jin Y, Wei Z, Gao R, Gunaydin H, Bentolila LA, Kural C, Ozcan A. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nature Methods 2019; 16(1):103-110. https://doi.org/10.1038/s41592-018-0239-0.
Rivenson Y, Göröcs Z, Günaydin H, Zhang Y, Wang H, Ozcan A. Deep learning microscopy. Optica 2017; 4:1437-1443. https://doi.org/10.1364/OPTICA.4.001437.
Christensen CN, Ward EN, Lu M, Lio P, Kaminski CF. ML-SIM: universal reconstruction of structured illumination microscopy images using transfer learning. Biomedical Optics Express 2021; 12(5):2720-2733. https://doi.org/10.1364/BOE.414680.
Midtvedt B, Helgadottir A, Argun A, Pineda J, Midtvedt D, Volpe G. Quantitative digital microscopy with deep learning. Applied Physics Reviews 2021; 8(1): 011310. https://doi.org/10.1063/5.0034891.
Ketabchi AM, Morova B, Uysalli Y, Aydin M, Eren F, Bavili N, Pysz D, Buczynski R, Kiraz A. Enhancing resolution and contrast in fibre bundle-based fluorescence microscopy using generative adversarial network. Journal of Microscopy 2024; 295, 236–242. https://doi.org/10.1111/jmi.13296.
Ketabchi AM, Morova B, Bavili N, Kiraz A. Contrast improvement through a generative adversarial network (GAN) by utilizing a dataset obtained from a line-scanning confocal microscope. SPIE Photonics Europe 2024; 12998. https://doi.org/10.1117/12.3016368.
Guo M, Wu Y, Hobson CM et al. Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy. Nature Communications 2025; 16, 313. https://doi.org/10.1038/s41467-024-55267-x.
Cui Z, Xing Y, Chen Y, Zheng X, Liu W, Kuang C, Chen Y. Real-Time Resolution Enhancement of Confocal Laser Scanning Microscopy via Deep Learning. Photonics 2024; 11, 983. https://doi.org/10.3390/photonics11100983.
Shuai Liu, Wenzhen Zou, Hao Sha, Xiaochen Feng, Bin Chen, Jian Zhang, Sanyang Han, Xiu Li, and Yongbing Zhang, ”Deep learning-enhanced snapshot hyperspectral confocal microscopy imaging system, Optics Express 2024; 32, 13918-13931. https://doi.org/10.1364/OE.519045. 152 A. KETABCHI, N. BAVILI, AND B. MOROVA
Muller MS, Green JJ, Baskaran K, Ingling AW, Clendenon JL, Gast TJ, Elsner AE. Non-mydriatic confocal retinal imaging using a digital light projector. Proceedings of SPIE - The International Society for Optical Engineering 2015;9376:93760E. https://doi.org/10.1117/12.2077704.
Isola P, Zhu JY, Zhou T, Efros AA. Image-to-Image Translation with Conditional Adversarial Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA; 2017. pp. 5967-5976. https://doi.org/10.1109/CVPR.2017.632.
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial networks. Communications of the ACM 2020; 63(11):139–144. https://doi.org/10.1145/3422622.
Morova B et al., Fabrication and characterization of large numerical aperture, high-resolution optical fiber bundles based on high-contrast pairs of soft glasses for fluorescence imaging. Optics Express 2019; 27(7): p.9502-9515. https://doi.org/10.1364/OE.27.009502.

Details

Primary Language

English

Subjects

Biomedical Imaging

Journal Section

Research Article

Authors

Amirmohammad Ketabchi This is me
0000-0002-3386-269X
Türkiye

Nima Bavili This is me
0000-0002-7843-3514
Türkiye

Berna Morova ^*
0000-0003-1293-7362
Türkiye

Publication Date

December 24, 2025

Submission Date

January 24, 2025

Acceptance Date

July 28, 2025

Published in Issue

Year 2025 Volume: 67 Number: 2

DOI

https://doi.org/10.33769/aupse.1621490

IZ

https://izlik.org/JA95HF74ZY

Cite

RIS / Bibtex

APA

Ketabchi, A., Bavili, N., & Morova, B. (2025). Deep learning microscopy image enhancement for a line scanning confocal microscope. Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering, 67(2), 135-152. https://doi.org/10.33769/aupse.1621490

AMA

1.Ketabchi A, Bavili N, Morova B. Deep learning microscopy image enhancement for a line scanning confocal microscope. Commun.Fac.Sci.Univ.Ank.Series A2-A3: Phys.Sci. and Eng. 2025;67(2):135-152. doi:10.33769/aupse.1621490

Chicago

Ketabchi, Amirmohammad, Nima Bavili, and Berna Morova. 2025. “Deep Learning Microscopy Image Enhancement for a Line Scanning Confocal Microscope”. Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering 67 (2): 135-52. https://doi.org/10.33769/aupse.1621490.

EndNote

Ketabchi A, Bavili N, Morova B (December 1, 2025) Deep learning microscopy image enhancement for a line scanning confocal microscope. Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering 67 2 135–152.

IEEE

[1]A. Ketabchi, N. Bavili, and B. Morova, “Deep learning microscopy image enhancement for a line scanning confocal microscope”, Commun.Fac.Sci.Univ.Ank.Series A2-A3: Phys.Sci. and Eng., vol. 67, no. 2, pp. 135–152, Dec. 2025, doi: 10.33769/aupse.1621490.

ISNAD

Ketabchi, Amirmohammad - Bavili, Nima - Morova, Berna. “Deep Learning Microscopy Image Enhancement for a Line Scanning Confocal Microscope”. Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering 67/2 (December 1, 2025): 135-152. https://doi.org/10.33769/aupse.1621490.

JAMA

1.Ketabchi A, Bavili N, Morova B. Deep learning microscopy image enhancement for a line scanning confocal microscope. Commun.Fac.Sci.Univ.Ank.Series A2-A3: Phys.Sci. and Eng. 2025;67:135–152.

MLA

Ketabchi, Amirmohammad, et al. “Deep Learning Microscopy Image Enhancement for a Line Scanning Confocal Microscope”. Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering, vol. 67, no. 2, Dec. 2025, pp. 135-52, doi:10.33769/aupse.1621490.

Vancouver

1.Amirmohammad Ketabchi, Nima Bavili, Berna Morova. Deep learning microscopy image enhancement for a line scanning confocal microscope. Commun.Fac.Sci.Univ.Ank.Series A2-A3: Phys.Sci. and Eng. 2025 Dec. 1;67(2):135-52. doi:10.33769/aupse.1621490