A Synopsis of Machine and Deep Learning in Medical Physics and Radiology

Year 2022, , 946 - 957, 29.09.2022

Zohal Emam , Emel Ada

https://doi.org/10.30621/jbachs.960154

Abstract

Machine learning (ML) and deep learning (DL) technologies introduced in the fields of medical physics, radiology, and oncology have made great strides in the past few years. A good many applications have proven to be an efficacious automated diagnosis and radiotherapy system. This paper outlines DL's general concepts and principles, key computational methods, and resources, as well as the implementation of automated models in diagnostic radiology and radiation oncology research. In addition, the potential challenges and solutions of DL technology are also discussed.

Keywords

Deep learning, Machine learning, Radiology, Oncology

Supporting Institution

Health Science Institute, Dokuz Eylul University

References

• Fouad F. The Fourth Industrial Revolution is the AI Revolution an Academy Prospective. IJISCS. 2019;8(5):155-67. doi: 10.30534/ijiscs/2019/01852019
• Choy G, Khalilzadeh O, Michalski M, et al. Current Applications and Future Impact of Machine Learning in Radiology. Radiology. 2018;288(2):318-28. doi: 10.1148/radiol.2018171820
• Sahiner B, Pezeshk A, Hadjiiski LM, et al. Deep learning in medical imaging and radiation therapy. Med Phys. 2019;46(1): e1-e36. doi: 10.1002/mp.13264
• Avanzo M, Trianni A, Botta F, Talamonti C, Stasi M, Iori M. Artificial Intelligence and the Medical Physicist: Welcome to the Machine. Appl Sci. 2021 Jan;11(4):1691. doi: 10.3390/app11041691
• Shen C, Nguyen D, Zhou Z, Jiang SB, Dong B, Jia X. An introduction to deep learning in medical physics: advantages, potential, and challenges. Phys Med Biol. 2020;65:05TR1. doi: 10.1088/1361-6560/ab6f51
• EC AI HLEG. A definition of Artificial Intelligence: main capabilities and scientific disciplines. High-Level Expert Group on Artificial Intelligence (AI HLEG). Brussels. 2019. https://digital-strategy.ec.europa.eu/en/library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines
• Alpaydin E. Introduction to machine learning. 4th ed. MIT press. Cambridge Massachusetts. 2020.
• Baraniuk R, Donoho D, Gavish M. The science of deep learning. Proc Natl Acad Sci. U S A. 2020;117(48):30029-32. doi: 10.1073/pnas.2020596117
• Sutton RS, Barto AG. Reinforcement learning: An introduction. 2nd ed. Cambridge MA: MIT press. 2018.
• Zaharchuk G, Gong E, Wintermark M, Rubin D, Langlotz CP. Deep Learning in Neuroradiology. AJNR Am J Neuroradiol. 2018;39(10):1776-84. doi:10.3174/ajnr.A5543
• Nair V, Hinton GE, editors. Rectified linear units improve restricted boltzmann machines. In Icml. 2010.
• Lecun Y, Bengio Y. Convolutional networks for images, speech, and time-series. In Arbib MA, editor, The handbook of brain theory and neural networks. MIT Press. 1995.
• Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. InInternational Conference on Medical image computing and computer-assisted intervention. Springer, Cham. 2015 Oct 5;234-41. doi: 10.1007/978-3-319-24574-4_28
• Milletari, F., Navab, N., Ahmadi, S.A. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 3DV. IEEE. 2016;565–71. doi: 10.1109/3DV.2016.79
• Hochreiter S, Schmidhuber J. LSTM can solve hard long time lag problems. NIPS. 1997;473-9. https://dl.acm.org/doi/10.5555/3104482.3104587
• Chung J, Gulcehre C, Cho K, Bengio YJapa. Empirical evaluation of gated recurrent neural networks on sequence modeling. NIPS deep learning workshop. 2014; arXiv:1412.3555.
• Baldi P. Autoencoders, unsupervised learning, and deep architectures. In Proc. ICML Workshop Unsupervised Transf. Learn. 2012;27:37-50. https://dl.acm.org/doi/10.5555/3045796.3045801
• Salakhutdinov R, Hinton G. Deep boltzmann machines. In Artificial intelligence and statistics. PMLR. 2009 Apr 15;5:448-55. http://proceedings.mlr.press/v5/salakhutdinov09a.html
• Hinton G. Boltzmann Machines. In: Sammut C., Webb G.I. (eds) Encyclopedia of Machine Learning. Springer, Boston, MA. 2011. doi: 10.1007/978-0-387-30164-8_83
• Kazeminia S, Baur C, Kuijper A, et al. GANs for medical image analysis. Artificial Intelligence in Medicine. 2020 Aug 9;101938. doi: 10.1016/j.artmed.2020.101938
• Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine. 2017 Nov 9;34(6):26-38. doi: 10.1109/msp.2017.2743240
• Yeung M, Sala E, Schönlieb CB, Rundo L. Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation. arXiv preprint arXiv:2102.04525. 2021 Feb 8. https://arxiv.org/abs/2102.04525
• Haji SH, Abdulazeez AM. COMPARISON OF OPTIMIZATION TECHNIQUES BASED ON GRADIENT DESCENT ALGORITHM: A REVIEW. PalArch's Journal of Archaeology of Egypt/Egyptology. 2021 Feb 18;18(4):2715-43. https://www.archives.palarch.nl/index.php/jae/article/view/6705
• Abadi M, Barham P, Chen J, et al. Tensorflow: A system for large-scale machine learning. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), USENIX Association. 2016;265-83. https://arxiv.org/abs/1605.08695
• Chollet, F. Keras: Deep learning for humans. GitHub. 2015. Retrieved [2021]. Available from: https://github.com/keras-team/keras
• Paszke, Adam, Sam Gross, et al. Automatic differentiation in pytorch. In NIPS Workshop. 2017. https://openreview.net/forum?id=BJJsrmfCZ
• Jia Y, Shelhamer E, Donahue J, et al. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of ACM Multimedia. 2014;675–8. doi: 10.1145/2647868.2654889
• MATLAB®, version 9.10.0.1613233 (R2021a). The Mathworks, Inc. Natick, MA. 2021. https://www.mathworks.com/
• Clark K, Vendt B, Smith K, et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging. 2013 Dec 1;26(6):1045-57. doi: 10.1007/s10278-013-9622-7
• Tomczak,K., Czerwinska,P. and Wiznerowicz,M. The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge. Contemp Oncol. Pozn. 2015;19(1A):A68–A77. doi: 10.5114/wo.2014.47136
• Liu X, Song L, Liu S, Zhang Y. A review of deep-learning-based medical image segmentation methods. Sustainability. 2021 Jan;13(3):1224. doi: 10.3390/su13031224
• Dalmış MU, Litjens G, Holland K, et al. Using deep learning to segment breast and fibroglandular tissue in MRI volumes. Med. Phys. 2017;44(2):533-46. doi: 10.1002/mp.12079
• Nie D, Wang L, Trullo R, et al. Segmentation of craniomaxillofacial bony structures from MRI with a 3D deep-learning based cascade framework. In: Wang Q., Shi Y., Suk HI., Suzuki K. (eds). MLMI. Lecture Notes in Computer Science. Springer. Cham. 2017;10541:266-73. doi: 10.1007/978-3-319-67389-9_31
• Cheng R, Roth HR, Lay N, et al. Automatic magnetic resonance prostate segmentation by deep learning with holistically nested networks. J. Med. Imaging 2017; 4:041302. doi: 10.1117/1.jmi.4.4.041302
• Avanzo M, Wei L, Stancanello J, et al. Machine and deep learning methods for radiomics. Med. Phys. 2020;47(5): e185-e202. doi: 10.1002/mp.13678
• Gao M, Huang S, Pan X, Liao X, Yang R, Liu J. Machine Learning-Based Radiomics Predicting Tumor Grades and Expression of Multiple Pathologic Biomarkers in Gliomas. Front Oncol. 2020; 10:1676. doi: 10.3389/fonc.2020.01676
• Castaldo R, Cavaliere C, Soricelli A, Salvatore M, Pecchia L, Franzese M. Radiomic and Genomic Machine Learning Method Performance for Prostate Cancer Diagnosis: Systematic Literature Review. J Med Internet Res. 2021;23(4): e22394. doi: 10.2196/22394
• Mitra S. Deep Learning with Radiogenomics towards Personalized Management of Gliomas. IEEE Rev Biomed Eng. Epub ahead of print. 2012; PMID: 33900921. doi: 10.1109/RBME.2021.3075500
• Smedley NF, Aberle DR, Hsu W. Using deep neural networks and interpretability methods to identify gene expression patterns that predict radiomic features and histology in non-small cell lung cancer. J Med Imaging (Bellingham). 2021;8(3):031906. doi: 10.1117/1.jmi.8.3.031906
• Siar M, Teshnehlab M, editors. Brain Tumor Detection Using Deep Neural Network and Machine Learning Algorithm. ICCKE. 2019;363-8. doi: 10.1109/iccke48569.2019.8964846
• Zhen SH, Cheng M, Tao YB, Wang YF, Juengpanich S, Jiang ZY, et al. Deep Learning for Accurate Diagnosis of Liver Tumor Based on Magnetic Resonance Imaging and Clinical Data. Front Oncol. 2020; 10:680. doi: 10.3389/fonc.2020.00680
• Jojoa Acosta MF, Caballero Tovar LY, Garcia-Zapirain MB, Percybrooks WS. Melanoma diagnosis using deep learning techniques on dermatoscopic images. BMC Med Imaging. 2021;21(1):6. doi: 10.1186/s12880-020-00534-8
• Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI. 2017 May; 3462–71. doi: 10.1109/cvpr.2017.
• Halder A, Dey D, Sadhu AK. Lung Nodule Detection from Feature Engineering to Deep Learning in Thoracic CT Images: a Comprehensive Review. J Digit Imaging. 2020;33(3):655-77. doi: 10.1007/s10278-020-00320-6
• Dou Q, Chen H, Yu L, et al. Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks. IEEE Trans Med Imaging. 2016;35(5):1182-95. doi: 10.1109/tmi.2016.2528129
• Yang W, Chen Y, Liu Y, et al. Cascade of multi-scale convolutional neural networks for bone suppression of chest radiographs in gradient domain. Med. Image Anal. 2017; 35:421-33. doi: 10.1016/j.media.2016.08.004
• Xiang L, Qiao Y, Nie D, et al. Deep auto-context convolutional neural networks for standard-dose PET image estimation from low-dose PET/MRI. Neurocomputing. 2017; 267:406-16. doi: 10.1016/j.neucom.2017.06.048. doi: 10.1016/j.neucom.2017.06.048
• Fu J, Yang Y, Singhrao K, et al. Deep learning approaches using 2D and 3D convolutional neural networks for generating male pelvic synthetic computed tomography from magnetic resonance imaging. Med Phys. 2019;46(9):3788-98. doi: 10.1002/mp.13672
• Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep Learning MR Imaging-based Attenuation Correction for PET/MR Imaging. Radiology. 2018;286(2):676-84. doi: 10.1148/radiol.2017170700
• Leynes AP, Yang J, Wiesinger F, et al. Direct PseudoCT generation for pelvis PET/MRI attenuation correction using deep convolutional neural networks with multi-parametric MRI: zero echo-time and dixon deep pseudoCT (ZeDD-CT). J Nucl Med. 2017;59:852–8. doi: 10.2967/jnumed.117.198051
• Choi H, Lee DS. Generation of structural MR images from amyloid PET: application to MR-less quantification. J. Nucl. Med. 2018;59(7):1111-7. doi: 10.2967/jnumed.117.199414
• Ben-Cohen A, Klang E, Raskin S.P, Amitai M.M, Greenspan H. Virtual PET Images from CT Data Using Deep Convolutional Networks: Initial Results. In: Tsaftaris S., Gooya A., Frangi A., Prince J. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2017. Lecture Notes in Computer Science. Springer, Cham. 2017;10557: 49-57. doi: 10.1007/978-3-319-68127-6_6
• Lee JH, Grant BR, Chung JH, Reiser I, Giger ML. Assessment of diagnostic image quality of computed tomography (CT) images of the lung using deep learning. In: Proc. SPIE Medical Imaging. 2018;10573:105731M. doi: 10.1117/12.2292070
• Esses SJ, Lu X, Zhao T, et al. Automated image quality evaluation of T2‐weighted liver MRI utilizing deep learning architecture. J. Magn. Reson. Imaging. 2018;47(3):723-8. doi: 10.1002/jmri.25779
• Schillaci O, Scimeca M, Toschi N, Bonfiglio R, Urbano N, Bonanno E. Combining Diagnostic Imaging and Pathology for Improving Diagnosis and Prognosis of Cancer. Contrast Media & Molecular Imaging. 2019;2019(9429761):1-10. doi: 10.1155/2019/9429761
• Van Dijk LV, Van den Bosch L, Aljabar P, et al. Improving automatic delineation for head and neck organs at risk by Deep Learning Contouring. Radiother. Oncol. 2020 Jan 1; 142:115-23. doi: 10.1016/j.radonc.2019.09.022
• Wang M, Zhang Q, Lam S, Cai J, Yang R. A Review on Application of Deep Learning Algorithms in External Beam Radiotherapy Automated Treatment Planning. Front Oncol. 2020;10:580919. doi: 10.3389/fonc.2020.580919
• Shen C, Nguyen D, Chen, et al. Operating a treatment planning system using a deep-reinforcement learning-based virtual treatment planner for prostate cancer intensity-modulated radiation therapy treatment planning. Med. Phys. 2020;47(6):2329-36. doi: 10.1002/mp.14114
• Moreau G, Francois-Lavet V, Desbordes P, Macq B. Reinforcement Learning for Radiotherapy Dose Fractioning Automation. Biomedicines. 2021 Feb;9(2):214. doi:10.3390/biomedicines9020214
• Kiser KJ, Fuller CD, Reed VK. Artificial intelligence in radiation oncology treatment planning: a brief overview. J Med Arti Intell. 2019;2:9. doi: 10.21037/jmai.2019.04.02
• Zhen X, Chen J, Zhong Z, Hrycushko B, Zhou L, Jiang S, et al. Deep convolutional neural network with transfer learning for rectum toxicity prediction in cervical cancer radiotherapy: a feasibility study. Physics in Medicine & Biology. 2017;62(21):8246. doi: 10.1088/1361-6560/aa8d09
• Kajikawa T, Kadoya N, Ito K, et al. Automated prediction of dosimetric eligibility of patients with prostate cancer undergoing intensity-modulated radiation therapy using a convolutional neural network. Radiol Phys Technol. 2018;11(3):320-7. doi: 10.1007/s12194-018-0472-3
• Kalet AM, Luk SM, Phillips MHJMp. Radiation therapy quality assurance tasks and tools: the many roles of machine learning. Med. Phys. 2020;47(5): e168-e77. doi: 10.1002/mp.13445
• Luo Y, Chen S, Valdes GJMp. Machine learning for radiation outcome modeling and prediction. Med. Phys. 2020;47(5): e178-e84. doi: 10.1002/mp.13570
• Huynh BQ, Antropova N, Giger ML. Comparison of breast DCE-MRI contrast time points for predicting response to neoadjuvant chemotherapy using deep convolutional neural network features with transfer learning. In: Armato SG, Petrick NA, eds. Proc. SPIE Medical Imaging. 2017;Vol. 10134:101340U. doi: 10.1117/12.2255316
• Cha KH, Hadjiiski L, Chan H-P, et al. Bladder cancer treatment response assessment in CT using radiomics with deep-learning. Sci. Rep. 2017;7(1):1-12. doi: 10.1038/s41598-017-09315-w
• Belkin M, Hsu D, Ma S, Mandal S. Reconciling modern machine-learning practice and the classical bias–variance trade-off. PNAS. 2019 Aug 6;116(32):15849-54. doi: 10.1073/pnas.1903070116
• Affane A, Lebre MA, Mittal U, Vacavant A. Literature Review of Deep Learning Models for Liver Vessels Reconstruction. In: IPTA. IEEE. 2020 Nov 9;1-6. doi: 10.1109/ipta50016.2020.9286639