Classification of Breast Cancer Images Using Ensembles of Transfer Learning

Abstract

It is a challenging task to estimate the cancerous cells and tissues via computer-aided diagnosis systems on high-resolution histopathological images. In this study, it is suggested to use transfer learning and ensemble learning methods together in order to reduce the difficulty of this task and better diagnose cancer patients. In the studies, histopathological images with 40× and 100× magnification factors are analyzed. In order to prove the success of the study with experimental studies, firstly, the results provided by pre-modeled deep learning architectures trained by histopathological image dataset, then the results acquired by different transfer learning approaches and the results obtained with the ensembles of deeply learned features using transfer learning methods are presented comparatively. Three different approaches are applied for transfer learning by fine-tuning the pre-trained convolution neural networks. In the experimental section, results of single classifiers (i.e., support vector machines, logistic regression, k-nearest neighbor and bagging) are presented by employing features of CNN models obtained by defined transfer learning approaches. Then, decisions of each classifier model are combined separately by weighted decision fusion (WDF) and stacking decision fusion (SDF) ensemble learning methods that have proven to improve the classification performance of the proposed classification system.

Keywords

• Histopathological images
• breast cancer
• deep learning
• transfer learning
• ensemble learning

References

1. F. Bray, J. Ferlay, I. Soerjomataram, R. L. Siegel, L. A. Torre, and A. Jemal, “Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries,” CA: A cancer Journal for Clinicians, vol. 68, no. 6, pp. 394–424, 2018.
2. A. Hamidinekoo, E. Denton, A. Rampun, K. Honnor, and R. Zwiggelaar, “Deep learning in mammography and breast histology, An overview and future trends,” Medical Image Analysis, vol. 47, pp. 45-67, 2018.
3. M. N. Gurcan, L. E. Boucheron, A. Can, A. Madabhushi, N. M. Rajpoot, and B. Yener, “Histopathological image analysis: A review,” IEEE Reviews in Biomedical Engineering, vol. 2, pp. 147-171, 2009.
4. N. Coudray, P. S. Ocampo, T. Sakellaropoulos, N. Narula, M. Snuderl, D. Fenyö, and A. Tsirigos, “Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning,” Nature Medicine, vol. 24, no. 10, pp. 1559-1567, 2018.
5. K. Sirinukunwattana, S. E. A. Raza, Y. W. Tsang, D. R. Snead, I. A. Cree, and N. M. Rajpoot, “Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1196-1206, 2016.
6. T. Majtner, S. Yildirim-Yayilgan, and J. Y. Hardeberg, “Combining deep learning and hand-crafted features for skin lesion classification,” IEEE 6th International Conference on Image Processing Theory, Tools and Applications, IPTA’16, pp. 1-6, 2016.
7. N. Hatipoglu and G. Bilgin, “Cell segmentation in histopathological images with deep learning algorithms by utilizing spatial relationships,” Medical & Biological Engineering & Computing, vol. 55, no. 10, pp. 1829-1848, 2017.
8. K. Guzel and G. Bilgin, “Textural feature extraction and ensemble of extreme learning machines for hyperspectral image classification,” IEEE 26th Signal Processing and Communications Applications Conference, SIU’18, pp. 1-4, 2018.

9. N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” International Conference on Computer Vision and Pattern Recognition, CVPR’15, vol. 2, pp. 886–893, 2005.
10. D. A. Clausi and M. E. Jernigan, “Designing Gabor filters for optimal texture separability,” Pattern Recognation, vol. 33, no. 11, pp. 1835–1849, 2000.
11. A. Abbas, T. Zehra, and F. Li, “Object detection in a cluttered scene using SURF for computer assisted histopathology,” International Conference on Electrical, Mechanical and Industrial Engineering, Atlantis Press, 2016.
12. A. Albayrak and G. Bilgin, “Automatic cell segmentation in histopathological images via two-staged superpixel-based algorithms,” Medical & Biological Engineering & Computing, vol. 57, no. 3, pp. 653–665, 2019.
13. A. F. Spanhol et al., “Deep features for breast cancer histopathological image classification,” IEEE International Conference on Systems, Man, and Cybernetics, SMC’17, pp. 1868-1873, 2017.
14. K. Guzel and G. Bilgin, “Classification of nuclei in colon cancer images using ensemble of deep learned features,” IEEE Medical Technologies Congress, TıpTekno’19, pp. 1–4, 2019.
15. Y. S. Vang, Z. Chen, and X. Xie, “Deep learning framework for multi-class breast cancer histology image classification,” International Conference Image Analysis and Recognition, ICIAR’18, pp. 914-922, 2018.
16. Z. Yang, L. Ran, S. Zhang, Y. Xia, and Y. Zhang, “EMS-Net: Ensemble of multiscale convolutional neural networks for classification of breast cancer histology images,” Neurocomputing, pp. 46-53, 2019.
17. F. Ponzio, E. Macii, E. Ficarra and S. Di Cataldo, “Going deeper into colorectal cancer histopathology,” International Joint Conference on Biomedical Engineering Systems and Technologies, BIOSTEC’18, pp. 114-131, 2018.
18. L. Nanni, S. Ghidoni, and S. Brahnam, “Ensemble of convolutional neural network for bioimage classification,” Applied Computing and Informatics, 2018.
19. M. Talo, “Automated classification of histopathology images using transfer learning,” Artificial Intelligence in Medicine, vol. 101, no. 101743, 2019.
20. C. Li et al., “Cervical histopathology image classification using ensembled transfer learning,” International Conference on Information Technologies in Biomedicine, ITIB’19, pp. 26-37, Springer, Cham, 2019.
21. B. Harangi, “Skin lesion classification with ensembles of deep convolutional neural networks,” Journal of Biomedical Informatics, vol. 86, pp. 25–32, 2018.
22. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
23. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” IEEE Conference on Computer Vision and Pattern Recognition, CVPR’16, pp. 770–778, 2016.
24. L. Liu, C. Shen, and A. van den Hengel, “The treasure beneath convolutional layers: Cross-convolutional-layer pooling for image classification,” IEEE Conference on Computer Vision and Pattern Recognition, CVPR’15, pp. 4749–4757, 2015.
25. L. I. Kuncheva and J. J. Rodríguez, “A weighted voting framework for classifiers ensembles,” Knowledge and Information Systems, vol. 38, no. 2, pp. 259–275, 2014.
26. J. Sill, G. Takács, L. Mackey, and D. Lin, “Feature-weighted linear stacking,” arXiv preprint arXiv:0911.0460, 2009.
27. F. A. Spanhol, L. S. Oliveira, C. Petitjean, and L. Heutte, “A dataset for breast cancer histopathological image classification,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 7, pp. 1455–1462, 2015.
28. T. L. Sellaro, R. Filkins, C. Hoffman, J. L. Fine, J. Ho, A. V. Parwani, and M. Montalto, “Relationship between magnification and resolution in digital pathology systems,” Journal of Pathology Informatics, vol. 4, 2013.
29. M. L. Giger and K. Suzuki, “Computer- aided diagnosis,” Biomedical Information Technology, Academic Press, pp. 359-XXII, 2008.
30. F. Chollet, “Keras: The python deep learning library,” Astrophysics Source Code Library, 2018.
31. F. Pedregosa, et al. “Scikit-learn: Machine learning in Python,” Journal of machine learning research, vol.12, pp. 2825-2830, 2011.
32. M. Abadi, et al. “Tensorflow: A system for large-scale machine learning,” 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI’16, pp. 265-283, 2016.
33. N. Bayramoglu, J. Kannala, and J. Heikkilä, “Deep learning for magnification independent breast cancer histopathology image classification,” IEEE 23rd International Conference on Pattern Recognition, ICPR’16, pp. 2440–2445, 2016.
34. S. H. Kassani, P. H. Wesolowski, M. J. Schneider, and K. A. Deters, “Classification of histopathological biopsy images using ensemble of deep learning networks,” arXiv preprint arXiv:1909.11870, 2019.
35. D. M. Vo, N. Q. Nguyen, and S. W. Lee, “Classification of breast cancer histology images using incremental boosting convolution networks,” Information Sciences, vol. 482, pp. 123-138, 2019.
36. M. Z. Alom, C. Yakopcic, M. S. Nasrin, T. M. Taha, and V. K. Asari, “Breast cancer classification from histopathological images with inception recurrent residual convolutional neural network,” Journal of Digital Imaging, vol. 32, no. 4, pp. 605-617, 2019.
37. F. A. Spanhol, L. S. Oliveira, C. Petitjean, and L. Heutte, “Breast cancer histopathological image classification using convolutional neural networks,” IEEE International Joint Conference on Neural Networks, IJCNN’16, pp. 2560-2567, 2016.
38. Z. Han, B. Wei, Y. Zheng, Y. Yin, K. Li, and S. Li, “Breast cancer multi-classification from histopathological images with structured deep learning model,” Scientific Reports, vol. 7, no. 1, pp. 1-10, 2017.
39. Z. Gandomkar, P. C. Brennan, and C. Mello-Thoms, “MuDeRN: Multi-category classification of breast histopathological image using deep residual networks,” Artificial Intelligence in Medicine, vol. 88, pp. 14-24, 2018.
40. R. Mehra, “Breast cancer histology images classification: Training from scratch or transfer learning?,” ICT Express, vol. 4, no. 4, pp. 247-254, 2018.
41. C. Zhu, F. Song, Y. Wang, H. Dong, Y. Guo, and J. Liu, “Breast cancer histopathology image classification through assembling multiple compact CNNs,” BMC Medical Informatics and Decision Making, vol. 19, no. 1, pp. 198, 2019.
42. A. Kumar, S. K. Singh, S. Saxena, K. Lakshmanan, A. K. Sangaiah, H. Chauhan, and R. K. Singh, “Deep feature learning for histopathological image classification of canine mammary tumors and human breast cancer,” Information Sciences, vol. 508, pp. 405-421, 2020.