Comparing Popular CNN Models for an Imbalanced Dataset of Dermoscopic Images

Abstract

In this study, the performance of popular convolution architectures against an imbalanced dataset is analyzed in detail with a multi-classing medical image processing application. Our selection for dermoscopic images is a large-scale and imbalanced dataset consisting of 10,015 colored lesion images belonging to 7 different skin diseases, was used as a benchmark. Images without pathological testing are labeled by specialist dermatologists who are members of International Skin Imaging Association. The f1-score was preferred as the measurement metric during the training phase of the convolution networks, which were trained with imbalanced dataset, and the area under the receiver operating characteristic curve and the confusion matrix of each model were calculated at the test phase. In the validation phase of convolution networks, k-fold cross validation technique was used. In addition, the filters obtained from the ImageNet dataset have been imported with the Transfer-Learning option. Fine-tuning was applied to the deepest convolution layers in order for these pre-trained models to develop themselves specific to our application. In order to prevent the overfit problem, the feature extraction outputs of the models were drop-out at a rate of 50% after flattening, and L2-regularization (weigh decay) was applied during the update phase of the weights. Although it is not the main purpose of the study, in order to partially improve the performance of convolution architectures, synthetic lesion images created with data-augmentation for the minor classes in the imbalanced dataset were included in the training process in a way that does not cause information leakage.

Keywords

• Popular CNN Models
• Dermoscopic Images
• Skin Diseases Diagnosis
• Imbalanced Dataset

Dengeli Dağılım Sergilemeyen Dermoskopik Görüntü Veri Seti Üzerinde CNN Modelleri Karşılaştırma

Abstract

Bu çalışmada, dengeli dağılım sergilemeyen bir veri seti karşısında popüler evrişimsel sinir ağı mimarilerinin nasıl bir performans sergileyebilecekleri çok-sınıflı bir medikal görüntü işleme uygulaması ile detaylı bir şekilde analiz edilmiştir. 7 farklı deri hastalığına ait 10.015 tane renkli lezyon resimlerinden oluşan büyük ölçekli ve dengesiz bir veri seti olan HAM10000(İnsan-Makineye Karşı-10000), test aracı olarak kullanılmıştır. Bu veri seti ile eğitilip performans karşılaştırması yapılan evrişimsel sinir ağı modellerinin eğitim aşamasında ölçüm metriği olarak f1-score değerleri ve test aşamasında ise her bir modelin hem hata matrisi hem de ROC eğrileri altında kalan AUC alanları hesaplanmıştır. Evrişimsel sinir ağı modellerinin eğitim süreçleri, k-fold çapraz doğrulama ile analiz edilmiştir. Ayrıca imagenet veri setindeki filtreler, Eğitim-Transferi seçeneği ile içe aktarılmıştır. Ön-eğitimli bu modellerin uygulamaya özgü kendilerini geliştirebilmeleri için ise en derindeki evrişim katmanlarına ince-ayar işlemi uygulanmıştır. Ezber yapma problemini engellemek amacı ile modellerin çıkardıkları özellikler, tek sütunlu vektör haline getirildikten sonra 50% oranında silme işlemi ve ağırlıkların güncelleme aşamasında ise L2-düzenleme(weigh decay) işlemi uygulanmıştır. Çalışmanın asıl amacı olmamak ile birlikte evrişim mimarilerin performanslarını kısmen de olsa iyileştirebilmek için HAM10000 sınıfındaki azınlık sınıfları için veri çeşitlendirme ile oluşturulan sentetik lezyon görüntüleri, bilgi sızıntısına neden olmayacak şekilde eğitim sürecine dâhil edilmiştir.

Keywords

• Popüler CNN Modelleri
• Dermoskopik Görüntüler
• Cilt Hastalıkları Teşhisi
• Dengeli Dağılım Sergilemeyen Veri Seti

References

1. Barata, C., Celebi, M. E., & Marques, J. S. (2019, 5). A Survey of Feature Extraction in Dermoscopy Image Analysis of Skin Cancer. IEEE Journal of Biomedical and Health Informatics, 23, 1096–1109. doi:10.1109/jbhi.2018.2845939
2. Bisla, D., Choromanska, A., Berman, R. S., Stein, J. A., & Polsky, D. (2019). Towards Automated Melanoma Detection With Deep Learning: Data Purification and Augmentation. (pp. 2720–2728). Long Beach, CA, USA: IEEE. doi:10.1109/CVPRW.2019.00330
3. Brinker, T. J., Hekler, A., Enk, A. H., Klode, J., Hauschild, A., Berking, C., . . . Schrüfer, P. (2019, 4). A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. European Journal of Cancer, 111, 148–154. doi:10.1016/j.ejca.2019.02.005
4. Brinker, T. J., Hekler, A., Enk, A. H., Klode, J., Hauschild, A., Berking, C., . . . Schrüfer, P. (2019, 5). Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. European Journal of Cancer, 113, 47–54. doi:10.1016/j.ejca.2019.04.001 Chollet, F. (2016, 10). Xception: Deep Learning with Depthwise Separable Convolutions.
5. Codella, N., Rotemberg, V., Tschandl, P., Celebi, M. E., Dusza, S., Gutman, D., . . . Halpern, A. (2019, 2). Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC).
6. Do, T. T., Hoang, T., Pomponiu, V., Zhou, Y., Chen, Z., Cheung, N. M., . . . Tan, S. H. (2017, 11). Accessible Melanoma Detection using Smartphones and Mobile Image Analysis.
7. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017, 1). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542, 115–118. doi:10.1038/nature21056
8. He, K., Zhang, X., Ren, S., & Sun, J. (2015, 12). Deep Residual Learning for Image Recognition.

9. He, K., Zhang, X., Ren, S., & Sun, J. (2016, 3). Identity Mappings in Deep Residual Networks.
10. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., . . . Adam, H. (2017, 4). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications.
11. Huang, G., Liu, Z., van der Maaten, L., & Weinberger, K. Q. (2016, 8). Densely Connected Convolutional Networks.
12. Ioffe, S., & Szegedy, C. (2015, 2). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.
13. Kassani, S. H., & Kassani, P. H. (2019, 6). A comparative study of deep learning architectures on melanoma detection. Tissue and Cell, 58, 76–83. doi:10.1016/j.tice.2019.04.009
14. Li, Y., & Shen, L. (2018, 2). Skin Lesion Analysis towards Melanoma Detection Using Deep Learning Network. Sensors, 18, 556. doi:10.3390/s18020556
15. Majtner, T., Yildirim-Yayilgan, S., & Hardeberg, J. Y. (2018, 10). Optimised deep learning features for improved melanoma detection. Multimedia Tools and Applications, 78, 11883–11903. doi:10.1007/s11042-018-6734-6
16. Mishra, N. K., & Celebi, M. E. (2016, 1). An Overview of Melanoma Detection in Dermoscopy Images Using Image Processing and Machine Learning.
17. Okuboyejo, D. A., & Olugbara, O. O. (2018, 4). A Review of Prevalent Methods for Automatic Skin Lesion Diagnosis. The Open Dermatology Journal, 12, 14–53. doi:10.2174/187437220181201014
18. Rezvantalab, A., Safigholi, H., & Karimijeshni, S. (2018, 10). Dermatologist Level Dermoscopy Skin Cancer Classification Using Different Deep Learning Convolutional Neural Networks Algorithms.
19. Rosendahl, C., McColl, A. C., & Wilkinson, D. (2012, 7). Dermatoscopy in routine practice - 'chaos and clues'. australian family physician, 41(7), 482-487.
20. Salido, J. A., & Jr., C. R. (2018, 2). Using Deep Learning for Melanoma Detection in Dermoscopy Images. International Journal of Machine Learning and Computing, 8, 61–68. doi:10.18178/ijmlc.2018.8.1.664
21. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018, 1). MobileNetV2: Inverted Residuals and Linear Bottlenecks. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4510-4520.
22. Simonyan, K., & Zisserman, A. (2014, 9). Very Deep Convolutional Networks for Large-Scale Image Recognition.
23. Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2016, 2). Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning.
24. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., . . . Rabinovich, A. (2014, 9). Going Deeper with Convolutions.
25. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2015, 12). Rethinking the Inception Architecture for Computer Vision.
26. Tan, M., & Le, Q. V. (2019, 5). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. International Conference on Machine Learning, 2019.
27. Tschandl, P., Rosendahl, C., & Kittler, H. (2018, 3). The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 (2018). doi:10.1038/sdata.2018.161
28. Vasconcelos, C. N., & Vasconcelos, B. N. (2017, 2). Convolutional Neural Network Committees for Melanoma Classification with Classical And Expert Knowledge Based Image Transforms Data Augmentation.
29. Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2017, 7). Learning Transferable Architectures for Scalable Image Recognition.