Performance Analysis of Efficient Deep Learning Models for Multi-Label Classification of Fundus Image

Abstract

​​Convolutional Neural Networks (CNNs) have demonstrated significant advancements in the domain of fundus images owing to their exceptional capability to learn meaningful features. By appropriately processing and analyzing fundus images, computer-aided diagnosis systems can furnish healthcare practitioners with valuable reference information for clinical diagnosis or screening purposes. Nevertheless, prior investigations have predominantly concentrated on detecting individual fundus diseases, while the simultaneous diagnosis of multiple fundus diseases continues to pose substantial challenges. Furthermore, the majority of previous studies have prioritized diagnostic accuracy as their main focus. Efficient Deep Learning constitutes a crucial concept that enables the utilization of deep learning models on edge devices, thereby reducing the computational carbon footprint. Facilitating the cost-effective diagnosis of eye diseases from fundus images on edge devices holds significance for researchers aiming to deploy these vital healthcare models into practical use. This study focuses on assessing the performance of well-known efficient deep learning models in addressing the multi-label classification problem of fundus images. The models underwent training and testing using the dataset provided by ODIR 2019 (Peking University International Competition on Ocular Disease Intelligent Recognition). The experimental findings demonstrate that the efficientnetb3 model outperforms the other models, exhibiting the highest level of performance. And also, when applying standard data augmentation techniques to the current dataset, we observe decreasing in f1-score and accuracy.

Keywords

• artificial intelligence
• deep learning
• efficient deep learning
• multi-label classification
• eye disease diagnosis
• fundus images

References

1. [1] J. B. Jonas, R. R. A. Bourne, R. A. White, S. R. Flaxman, J. Keeffe, J. Leasher, K. Naidoo, K. Pesudovs, H. Price, T. Y. Wong, S. Resnikoff, and H. R. Taylor, ‘‘Visual impairment and blindness due to macular diseases globally: A systematic review and meta-analysis,’’ Amer. J. Ophthalmol., vol. 158, no. 4, pp. 808–815, Oct. 2014.
2. [2] Liu, Y. C., Wilkins, M., Kim, T., Malyugin, B., & Mehta, J. S. (2017). Cataracts. The Lancet, 390(10094), 600-612.
3. [3] J. L. Leasher, R. R. A. Bourne, S. R. Flaxman, J. B. Jonas, J. Keeffe, K. Naidoo, K. Pesudovs, H. Price, R. A. White, T. Y. Wong, S. Resnikoff, and H. R. Taylor, ‘‘Global estimates on the number of people blind or visually impaired by diabetic retinopathy: A meta-analysis from 1990 to 2010,’’ Diabetes Care, vol. 39, pp. 1643–1649, Sep. 2016, doi: 10.2337/dc15- 2171
4. [4] O. B. Walton, R. B. Garoon, C. Y. Weng, J. Gross, A. K. Young, K. A. Camero, H. Jin, P. E. Carvounis, R. E. Coffee, and Y. I. Chu, ‘‘Evaluation of automated teleretinal screening program for diabetic retinopathy,’’ JAMA Ophthalmol., vol. 134, no. 2, pp. 204–209, Feb. 2016, doi: 10.1001/jamaophthalmol.2015.5083.
5. [5] H. Ye, Q. Zhang, X. Liu, X. Cai, W. Yu, S. Yu, T. Wang, W. Lu, X. Li, H. Jin, Y. Hu, X. Kang, and P. Zhao, ‘‘Prevalence of age-related macular degeneration in an elderly urban Chinese population in China: The Jiangning eye study,’’ Investigative Ophthalmol. Vis. Sci., vol. 55, no. 10, pp. 6374–6380, Sep. 2014, doi: 10.1167/iovs.14-14899.
6. [6] Tham, Y. C., Goh, J. H. L., Anees, A., Lei, X., Rim, T. H., Chee, M. L., ... & Cheng, C. Y. (2022). Detecting visually significant cataract using retinal photograph-based deep learning. Nature Aging, 2(3), 264-271.
7. [7] Tan, M., & Le, Q. (2019, May). Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning (pp. 6105-6114). PMLR.
8. [8] Wang, J., Yang, L., Huo, Z., He, W., & Luo, J. (2020). Multi-label classification of fundus images with efficientnet. IEEE Access, 8, 212499-212508.

9. [9] Zhou, Y., Wang, B., Huang, L., Cui, S., & Shao, L. (2020). A benchmark for studying diabetic retinopathy: segmentation, grading, and transferability. IEEE Transactions on Medical Imaging, 40(3), 818-828.
10. [10] He, J., Li, C., Ye, J., Wang, S., Qiao, Y., & Gu, L. (2020, April). Classification of ocular diseases employing attention-based unilateral and bilateral feature weighting and fusion. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) (pp. 1258-1261). IEEE.
11. [11] Bhati, A., Gour, N., Khanna, P., & Ojha, A. (2023). Discriminative kernel convolution network for multi-label ophthalmic disease detection on imbalanced fundus image dataset. Computers in Biology and Medicine, 106519.
12. [12] Wang, K., Xu, C., Li, G., Zhang, Y., Zheng, Y., & Sun, C. (2023). Combining convolutional neural networks and self-attention for fundus diseases identification. Scientific Reports, 13(1), 76.
13. [13] Peking University International Competition on Ocular Disease Intelligent Recognition (ODIR-2019), [online] Available: https://odir2019.grand-challenge.org/dataset/.
14. [14] Kaggle. “Ocular Disease Recognition”, Access Date: 11.06.2023. https://www.kaggle.com/datasets/andrewmvd/ocular-disease-recognition-odir5k
15. [15] Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
16. [16] Google Research. “EfficientNet: Improving Accuracy and Efficiency through AutoML and Model Scaling”, Access Date: 11.06.2023. https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html
17. [17] Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360.
18. [18] Pytorch. “AutoAugment”, Access Date: 11.06.2023. https://pytorch.org/vision/master/generated/torchvision.transforms.AutoAugment.html#torchvision.transforms.AutoAugment