Classification of Tympanic Membrane Images based on VGG16 Model

Abstract

Otitis Media (OM) is a type of infectious disease caused by viruses and/or bacteria in the middle ear cavity. In the current study, it is aimed to detect the eardrum region in middle ear images for diagnosing OM disease by using artificial intelligence methods. The Convolution Neural Networks (CNN) model and the deep features of this model and the images obtained with the otoscope device were used. In order to separate these images as Normal and Abnormal, the end-to-end VGG16 model was directly used in the first stage of the experimental work. In the second stage of the experimental study, the activation maps of the fc6 and fc7 layers consisting of 4096 features and the fc8 layer consisting of 1000 features of the VGG16 CNN model were obtained. Then, it was given as input to Support Vector Machines (SVM). Then, the deep features obtained from all activation maps were combined and a new feature set was obtained. In the last stage, this feature set is given as an input to SVM. Thus, the effect of the VGG16 model and the features obtained from the layers of this model on the success of distinguishing images of the eardrum was investigated. Experimental studies show that, the best performance results were obtained for the fc6 layer with an accuracy rate of 82.17%. In addition, 71.43%, 90.62% and 77.92% performance criteria were obtained for sensitivity, specificity and f-score values, respectively. Consequently, it has been shown that OM disease could be accurately detected by using a deep CNN architecture. The proposed deep learning-based classification system promises highly accurate results for disease detection.

Keywords

• Convolution Neural Networks
• VGG16
• Tympanic Membrane
• Otitis Media
• Classification
• Support Vector Machines

References

1. [1] Chittka L., Brockmann, A., 2005. Perception space the final frontier. PLoS biology, 3(4), pp. 564-568.
2. [2] Wu Z., Lin Z., Li L., Pan H., Chen G., Fu Y., Qiu Q., 2021. Deep learning for classification of pediatric otitis media. The Laryngoscope, 131(7), E2344-E2351.
3. [3] Cetinkaya E. A., Topsakal V., 2022. Acute Otitis Media. In Pediatric ENT Infections, Springer, Cham, pp. 381-392.
4. [4] Manju K., Paramasivam M. E., Nagarjun S., Mokesh A., Abishek A., Meialagan, K., 2022. Deep Learning Algorithm for Identification of Ear Disease. In Proceedings of International Conference on Data Science and Applications, Springer, Singapore, pp. 491-502.
5. [5] Shie C. K., Chang H. T., Fan F. C., Chen C. J., Fang T. Y., Wang P. C., 2014. A hybrid feature-based segmentation and classification system for the computer aided self-diagnosis of otitis media. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, pp. 4655-4658.
6. [6] Cheng L., Liu J., Roehm C. E., Valdez T. A., 2011. Enhanced video images for tympanic membrane characterization. In 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, pp. 4002-4005.
7. [7] Kuruvilla A., Li J., Yeomans P. H., Quelhas P., Shaikh N., Hoberman A., Kovačević J., 2012. Otitis media vocabulary and grammar. In 2012 19th IEEE International Conference on Image Processing, IEEE, pp. 2845-2848.
8. [8] Kuruvilla A., Shaikh N., Hoberman A., Kovačević J., 2013. Automated diagnosis of otitis media: vocabulary and grammar. International Journal of Biomedical Imaging.

9. [9] Başaran E., Şengür A., Cömert Z., Budak Ü., Çelik Y., Velappan S., 2019. Normal and acute tympanic membrane diagnosis based on gray level co-occurrence matrix and artificial neural networks. In 2019 international artificial intelligence and data processing symposium (IDAP), IEEE, pp. 1-6.
10. [10] Cai Y., Yu J. G., Chen Y., Liu C., Xiao L., Grais E. M., Zhao F., Lan L., Zeng S., Zeng J., Wu M., Su Y., Li Y., Zheng Y., 2021. Investigating the use of a two-stage attention-aware convolutional neural network for the automated diagnosis of otitis media from tympanic membrane images: a prediction model development and validation study. BMJ open, 11(1), e041139.
11. [11] Albashish, D., Al-Sayyed, R., Abdullah, A., Ryalat, M. H., Almansour, N. A., 2021. Deep CNN model based on VGG16 for breast cancer classification. In 2021 International Conference on Information Technology (ICIT), IEEE, pp. 805-810.
12. [12] Tripathi, S., Verma, A., Sharma, N., 2021. Automatic segmentation of brain tumour in MR images using an enhanced deep learning approach. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 9(2), pp. 121-130.
13. [13] Başaran E., Cömert Z., Çelik Y., 2020. Convolutional neural network approach for automatic tympanic membrane detection and classification. Biomedical Signal Processing and Control, 56, 101734.
14. [14] Rehman, A., Naz, S., Razzak, M. I., Akram, F., Imran, M., 2020. A deep learning-based framework for automatic brain tumors classification using transfer learning. Circuits, Systems, and Signal Processing, 39(2), pp. 757-775.
15. [15] Zafer C., 2020. Fusing fine-tuned deep features for recognizing different tympanic membranes. Biocybernetics and Biomedical Engineering, 40(1), pp. 40-51.
16. [16] Hiremani V. A., Senapati K. K., 2021. Quantifying apt of RNN and CNN in Image Classification. In Proceeding of Fifth International Conference on Microelectronics, Computing and Communication Systems, Springer, Singapore, pp. 721-733.
17. [17] Singh S. P., Wang L., Gupta S., Goli H., Padmanabhan P., Gulyás B., 2020. 3D deep learning on medical images: a review. Sensors, 20(18), 5097.
18. [18] Li Y., Sixou B., Peyrin F., 2021. A review of the deep learning methods for medical images super resolution problems, IRBM, 42(2), pp. 120-133.
19. [19] Kattenborn T., Leitloff J., Schiefer F., Hinz S., 2021. Review on Convolutional Neural Networks (CNN) in vegetation remote sensing. ISPRS Journal of Photogrammetry and Remote Sensing, 173, pp. 24-49.
20. [20] Lee J. Y., Choi S. H., Chung J. W., 2019. Automated classification of the tympanic membrane using a convolutional neural network, Applied Sciences, 9(9), 1827.
21. [21] Tripathi, M., 2021. Analysis of convolutional neural network based image classification techniques. Journal of Innovative Image Processing (JIIP), 3(02), pp. 100-117.
22. [22] Dhillon A., Verma G. K., 2020. Convolutional neural network: a review of models, methodologies and applications to object detection. Progress in Artificial Intelligence, 9(2), pp. 85-112.
23. [23] Yao G., Lei T., Zhong J., 2019. A review of convolutional-neural-network-based action recognition. Pattern Recognition Letters, 118, pp. 14-22.
24. [24] Nalepa J., Kawulok M., 2019. Selecting training sets for support vector machines: a review. Artificial Intelligence Review, 52(2), pp. 857-900.
25. [25] Uçar M., Akyol K., Atila Ü., Uçar E., 2021. Classification of different tympanic membrane conditions using fused deep hypercolumn features and bidirectional LSTM. IRBM.
26. [26] Wang Z., Cha Y. J., 2021. Unsupervised deep learning approach using a deep auto-encoder with a one-class support vector machine to detect damage. Structural Health Monitoring, 20(1), pp. 406-425.
27. [27] Alhudhaif A., Cömert Z., Polat K., 2021. Otitis media detection using tympanic membrane images with a novel multi-class machine learning algorithm. PeerJ Computer Science, 7, e405.