Multi-species Fish Identification using Hybrid DeepCNN with Refined Squeeze and Excitation Architecture

Year 2022, Volume: 37 Issue: 4, 220 - 228, 20.10.2022

Jansi Rani Sella Veluswami Nivetha P

https://doi.org/10.26650/ASE202221163202

Abstract

Fish play a prominent role in the food web and fish farming has value for both human consumption and tourist attractions. Due to the increasing importance of marine biodiversity, recognition of fish species has become a prominent task in monitoring the mislabelling of seafood and extinct species. This problem can be solved using traditional manual annotation on the images. To reduce manpow-er, cost, and tremendous time, deep learning approaches are used which always require large data-sets. Therefore, fish species identification is a challenging task using disproportionately small data sets. In this research, we develop a new method by refining the squeeze and excitation network for the automatic fish species classification model to identify 23 different types of fish species. To achieve this, a hybrid framework using deep learning is proposed on a large-scale dataset and implemented transfer learning for a small-scale dataset. Deep learning methods can be used to identify fish in un-derwater images. In this study, we have proposed a new method of hybrid Deep Convolutional Neu-ral Network (CNN) along with a Support Vector Machine (SVM) for classification. Additionally, the Squeeze and Excitation (SE) block has been improved for improved feature extraction. The proposed method achieved an accuracy of 97.90%. Then post-training with the small-scale dataset (Croatian) achieved an accuracy of 94.99% with an 11% improvement compared to Bilinear CNN (B-CNN) (Qui et al., 2018) and can be used in any underwater applications to identify fish species and avoid misla-belling of seafood.

Keywords

Deep Learning, Convolutional Neural Network, Squeeze and Excitation, Fish species, Fish4knowledge dataset

References

• Allken, V., Handegard, N. O., Rosen, S., Schreyeck, T., Mahiout, T., & Malde, K. (2019). Fish species identification using a convolutional neural network trained on synthetic data. ICES Journal of Marine Science, 76(1), 342-349. google scholar
• B. B. B. Phoenix X. Huang and R. B. Fishera.Fish4KnowledgeDataset: https://homepages.inf.ed.ac.uk/rbf/Fish4Knowledge/ GROUNDTRUTH/RECOG/. 2013. google scholar
• Chen, P. Y., Ho, C. W., Chen, A. C., Huang, C. Y., Liu, T. Y., & Liang, K. H. (2020). Inves-tigating seafood substitution problems and consequences in Taiwan using molecular barcoding and deep microbiome profiling. Scientific reports, 10(1), 1-9. google scholar
• Crozier, L. G., McClure, M. M., Beechie, T., Bograd, S. J., Boughton, D. A., Carr, M.,... & Willis-Norton, E. (2019). Climate vulnerability assessment for Pacific salmon and steelhead in the California Current Large Marine Ecosystem. PloS one, 14(7), e0217711. google scholar
• Dagoudo, M., Qiang, J., & Solevo, M. P. (2022). Status in science and technology devel- opments in Benin’s aquaculture industry: a review. Aquaculture International, 1-15. google scholar Data Science Glossary. https://c3.ai/glossary/data-science/. google scholar
• Deep, B. V., & Dash, R. (2019, March). Underwater fish species recognition using deep learning techniques. In 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN) (pp. 665-669). IEEE. google scholar
• Dos Santos, A. A., & Goncalves, W. N. (2019). Improving Pantanal fish species recogni- tion through taxonomic ranks in convolutional neural networks. Ecological Informatics, 53, 100977. google scholar
• Du, J., Zhou, H., Qian, K., Tan, W., Zhang, Z., Gu, L., & Yu, Y. (2020). RGB-IR cross input and sub-pixel upsampling network for infrared image super-resolution. Sensors, 20(1), 281. google scholar
• Fouad, M. M. M., Zawbaa, H. M., El-Bendary, N., & Hassanien, A. E. (2013, December). Automatic nile tilapia fish classification approach using machine learning techniques. In 13th international conference on hybrid intelligent systems (HIS 2013) (pp. 173-178). IEEE. google scholar
• Hu, J., Shen, L., & Sun, G. (2018).’Squeeze-and-excitation networks.’ In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141). google scholar ImageNet Dataset. https://image-net.org/download.php. 2013. google scholar
• Jalal, A., Salman, A., Mian, A., Shortis, M., & Shafait, F. (2020). Fish detection and species classification in underwater environments using deep learning with temporal information. Ecological Informatics, 57, 101088. google scholar
• Jin, X., Xie, Y., Wei, X. S., Zhao, B. R., Chen, Z. M., & Tan, X. (2022). Delving deep into spatial pooling for squeeze-and-excitation networks. Pattern Recognition, 121, 108159. google scholar
• Kroetz, K., Luque, G. M., Gephart, J. A., Jardine, S. L., Lee, P., Chicojay Moore, K., ... & Donlan, C. J. (2020). Consequences of seafood mislabeling for marine populations and fisheries management. Proceedings of the National Academy of Sciences, 117(48), 3031830323. google scholar
• Labao, A. B., & Naval Jr, P. C. (2019). Cascaded deep network systems with linked ensemble components for underwater fish detection in the wild. Ecological Informatics, 52, 103-121. google scholar
• Ovalle, J. C., Vilas, C., & Antelo, L. T. (2022). On the use of deep learning for fish species recognition and quantification on board fishing vessels. Marine Policy, 139, 105015. google scholar
• Meng, L., Hirayama, T., & Oyanagi, S. (2018). Underwater-drone with panoramic camera for automatic fish recognition based on deep learning. Ieee Access, 6, 17880-17886. google scholar
• Murugaiyan, J. S., Palaniappan, M., Durairaj, T., & Muthukumar, V. (2021). Fish species recognition using transfer learning techniques. International Journal of Advances in Intelli- gent Informatics, 7(2), 188-197. google scholar
• Naaum, A. M., Warner, K., Mariani, S., Hanner, R. H., & Carolin, C. D. (2016). Seafood mislabeling incidence and impacts. In Seafood Authenticity and Traceability (pp. 3-26). Academic Press. google scholar
• Pollack, S. J., Kawalek, M. D., Williams-Hill, D. M., & Hellberg, R. S. (2018). Evaluation of DNA barcoding methodologies for the identification of fish species in cooked products. Food Control, 84, 297-304. google scholar
• Prasetyo, E., Suciati, N., & Fatichah, C. (2021). Multi-level residual network VGGNet for fish species classification. Journal of King Saud University-Computer and Information Sciences. google scholar
• Qiu, C., Zhang, S., Wang, C., Yu, Z., Zheng, H., & Zheng, B. (2018). Improving transfer learning and squeeze-and-excitation networks for small-scale fine-grained fish image classification. IEEE Access, 6, 78503-78512. google scholar
• Rauf, H. T., Lali, M. I. U., Zahoor, S., Shah, S. Z. H., Rehman, A. U., & Bukhari, S. A. C. (2019).Visual features based automated identification of fish species using deep convolutional neural networks. Computers and electronics in agriculture, 167, 105075. google scholar
• Vilas, C., Antelo, L. T., Martin-Rodriguez, F., Morales, X., Perez-Martin, R. I., Alonso, A. A., ... & Barral-Martinez, M. (2020). Use of computer vision onboard fishing vessels to quantify catches: The iObserver. Marine Policy, 116, 103714. google scholar
• Villon, S., lovan, C., Mangeas, M., Claverie, T., Mouillot, D., Villeger, S., & Vigliola, L. (2021).’Automatic underwater fish species classification with limited data using few-shot learning. Ecological Informatics, 63, 101320. google scholar
• Villon, S., Mouillot, D., Chaumont, M., Darling, E. S., Subsol, G., Claverie, T., & Villeger, S. (2018). A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecological informatics, 48, 238-244. google scholar
• Xu, X., Li, W., & Duan, Q. (2021). Transfer learning and SE-ResNet152 networks-based for small-scale unbalanced fish species identification. Computers and Electronics in Agriculture, 180, 105878. google scholar
• Zhang, Y., Zhang, F., Cheng, J., & Zhao, H. (2021). Classification and Recognition of Fish Farming by Extraction New Features to Control the Economic Aquatic Product.’ Complexity, 2021. google scholar