Multi-species Fish Identification using Hybrid DeepCNN with Refined Squeeze and Excitation Architecture

Abstract

Fish play a prominent role in the food web and fish farming has value for both human consumption and tourist attractions. Due to the increasing importance of marine biodiversity, recognition of fish species has become a prominent task in monitoring the mislabelling of seafood and extinct species. This problem can be solved using traditional manual annotation on the images. To reduce manpow-er, cost, and tremendous time, deep learning approaches are used which always require large data-sets. Therefore, fish species identification is a challenging task using disproportionately small data sets. In this research, we develop a new method by refining the squeeze and excitation network for the automatic fish species classification model to identify 23 different types of fish species. To achieve this, a hybrid framework using deep learning is proposed on a large-scale dataset and implemented transfer learning for a small-scale dataset. Deep learning methods can be used to identify fish in un-derwater images. In this study, we have proposed a new method of hybrid Deep Convolutional Neu-ral Network (CNN) along with a Support Vector Machine (SVM) for classification. Additionally, the Squeeze and Excitation (SE) block has been improved for improved feature extraction. The proposed method achieved an accuracy of 97.90%. Then post-training with the small-scale dataset (Croatian) achieved an accuracy of 94.99% with an 11% improvement compared to Bilinear CNN (B-CNN) (Qui et al., 2018) and can be used in any underwater applications to identify fish species and avoid misla-belling of seafood.

Keywords

• Deep Learning
• Convolutional Neural Network
• Squeeze and Excitation
• Fish species
• Fish4knowledge dataset

References

1. Allken, V., Handegard, N. O., Rosen, S., Schreyeck, T., Mahiout, T., & Malde, K. (2019). Fish species identification using a convolutional neural network trained on synthetic data. ICES Journal of Marine Science, 76(1), 342-349. google scholar
2. B. B. B. Phoenix X. Huang and R. B. Fishera.Fish4KnowledgeDataset: https://homepages.inf.ed.ac.uk/rbf/Fish4Knowledge/ GROUNDTRUTH/RECOG/. 2013. google scholar
3. Chen, P. Y., Ho, C. W., Chen, A. C., Huang, C. Y., Liu, T. Y., & Liang, K. H. (2020). Inves-tigating seafood substitution problems and consequences in Taiwan using molecular barcoding and deep microbiome profiling. Scientific reports, 10(1), 1-9. google scholar
4. Crozier, L. G., McClure, M. M., Beechie, T., Bograd, S. J., Boughton, D. A., Carr, M.,... & Willis-Norton, E. (2019). Climate vulnerability assessment for Pacific salmon and steelhead in the California Current Large Marine Ecosystem. PloS one, 14(7), e0217711. google scholar
5. Dagoudo, M., Qiang, J., & Solevo, M. P. (2022). Status in science and technology devel- opments in Benin’s aquaculture industry: a review. Aquaculture International, 1-15. google scholar Data Science Glossary. https://c3.ai/glossary/data-science/. google scholar
6. Deep, B. V., & Dash, R. (2019, March). Underwater fish species recognition using deep learning techniques. In 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN) (pp. 665-669). IEEE. google scholar
7. Dos Santos, A. A., & Goncalves, W. N. (2019). Improving Pantanal fish species recogni- tion through taxonomic ranks in convolutional neural networks. Ecological Informatics, 53, 100977. google scholar
8. Du, J., Zhou, H., Qian, K., Tan, W., Zhang, Z., Gu, L., & Yu, Y. (2020). RGB-IR cross input and sub-pixel upsampling network for infrared image super-resolution. Sensors, 20(1), 281. google scholar

9. Fouad, M. M. M., Zawbaa, H. M., El-Bendary, N., & Hassanien, A. E. (2013, December). Automatic nile tilapia fish classification approach using machine learning techniques. In 13th international conference on hybrid intelligent systems (HIS 2013) (pp. 173-178). IEEE. google scholar
10. Hu, J., Shen, L., & Sun, G. (2018).’Squeeze-and-excitation networks.’ In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141). google scholar ImageNet Dataset. https://image-net.org/download.php. 2013. google scholar
11. Jalal, A., Salman, A., Mian, A., Shortis, M., & Shafait, F. (2020). Fish detection and species classification in underwater environments using deep learning with temporal information. Ecological Informatics, 57, 101088. google scholar
12. Jin, X., Xie, Y., Wei, X. S., Zhao, B. R., Chen, Z. M., & Tan, X. (2022). Delving deep into spatial pooling for squeeze-and-excitation networks. Pattern Recognition, 121, 108159. google scholar
13. Kroetz, K., Luque, G. M., Gephart, J. A., Jardine, S. L., Lee, P., Chicojay Moore, K., ... & Donlan, C. J. (2020). Consequences of seafood mislabeling for marine populations and fisheries management. Proceedings of the National Academy of Sciences, 117(48), 3031830323. google scholar
14. Labao, A. B., & Naval Jr, P. C. (2019). Cascaded deep network systems with linked ensemble components for underwater fish detection in the wild. Ecological Informatics, 52, 103-121. google scholar
15. Ovalle, J. C., Vilas, C., & Antelo, L. T. (2022). On the use of deep learning for fish species recognition and quantification on board fishing vessels. Marine Policy, 139, 105015. google scholar
16. Meng, L., Hirayama, T., & Oyanagi, S. (2018). Underwater-drone with panoramic camera for automatic fish recognition based on deep learning. Ieee Access, 6, 17880-17886. google scholar
17. Murugaiyan, J. S., Palaniappan, M., Durairaj, T., & Muthukumar, V. (2021). Fish species recognition using transfer learning techniques. International Journal of Advances in Intelli- gent Informatics, 7(2), 188-197. google scholar
18. Naaum, A. M., Warner, K., Mariani, S., Hanner, R. H., & Carolin, C. D. (2016). Seafood mislabeling incidence and impacts. In Seafood Authenticity and Traceability (pp. 3-26). Academic Press. google scholar
19. Pollack, S. J., Kawalek, M. D., Williams-Hill, D. M., & Hellberg, R. S. (2018). Evaluation of DNA barcoding methodologies for the identification of fish species in cooked products. Food Control, 84, 297-304. google scholar
20. Prasetyo, E., Suciati, N., & Fatichah, C. (2021). Multi-level residual network VGGNet for fish species classification. Journal of King Saud University-Computer and Information Sciences. google scholar
21. Qiu, C., Zhang, S., Wang, C., Yu, Z., Zheng, H., & Zheng, B. (2018). Improving transfer learning and squeeze-and-excitation networks for small-scale fine-grained fish image classification. IEEE Access, 6, 78503-78512. google scholar
22. Rauf, H. T., Lali, M. I. U., Zahoor, S., Shah, S. Z. H., Rehman, A. U., & Bukhari, S. A. C. (2019).Visual features based automated identification of fish species using deep convolutional neural networks. Computers and electronics in agriculture, 167, 105075. google scholar
23. Vilas, C., Antelo, L. T., Martin-Rodriguez, F., Morales, X., Perez-Martin, R. I., Alonso, A. A., ... & Barral-Martinez, M. (2020). Use of computer vision onboard fishing vessels to quantify catches: The iObserver. Marine Policy, 116, 103714. google scholar
24. Villon, S., lovan, C., Mangeas, M., Claverie, T., Mouillot, D., Villeger, S., & Vigliola, L. (2021).’Automatic underwater fish species classification with limited data using few-shot learning. Ecological Informatics, 63, 101320. google scholar
25. Villon, S., Mouillot, D., Chaumont, M., Darling, E. S., Subsol, G., Claverie, T., & Villeger, S. (2018). A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecological informatics, 48, 238-244. google scholar
26. Xu, X., Li, W., & Duan, Q. (2021). Transfer learning and SE-ResNet152 networks-based for small-scale unbalanced fish species identification. Computers and Electronics in Agriculture, 180, 105878. google scholar
27. Zhang, Y., Zhang, F., Cheng, J., & Zhao, H. (2021). Classification and Recognition of Fish Farming by Extraction New Features to Control the Economic Aquatic Product.’ Complexity, 2021. google scholar