Classification of Images in Bad Weather Conditions with Convolutional Neural Networks

Abstract

Weather conditions are one of the major factors significantly influencing the daily lives of individuals. Unfavorable weather conditions adversely affect their lives and directly impede the progress of the subsequent image-processing steps necessary for real-world vision tasks such as object detection and autonomous driving. For this reason, the correct classification of the weather conditions is of great importance. Although traditional classification methods achieve high accuracy in various tasks, they cannot achieve the same success in classifying weather conditions. In this paper, we propose a novel convolutional neural network (CNN) framework for the classification of weather conditions with high accuracy. The proposed network outperforms the existing methods with 95.50% accuracy for a classification problem with five different scenarios.

Keywords

• Multi-class classification
• deep learning
• convolutional neural networks
• weather classification

References

1. [1] S. Zang, M. Ding, D. Smith, P. Tyler, T. Rakotoarivelo, and M. A. Kaafar, “The impact of adverse weather conditions on autonomous vehicles: How rain, snow, fog, and hail affect the performance of a self-driving car,” IEEE vehicular technology magazine, vol. 14, no. 2, pp. 103–111, 2019.
2. [2] X. Zhao, P. Liu, J. Liu, and X. Tang, “A time, space and colorbased classification of different weather conditions,” in 2011 Visual Communications and Image Processing (VCIP). IEEE, 2011, pp. 1–4.
3. [3] Q. Li, Y. Kong, and S.-m. Xia, “A method of weather recognition based on outdoor images,” in 2014 International Conference on Computer Vision Theory and Applications (VISAPP), vol. 2. IEEE, 2014, pp. 510–516.
4. [4] H. Kurihata, T. Takahashi, I. Ide, Y. Mekada, H. Murase, Y. Tamatsu, and T. Miyahara, “Rainy weather recognition from in-vehicle camera images for driver assistance,” in IEEE Proceedings. Intelligent Vehicles Symposium, 2005. IEEE, 2005, pp. 205–210.
5. [5] X. Yan, Y. Luo, and X. Zheng, “Weather recognition based on images captured by vision system in vehicle,” in Advances in Neural Networks– ISNN 2009: 6th International Symposium on Neural Networks, ISNN 2009 Wuhan, China, May 26-29, 2009 Proceedings, Part III 6. Springer, 2009, pp. 390–398.
6. [6] Z. Chen, F. Yang, A. Lindner, G. Barrenetxea, and M. Vetterli, “How is the weather: Automatic inference from images,” in 2012 19th IEEE International conference on image processing. IEEE, 2012, pp. 1853– 1856.
7. [7] M. Roser and F. Moosmann, “Classification of weather situations on single color images,” in 2008 IEEE intelligent vehicles symposium. IEEE, 2008, pp. 798–803.
8. [8] C. Lu, D. Lin, J. Jia, and C.-K. Tang, “Two-class weather classification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3718–3725.

9. [9] B. Zhao, X. Li, X. Lu, and Z. Wang, “A cnn–rnn architecture for multilabel weather recognition,” Neurocomputing, vol. 322, pp. 47–57, 2018.
10. [10] S. Chen, T. Shu, H. Zhao, and Y. Y. Tang, “Mask-cnn-transformer for real-time multi-label weather recognition,” Knowledge-Based Systems, vol. 278, p. 110881, 2023.
11. [11] R. Saravanan and P. Sujatha, “A state of art techniques on machine learning algorithms: a perspective of supervised learning approaches in data classification,” in 2018 Second international conference on intelligent computing and control systems (ICICCS). IEEE, 2018, pp. 945–949.
12. [12] M. Cord and P. Cunningham, Eds., Machine Learning Techniques for Multimedia - Case Studies on Organization and Retrieval. Springer, 2008.
13. [13] M. A. T. Figueiredo and A. K. Jain, “Unsupervised learning of finite mixture models,” IEEE Transactions on pattern analysis and machine intelligence, vol. 24, no. 3, pp. 381–396, 2002.
14. [14] P. Thomas, “Review of” semi-supervised learning” by o. chapelle, b. sch¨olkopf, and a. zien, eds. london, uk, mit press, 2006,” IEEE Transactions on Neural Networks, vol. 20, no. 3, pp. 542–542, 2009.
15. [15] L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: A survey,” Journal of artificial intelligence research, vol. 4, pp. 237–285, 1996.
16. [16] J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural networks, vol. 61, pp. 85–117, 2015.
17. [17] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems, vol. 25, 2012.
18. [18] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255.
19. [19] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
20. [20] M. Lin, Q. Chen, and S. Yan, “Network in network. arxiv 2013,” arXiv preprint arXiv:1312.4400, 2013.
21. [21] B. Mihaljevi´c, C. Bielza, and P. Larra˜naga, “Bayesian networks for interpretable machine learning and optimization,” Neurocomputing, vol. 456, pp. 648–665, 2021.
22. [22] P. Sarshar, J. Radianti, and J. J. Gonzalez, “Modeling panic in ship fire evacuation using dynamic bayesian network,” in Third International Conference on Innovative Computing Technology (INTECH 2013). IEEE, 2013, pp. 301–307.
23. [23] P. Wang, E. Fan, and P. Wang, “Comparative analysis of image classification algorithms based on traditional machine learning and deep learning,” Pattern Recognition Letters, vol. 141, pp. 61–67, 2021.
24. [24] C.-Y. Liou, W.-C. Cheng, J.-W. Liou, and D.-R. Liou, “Autoencoder for words,” Neurocomputing, vol. 139, pp. 84–96, 2014.
25. [25] Y. Xiao, J. Wu, Z. Lin, and X. Zhao, “A semi-supervised deep learning method based on stacked sparse auto-encoder for cancer prediction using rna-seq data,” Computer methods and programs in biomedicine, vol. 166, pp. 99–105, 2018.
26. [26] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and L. Bottou, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.” Journal of machine learning research, vol. 11, no. 12, 2010.
27. [27] P. Kumar Mallick, S. H. Ryu, S. K. Satapathy, S. Mishra, G. N. Nguyen, and P. Tiwari, “Brain mri image classification for cancer detection using deep wavelet autoencoder-based deep neural network,” IEEE Access, vol. 7, pp. 46 278–46 287, 2019.
28. [28] P. Smolensky et al., “Information processing in dynamical systems: Foundations of harmony theory,” 1986.
29. [29] L.-W. Kim, “Deepx: Deep learning accelerator for restricted boltzmann machine artificial neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 5, pp. 1441–1453, 2018.
30. [30] C. L. P. Chen, C.-Y. Zhang, L. Chen, and M. Gan, “Fuzzy restricted boltzmann machine for the enhancement of deep learning,” IEEE Transactions on Fuzzy Systems, vol. 23, no. 6, pp. 2163–2173, 2015.
31. [31] N. Le Roux and Y. Bengio, “Representational power of restricted boltzmann machines and deep belief networks,” Neural computation, vol. 20, no. 6, pp. 1631–1649, 2008.
32. [32] L. Shen and P. Tan, “Photometric stereo and weather estimation using internet images,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2009, pp. 1850–1857.
33. [33] M. Elhoseiny, S. Huang, and A. Elgammal, “Weather classification with deep convolutional neural networks,” in 2015 IEEE international conference on image processing (ICIP). IEEE, 2015, pp. 3349–3353.
34. [34] L.-W. Kang, K.-L. Chou, and R.-H. Fu, “Deep learning-based weather image recognition,” in 2018 International Symposium on Computer, Consumer and Control (IS3C). IEEE, 2018, pp. 384–387.
35. [35] X. Zhao and C. Wu, “Weather classification based on convolutional neural networks,” in 2021 International Conference on Wireless Communications and Smart Grid (ICWCSG). IEEE, 2021, pp. 293–296.
36. [36] J. M. Gandarias, A. J. Garcia-Cerezo, and J. M. G´omez-de Gabriel, “Cnn-based methods for object recognition with high-resolution tactile sensors,” IEEE Sensors Journal, vol. 19, no. 16, pp. 6872–6882, 2019.
37. [37] R. Vaddi and P. Manoharan, “Hyperspectral image classification using cnn with spectral and spatial features integration,” Infrared Physics & Technology, vol. 107, p. 103296, 2020.
38. [38] J. Xu, Z. Li, B. Du, M. Zhang, and J. Liu, “Reluplex made more practical: Leaky relu,” in 2020 IEEE Symposium on Computers and communications (ISCC). IEEE, 2020, pp. 1–7.
39. [39] C. O. Ancuti, C. Ancuti, R. Timofte, and C. De Vleeschouwer, “O-haze: a dehazing benchmark with real hazy and haze-free outdoor images,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018, pp. 754–762.
40. [40] H. Halmaoui, A. Cord, and N. Hauti`ere, “Contrast restoration of road images taken in foggy weather,” in 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). IEEE, 2011, pp. 2057–2063.
41. [41] G. Viyaj, “Keras-multiclass-image-classification,” https://github.com/ vijayg15/Keras-MultiClass-Image-Classification/tree/master, 2021.
42. [42] X. Fu, J. Huang, X. Ding, Y. Liao, and J. Paisley, “Clearing the skies: A deep network architecture for single-image rain removal,” IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2944–2956, 2017.
43. [43] Y.-F. Liu, D.-W. Jaw, S.-C. Huang, and J.-N. Hwang, “Desnownet: Context-aware deep network for snow removal,” IEEE Transactions on Image Processing, vol. 27, no. 6, pp. 3064–3073, 2018.
44. [44] Y. P. Loh and C. S. Chan, “Getting to know low-light images with the exclusively dark dataset,” Computer Vision and Image Understanding, vol. 178, pp. 30–42, 2019.
45. [45] J. Cai, S. Gu, and L. Zhang, “Learning a deep single image contrast enhancer from multi-exposure images,” IEEE Transactions on Image Processing, vol. 27, no. 4, pp. 2049–2062, 2018.
46. [46] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin et al., “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” arXiv preprint arXiv:1603.04467, 2016.
47. [47] M.-L. Zhang and Z.-H. Zhou, “Ml-knn: A lazy learning approach to multi-label learning,” Pattern recognition, vol. 40, no. 7, pp. 2038–2048, 2007.
48. [48] F. Zhu, H. Li, W. Ouyang, N. Yu, and X. Wang, “Learning spatial regularization with image-level supervisions for multi-label image classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 5513–5522.
49. [49] P. Flach and M. Kull, “Precision-recall-gain curves: Pr analysis done right,” Advances in neural information processing systems, vol. 28, 2015.