Smoke detection from foggy environment based on color spaces

Abstract

Detection of smoke from videos captured by surveillance cameras in outdoor environments is one of the useful outcome of Internet of Things (IoT) applications. The potential benefit increases when deep learning (DL) architectures are involved. However, an inherent difficulty is to detect smoke while natural events like fog exists. The effectiveness of color spaces in detection performance has not yet fully evaluated in those architectures. Moreover, the energy and memory requirements of DL architectures may not be applicable for handling IoT implementation demands. Therefore, in this work, a DL architecture with a suitable color space model, applicable for IoT implementations is proposed to detect smoke from videos in foggy environment. By collecting several videos including smoke samples, the performance comparison of popular and the state-of-the-art DL architectures denoted the outperforming result according to both accuracy and memory usage.

Keywords

• Color space
• Deep learning
• IoT
• Smoke detection

References

1. [1] A. Sharma, P. K. Singh, and Y. Kumar, “An integrated fire detection system using IoT and image processing technique for smart cities,” Sustainable Cities and Society, vol. 61, 102332, 2020.
2. [2] A. E. Çetin, K. Dimitropoulos, B. Gouverneur, N. Grammalidis, O. Günay, Y. H. Habiboğlu, B. U. Töreyin, and S. Verstockt, “Video fire detection - Review,” Digital Signal Processing, vol. 23, no. 6, pp. 1827-1843, 2013.
3. [3] P. Li and W. Zhao, “Image fire detection algorithms based on convolutional neural networks,” Case Studies in Thermal Engineering, vol. 19, 100625, 2020.
4. [4] B. U. Töreyin, Y. Dedeoğlu, and A. E. Çetin, “Wavelet based real-time smoke detection in video,” in 13th European Signal Processing Conference, Antalya, Turkey, 2005, pp. 1-4.
5. [5] A. Genovese, R. D. Labati, V. Piuri, and F. Scotti, “Wildfire smoke detection using computational intelligence techniques,” in 2011 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA) Proceedings, Ottawa, ON, Canada, 2011, pp. 1-6.
6. [6] R. D. Labati, A. Genovese, V. Piuri, and F. Scotti, “Wildfire smoke detection using computational intelligence techniques enhanced with synthetic smoke plume generation,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 4, pp. 1003-1012, July 2013.
7. [7] K. Zhou and X. Zhang, “Design of outdoor fire intelligent alarm system based on image recognition,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 34, no. 07, 2050018, 2020.
8. [8] X. Wu, Y. Cao, X. Lu, and H. Leung, “Patchwise dictionary learning for video forest fire smoke detection in wavelet domain,” Neural Computing and Applications, vol. 33, pp. 7965-7977, 2021.

9. [9] T.-H. Chen, P.-H. Wu, and Y.-C. Chiou, “An early fire-detection method based on image processing,” in International Conference on Image Processing, Singapore, 2004, pp. 1707-1710, vol. 3.
10. [10] T. Çelik and H. Demirel, “Fire detection in video sequences using a generic color model,” Fire Safety Journal, vol. 44, no. 2, pp. 147-158, 2009.
11. [11] Y. Chunyu, F. Jun, W. Jinjun, and Z. Yongming, “Video fire smoke detection using motion and color features,” Fire Technology, vol. 46, pp. 651-663, 2010.
12. [12] P. Foggia, A. Saggese, and M. Vento, “Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 9, pp. 1545-1556, Sept. 2015.
13. [13] C. E. Prema, S. S. Vinsley, and S. Suresh, “Multi feature analysis of smoke in YUV color space for early forest fire detection,” Fire Technology, vol. 52, pp. 1319-1342, 2016.
14. [14] A. Khalil, S. U. Rahman, F. Alam, I. Ahmad, and I. Khalil, “Fire detection using multi color space and background modeling,” Fire Technology, vol. 57, pp. 1221-1239, 2021.
15. [15] K.-M. Park and C.-O. Bae, “Smoke detection in ship engine rooms based on video images,” IET Image Processing, vol. 14, no. 6, pp. 1141-1149, 2020.
16. [16] S. Frizzi, R. Kaabi, M. Bouchouicha, J. Ginoux, E. Moreau, and F. Fnaiech, “Convolutional neural network for video fire and smoke detection,” in IECON 2016 - 42nd Annual Conference of the IEEE Industrial Electronics Society, Florence, Italy, 2016, pp. 877-882.
17. [17] C. Tao, J. Zhang, and P. Wang, “Smoke detection based on deep convolutional neural networks,” in International Conference on Industrial Informatics - Computing Technology, Intelligent Technology, Industrial Information Integration (ICIICII), Wuhan, China, 2016, pp. 150-153.
18. [18] Y. Luo, L. Zhao, P. Liu, and D. Huang, “Fire smoke detection algorithm based on motion characteristic and convolutional neural networks,” Multimedia Tools and Applications, vol. 77, no. 12, pp. 15075-15092, 2018.
19. [19] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014. Online available: arXiv:1409.1556.
20. [20] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems, 2012, pp. 1097-1105.
21. [21] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1-9.
22. [22] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778.
23. [23] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 2261-2269.
24. [24] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 4510-4520.
25. [25] S. Saponara, A. Elhanashi, and A. Gagliardi, “Real-time video fire/smoke detection based on CNN in antifire surveillance systems,” Journal of Real-Time Image Processing, vol.18, pp. 889-900, 2021.
26. [26] Z. Yin, B. Wan, F. Yuan, X. Xia, and J. Shi, “A deep normalization and convolutional neural network for image smoke detection,” IEEE Access, vol. 5, pp. 18429-18438, 2017.
27. [27] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” Advances in Neural Information Processing Systems, 28, 2015.
28. [28] Q.-x. Zhang, G.-h. Lin, Y.-m. Zhang, G. Xu, and J.-j. Wang, “Wildland forest fire smoke detection based on faster R-CNN using synthetic smoke images,” Procedia Engineering, vol. 211, pp. 441-446, 2018.
29. [29] F. Yuan, L. Zhang, B. Wan, X. Xia, and J. Shi, “Convolutional neural networks based on multi-scale additive merging layers for visual smoke recognition,” Machine Vision and Applications, vol. 30, pp. 345-358, 2019.
30. [30] T. Liu, J. Cheng, X. Du, X. Luo, L. Zhang, B. Cheng, and Y. Wang, “Video smoke detection method based on change-cumulative image and fusion deep network,” Sensors, vol. 19, no. 23, 5060, 2019.
31. [31] A. Jadon, M. Omama, A. Varshney, M. S. Ansari, and R. Sharma, “FireNet: a specialized lightweight fire & smoke detection model for real-time IoT applications,” arXiv:190511922, 2019.
32. [32] S. Khan, K. Muhammad, S. Mumtaz, S. W. Baik, and V. H. C. de Albuquerque, “Energy-efficient deep CNN for smoke detection in foggy IoT environment,” IEEE Internet of Things Journal, vol. 6, no. 6, pp. 9237-9245, 2019.
33. [33] K. Muhammad, S. Khan, M. Elhoseny, S. H. Ahmed, and S. W. Baik, “Efficient fire detection for uncertain surveillance environment,” IEEE Transactions on Industrial Informatics, vol. 15, no. 5, pp. 3113-3122, May 2019.
34. [34] K. Muhammad, S. Khan, V. Palade, I. Mehmood, and V. H. C. de Albuquerque, “Edge intelligence-assisted smoke detection in foggy surveillance environments,” IEEE Transactions on Industrial Informatics, vol. 16, no. 2, pp. 1067-1075, February 2020.
35. [35] L. He, X. Gong, S. Zhang, L. Wang, F. Li, “Efficient attention based deep fusion CNN for smoke detection in fog environment,” Neurocomputing, vol. 434, pp. 224-238, 2021.
36. [36] S. Khan, K. Muhammad, T. Hussain, J. der Ser, F. Cuzzolin, S. Bhattacharyya, Z. Akhtar, and V. H. C. de Albuquerque, “DeepSmoke: Deep learning model for smoke detection and segmentation in outdoor environments,” Expert Systems with Applications, 115125, 2021.
37. [37] S. Aslan, U. Güdükbay B. U. Töreyin, and A. E. Çetin, “Early wildfire smoke detection based on motion-based geometric image transformation and deep convolutional generative adversarial networks,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 2019, pp. 8315-8319.
38. [38] T. Li, E. Zhao, J. Zhang, and C. Hu, “Detection of wildfire smoke images based on a densely dilated convolutional network,” Electronics, vol. 8, no. 10, 1131, Oct. 2019.
39. [39] G. Xu, Y. Zhang, Q. Zhang, G. Lin, Z. Wang, Y. Jia, and J. Wang, “Video smoke detection based on deep saliency network,” Fire Safety Journal, vol. 105, pp. 277-285, 2019.
40. [40] K. Gu, Z. Xia, J. Qiao, and W. Lin, “Deep dual-channel neural network for image-based smoke detection,” IEEE Transactions on Multimedia, vol. 22, no. 2, pp. 311-323, 2020.
41. [41] F. Zhang, W. Qin, Y. Liu, Z. Xiao, J. Liu, Q. Wang, and K. Liu, “A dual-channel convolution neural network for image smoke detection,” Multimedia Tools and Applications, vol. 79, pp. 34587-34603, 2020.
42. [42] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd edition, Prentice Hall, 2007.
43. [43] M. S. Nixon and A. S. Aguado, Feature Extraction & Image Processing for Computer Vision, 3rd edition, Academic Press, 2012.
44. [44] M. Bugaric, T. Jakovcevic, D. Stipanicev, “Adaptive estimation of visual smoke detection parameters based on spatial data and fire risk index,” Computer Vision and Image Understanding, vol. 118, pp. 184-196, 2014.
45. [45] K. Dimitropoulos, P. Barmpoutis and N. Grammalidis, “Spatio-temporal flame modeling and dynamic texture analysis for automatic video-based fire detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 2, pp. 339-351, Feb. 2015.
46. [46] U. E. Yıldız and M. E. Özbek, “Deep learning based smoke detection for foggy environments,” in 12th International Conference on Electrical and Electronics Engineering (ELECO), Bursa, Turkey, 2020, pp. 237-240.