Segmentation-guided explainable smoking detection using SAM and CNN

Halil İbrahim Sarı; Kübra Keser

doi:10.59313/jsr-a.1958126

Segmentation-guided explainable smoking detection using SAM and CNN

Abstract

Automatic detection of smoking behavior is of great importance to public health. High classification accuracy is reported in deep learning, but how this works and how it relates to various biases are difficult to explain. To address this gap, a smoking detection model based on Segment Anything Model (SAM), incorporating evolutionary neural networks and Gradient Weighted Class Activation Mapping (Grad-CAM) analyses to suppress background noise, has been proposed. In a study of 1120 images, an accuracy of 88.84% and a Receiver Operating Characteristic - Area Under the Curve (ROC-AUC) value of 0.9613 were obtained. Furthermore, data augmentation strategies were included to improve the robustness and generalization performance of the model, and statistical significance analysis was added to show that the performance improvements provided by the SAM-based approach were not due to random variation. At the same time, the false negative rate, a crucial concept for the health field, was relatively reduced. The analyses confirmed that there is an agreement between the model and hand-to-mouth-smoking interactions. It offers a focused, interpretable, reliable, and practical solution for enclosed spaces.

Keywords

References

[1] World Health Organization, “WHO report on the global tobacco epidemic 2023: Protect people from tobacco smoke,” Geneva: World Health Organization, 2023. [Online]. Available: https://www.who.int/publications/i/item/9789240077164
[2] D. B. Abrams, A. M. Glasser, J. L. Pearson, A. C. Villanti, L. K. Collins, and R. S. Nicura, “Harm minimization and tobacco control: reframing societal views of nicotine use to rapidly save lives,” Annu. Rev. Public Health, vol. 39, no. 1, pp. 193-213, 2018.
[3] E. Akdeniz, “Sigara bağımlılığı sağlığa etkileri ve sigara bıraktırmada kullanılan transteoretik model,” Kırşehir Ahi Evran Üniversitesi Sağlık Bilimleri Dergisi, vol. 2, no. 3, pp. 11-25, 2019.
[4] P. Jha, C. Ramasundarahettige, V. Landsman, B. Rostron, M. Thun, R. N. Anderson, and R. Peto, “21st-century hazards of smoking and benefits of cessation in the United States,” N. Engl. J. Med., vol. 368, no. 4, pp. 341-350, 2013.
[5] W. C. Su, H. L. Juan, J. I. Lee, H. P. Huang, S. C. Chen, and J. H. Geng, “Secondhand smoke increases the risk of developing chronic obstructive pulmonary disease,” Sci. Rep., vol. 14, no. 1, p. 7481, 2024, doi: 10.1038/s41598-024-58038-2.
[6] L. S. Flor, J. A. Anderson, N. Ahmad, A. Aravkin, S. Carr, X. Dai, and E. Gakidou, “Health effects associated with exposure to secondhand smoke: a Burden of Proof study,” Nat. Med., vol. 30, pp. 149-167, 2024, doi: 10.1038/s41591-023- 02743-4.
[7] C. Karabiber, N. Azboy, F. Altıner, B. Avlamaz, B. Özay, Ö. Ulutaş, and T. İnandı, “Tıp fakültesi öğrencilerinin tütün kullanımı hakkında bilgi, tutum ve davranışları,” The Medical Journal of Mustafa Kemal University, vol. 9, no. 33, pp. 21-32, 2018.
[8] A. Navas-Acien, A. Çarkoğlu, G. Ergör, M. Hayran, T. Ergüder, B. Kaplan, and J. E. Cohen, “Compliance with smoke-free legislation within public buildings: a cross-sectional study in Turkey,” Bull. World Health Organ., vol. 94, no. 2, p. 92-102, 2016, doi: 10.2471/BLT.15.158238.

[9] J. H. Cho, “Detection of smoking in indoor environment using machine learning,” Appl. Sci., vol. 10, no. 24, p. 8912, 2020, doi: 10.3390/app10248912.
[10] O. Amft and G. Troster, “On-body sensing solutions for automatic dietary monitoring,” IEEE Pervasive Comput., vol. 8, no. 2, pp. 62-70, Apr. 2009.
[11] P. Lopez-Meyer, S. Tiffany, Y. Patil, and E. Sazonov, “Monitoring of cigarette smoking using wearable sensors and support vector machines,” IEEE Trans. Biomed. Eng., vol. 60, no. 7, pp. 1867-1872, 2013, doi: 10.1109/TBME.2013.2243729.
[12] V. Y. Senyurek, M. H. Imtiaz, P. Belsare, S. Tiffany, and E. Sazonov, “Smoking detection based on regularity analysis of hand to mouth gestures,” Biomed. Signal Process. Control, vol. 51, pp. 106-112, 2019, doi: 10.1016/j.bspc.2019.01.026.
[13] A. Khan, M. A. Elhassan, S. Khan, and H. Deng, “Deep learning-based smoker classification and detection: An overview and evaluation,” Expert Syst. Appl., vol. 267, p. 126208, 2025, doi: 10.1016/j.eswa.2024.126208.
[14] C. Ruilong, L. Lei, and C. Zhiping, “Algorithm for Real-Time Smoking Detection Based on Deep Learning,” Journal of Frontiers of Computer Science & Technology, vol. 15, no. 2, 2021, doi. 10.3778/j.issn.1673-9418.2009069.
[15] Z. Zhang, Chen, H., R. Xiao, and Q. Li, “Research on smoking detection based on deep learning,” in Journal of Physics: Conference Series, vol. 2024, p. 012042, IOP Publishing, 2021, doi:10.1088/1742-6596/2024/1/012042.
[16] T. Singh, A. Singh, D. Sahu, A. Tomar, and S. Pandey, “Real Time Cigarette Detection using Deep Learning Models,” International Journal for Research in Applied Science and Engineering Technology, pp. 1946-1948, 2023. https://doi.org/10.22214/ijraset.2023.51977.
[17] A. Khan, S. Khan, B. Hassan, and Z. Zheng, “CNN-based smoker classification and detection in smart city application,” Sensors, vol. 22, no. 3, p. 892, 2022, doi: 10.3390/s22030892.
[18] A. Khan, S. Khan, B. Hassan, R. Khan, and Z. Zheng, “SmokerViT: A transformer-based method for smoker recognition,” Computers, Materials, & Continua, vol. 77, no. 1, p. 403, 2023, https://doi.org/10.32604/cmc.2023.040251.
[19] N. K. Torun, O. Tatli, and M. Önalan, “Kamu sağlığı ve çevresel sürdürülebilirlik için YOLOv5 tabanlı sigara tespiti ve sesli uyarı sistemi,” Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji, 2025, doi:10.29109/gujsc.1620772.
[20] M. Ismail, A. S. A. Yeon, S. M. M. Syed Zakaria, A. Zakaria, L. M. Kamarudin, R. Visvanathan, and M. A. A. Baharom, “Automated Smokıng Detectıon In Restrıcted Zones: A Yolov8-Based Approach Wıth Gender Identıfıcatıon,” Journal of Information System and Technology Management, vol. 10, pp. 169–183, 2025, doi: 10.35631/JISTM.1039011.
[21] R. Lakatos, P. Pollner, A. Hajdu, and T. Joó, “A multimodal deep learning architecture for smoking detection with a small data approach,” Front. Artif. Intell., vol. 7, p. 1326050, 2024, doi: 10.3389/frai.2024.1326050.
[22] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137-1149, Jun. 2016, doi: 10.1109/TPAMI.2016.2577031.
[23] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint, arXiv:2010.11929, 2020.
[24] Y. Zhang, X. Li, C. Liu, B. Shuai, Y. Zhu, B. Brattoli, and J. Tighe, “Vidtr: Video transformer without convolutions,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 13577-13587.
[25] K. Alomar, H. I. Aysel, and X. Cai, “RNNs, CNNs, and Transformers in human action recognition: a survey and a hybrid model,” Artif. Intell. Rev., vol. 58, 2025, doi: 10.1007/s10462- 025-11388-3.
[26] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, and R. Girshick, “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 4015-4026.
[27] P. Belsare, V. Y. Senyurek, M. H. Imtiaz, S. T. Tiffany, and E. Sazonov, “DeepPuff: utilizing deep learning for smoking behavior identification in free-living environment,” in Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), IEEE, Jul. 2023, pp. 1-5, doi: 10.1109/EMBC 40787.2023.10340528.
[28] V. Y. Senyurek, M. H. Imtiaz, P. Belsare, S. Tiffany, and E. Sazonov, “A CNN-LSTM neural network for recognition of puffing in smoking episodes using wearable sensors,” Biomed. Eng. Lett., vol. 10, no. 2, pp. 195-203, 2020, doi: 10.1007/s13534-020-00147-8.
[29] O. Mazurets, M. Molchanova, O. Zalutska, I. Kok, V. Levashenko, A. B. M. Salem, (2025). Vision transformers transfer learning for smoking detection in public spaces with transparent AI decisions. Proceedings of the 1st Workshop on Advanced AI in Explainability and Ethics for the Sustainable Development Goals (ExplAI-2025) (pp. 142-152).
[30] V. Jampani, R. Gadde, and P. V. Gehler, “Video propagation networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 451-461.
[31] Y. Li, H. Zhou, J. Feng, X. Li, X. Xu, P. Hou, and X. Hu, “An improved smoking behavior detection algorithm via incorporating an interference information filtering network,” Eng. Appl. Artif. Intell., vol. 136, p. 109050, 2024, doi: 10.1016/j.engappai.2024.109050.
[32] A. B. Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, and F. Herrera, “Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI,” Inf. Fusion, vol. 58, pp. 82-115, 2020, doi: 10.1016/j.inffus.2019.12.012.
[33] R. Liu, H. Liu, G. Li, H. Hou, T. Yu, and T. Yang, “Contextual debiasing for visual recognition with causal mechanisms,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 12745-12755, doi: 10.1109/CVPR52688.2022.01242.
[34] S. T. Erukude, A. Joshi, and L. Shamir, “Identifying bias in deep neural networks using image transforms,” Computers, vol. 13, no. 12, p. 341, 2024, doi: 10.3390/computers13120341.
[35] G. Schwalbe, G. Mikriukov, E. Heinert, S. Gerolymatos, M. Keser, A. Knoll, and A. Mütze, “On Background Bias of Post-Hoc Concept Embeddings in Computer Vision DNNs,” Explainable Artificial Intelligence, Cham: Springer Nature Switzerland, Jul. 2025, pp. 45-71, https://doi.org/10.1007/978-3-032-08330-2_3.
[36] S. Fabbrizzi, S. Papadopoulos, E. Ntoutsi, and I. Kompatsiaris, “A survey on bias in visual datasets,” Comput. Vision Image Understanding, vol. 223, doi: 10.1016/j.cviu.2022.103552.
[37] D. Miller, P. Moghadam, M. Cox, M. Wildie, and R. Jurdak, “What’s in the black box? the false negative mechanisms inside object detectors,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8510-8517, 2022, doi: 10.1109/LRA.2022.3187831.
[38] A. Khan, “Smoker detection dataset,” Mendeley Data, vol. 1, 2022, 10.17632/j45dj8bgfc.1.
[39] Z. Zhong, L. Zheng, G. Kang, S. Li, and Y. Yang, “Random erasing data augmentation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, Apr. 2020, pp. 13001-13008.
[40] Y. Xie, F. Li, Y. Wu, S. Yang, and Y. Wang, “HearSmoking: Smoking detection in driving environment via acoustic sensing on smartphones,” IEEE Trans. Mobile Comput., vol. 21, no. 8, pp. 2847-2860, Aug. 2021, doi: 10.1109/TMC.2020.3048785.
[41] A. Pundhir, D. Verma, P. Kumar, and B. Raman, “Region extraction based approach for cigarette usage classification using deep learning,” Computer Vision and Image Processing, Cham: Springer International Publishing, Dec. 2021, https://doi.org/10.1007/978-3-031-11349-9_33.

Details

Primary Language

English

Subjects

Biomedical Engineering (Other)

Journal Section

Research Article

Authors

Halil İbrahim Sarı
0009-0006-3964-3101
Türkiye

Kübra Keser ^*
0000-0001-7215-7232
Türkiye

Publication Date

September 30, 2026

Submission Date

May 24, 2026

Acceptance Date

August 19, 2026

Published in Issue

Year 2026 Number: 066

DOI

https://doi.org/10.59313/jsr-a.1958126

IZ

https://izlik.org/JA38TT32JZ

Cite

RIS / Bibtex

APA

Sarı, H. İ., & Keser, K. (2026). Segmentation-guided explainable smoking detection using SAM and CNN. Journal of Scientific Reports-A, 066, 16-34. https://doi.org/10.59313/jsr-a.1958126

AMA

1.Sarı Hİ, Keser K. Segmentation-guided explainable smoking detection using SAM and CNN. JSR-A. 2026;(066):16-34. doi:10.59313/jsr-a.1958126

Chicago

Sarı, Halil İbrahim, and Kübra Keser. 2026. “Segmentation-Guided Explainable Smoking Detection Using SAM and CNN”. Journal of Scientific Reports-A, nos. 066: 16-34. https://doi.org/10.59313/jsr-a.1958126.

EndNote

Sarı Hİ, Keser K (September 1, 2026) Segmentation-guided explainable smoking detection using SAM and CNN. Journal of Scientific Reports-A 066 16–34.

IEEE

[1]H. İ. Sarı and K. Keser, “Segmentation-guided explainable smoking detection using SAM and CNN”, JSR-A, no. 066, pp. 16–34, Sept. 2026, doi: 10.59313/jsr-a.1958126.

ISNAD

Sarı, Halil İbrahim - Keser, Kübra. “Segmentation-Guided Explainable Smoking Detection Using SAM and CNN”. Journal of Scientific Reports-A. 066 (September 1, 2026): 16-34. https://doi.org/10.59313/jsr-a.1958126.

JAMA

1.Sarı Hİ, Keser K. Segmentation-guided explainable smoking detection using SAM and CNN. JSR-A. 2026;:16–34.

MLA

Sarı, Halil İbrahim, and Kübra Keser. “Segmentation-Guided Explainable Smoking Detection Using SAM and CNN”. Journal of Scientific Reports-A, no. 066, Sept. 2026, pp. 16-34, doi:10.59313/jsr-a.1958126.

Vancouver

1.Halil İbrahim Sarı, Kübra Keser. Segmentation-guided explainable smoking detection using SAM and CNN. JSR-A. 2026 Sep. 1;(066):16-34. doi:10.59313/jsr-a.1958126