Evaluation of Deep Learning Models for Smoking Recognition with Smartwatch and Smartphone Sensors

Abstract

Smartwatches and smartphones are extensively used in human activity recognition, particularly for step counting and daily sports applications, thanks to the motion sensors integrated into these devices. Machine learning algorithms are often utilized to process sensor data and classify the activities. There are many studies that explore the use of traditional classification algorithms in activity recognition, however, recently, deep learning approaches are also receiving attention. In this paper, we use a dataset that particularly consists of smoking-related activities and explores the recognition performance of three deep learning architectures, namely Long-Short Term Memory (LSTM)}, Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN). We evaluate their performances according to different hyperparameters, different sensor types and device types. The results show that the performance of LSTM is much higher than that of CNN and RNN. Moreover, the use of magnetometer and gyroscope together with accelerometer data improves the performance. Use of data from smartphone sensors also enhances the performance results and the final accuracy with the best parameter combinations is observed to be 98%.

Keywords

• Deep learning
• Sensors
• Activity recognition

References

1. [1] Bulling A, Blanke U, Schiele B. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 2014; 46 (3): 33. doi: 10.1145/2499621
2. [2] Shoaib M, Bosch S, Incel OD, Scholten H, Havinga P. A survey of online activity recognition using mobile phones. Sensors 2015; 15 (1): 2059-2085. doi: 10.3390/s150102059
3. [3] Gjoreski H, Lustrek M, Gams M. Accelerometer placement for posture recognition and fall detection. In: Intelligent Environments (IE), 7th International Conference on Intelligent Environments; Nottingham, United Kingdom; 2011. pp. 47-54.
4. [4] Agac S, Shoaib M, Durmaz Incel O. Smoking recognition with smartwatch sensors in different postures and impact of user's height. Journal of Ambient Intelligence and Smart Environments. 2020(Preprint):1-23.
5. [5] Shoaib M, Scholten H, Havinga P, Incel O. A hierarchical lazy smoking detection algorithm using smartwatch sensors. In: 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services; Munich, Germany; 2016. pp. 1-6.
6. [6] Ordóñez FJ, Roggen D. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 2016; 16 (1): 115. doi: 10.3390/s16010115
7. [7] Wang J, Chen Y, Hao S, Peng X, Hu L. Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters. 2019. 119: 3-11.
8. [8] Alharbi F, Farrahi K. A convolutional neural network for smoking activity recognition. In: 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services; Ostrava, Czech Republic; 2018. pp. 1-6.

9. [9] Parameswarappa G. Human activity recognition using deep recurrent neural nets, lstm and tensorflow on smartphones. MS, University of Massachusetts Dartmouth, Dartmouth, Massachusetts, USA, 2017.
10. [10] Kwapisz J, Weiss GW, Moore SA. Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newsletter 12.2 (2011): 74-82.
11. [11] San-Segundo R, Blunck H, Moreno-Pimentel J, Stisen A, Gil-Martin M. Robust human activity recognition using smartwatches and smartphones. Engineering Applications of Artificial Intelligence 2018; 72: 190-202.
12. [12] Liu Q, Zhou Z, Shakya SR, Uduthalapally P, Qiao M et al. Smartphone sensor-based activity recognition by using machine learning and deep learning algorithms. International Journal of Machine Learning and Computing 2018; 8 (2): 121-6.
13. [13] Chen Y, Xue Y. A deep learning approach to human activity recognition based on single accelerometer. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics; Hong Kong, China; 2015. pp. 1488-1492.
14. [14] Goodfellow I, Bengio Y, Courville A. (2016). Deep learning. MIT press.
15. [15] Abadi M, et al. Tensorow: A system for large-scale machine learning. In: 12th USENIX symposium on operating systems design and implementation (OSDI 16); Savannah, GA, USA; 2016. pp. 265-283.