Human Activity Recognition with Smartwatch Data by using Mahalanobis Distance-Based Outlier Detection and Ensemble Learning Methods

Abstract

Recognition of human activities is part of smart healthcare applications. In this context, the detection of human actions with high accuracy has been a field that has been working for many years. With the increase in the usage of smart devices, smartphones and smartwatches have become the constant equipment of these studies thanks to their internal sensors. Sometimes abnormal data are included in data sets due to the way the data were collected and for reasons arising from the sensors. For this reason, it becomes important to detect outlier data. In this study, step counter and heart rate sensors were used in addition to an accelerometer and gyroscope in order to detect human activities. Afterward, the outliers were detected and cleared with a Mahalanobis distance-based approach. With the aim of achieving a better classification performance, machine learning methods were used by strengthening them with ensemble learning methods. The obtained results showed that step counter, heart rate sensors, and ensemble learning methods positively affect the success of the classification. In addition, it was found that the Mahalanobis distance-based outlier detection method increased the classification accuracy significantly.

Keywords

• activity recognition
• smartwatch
• mahalanobis distance
• ensemble learning
• sensor data

References

1. S. Tian, W. Yang, J.M. Le Grange, P. Wang, W. Huang, and Z Ye, “Smart healthcare: making medical care more intelligent”, Global Health Journal, vol. 3, no. 3, pp. 62-65, 2019. https://doi.org/10.1016/j.glohj.2019.07.001
2. B. Mortazavi, S. Nyamathi, S.I. Lee, T. Wilkerson, H. Ghasemzadeh, and M. Sarrafzadeh, “Near-realistic mobile exergames with wireless wearable sensors”, IEEE Journal of Biomedical and Health Informatics, vol. 18, no. 2, pp. 449-456, 2013. https://doi.org/10.1109/JBHI.2013.2293674
3. M.C. Limas, J.B.O. Meré, F.J.M. de Pisón Ascacibar, and E.P.V. González, “Outlier detection and data cleaning in multivariate non-normal samples: the PAELLA algorithm”, Data Mining and Knowledge Discovery, vol. 9, no. 2, pp. 171-187, 2004. https://doi.org/10.1023/B:DAMI.0000031630.50685.7c
4. M. Gjoreski, H. Gjoreski, M. Luštrek, and M. Gams, “How accurately can your wrist device recognize daily activities and detect falls?” Sensors, vol. 16, no. 6, pp. 800, 2016. https://doi.org/10.3390/s16060800
5. E.A. Sağbaş, and S. Ballı, “Transportation mode detection by using smartphone sensors and machine learning”, Pamukkale University Journal of Engineering Sciences, vol. 22, no. 5, pp. 376-383, 2016. https://doi.org/10.5505/pajes.2015.63308
6. M.C. Kwon, and S. Choi, S. “Recognition of daily human activity using an artificial neural network and smartwatch”, Wireless Communications and Mobile Computing, 2018. https://doi.org/10.1155/2018/2618045
7. R.A. Voicu, C. Dobre, L. Bajenaru, and R.I. Ciobanu, “Human physical activity recognition using smartphone sensors”, Sensors, vol. 19, no. 3, pp. 458, 2019. https://doi.org/10.3390/s19030458
8. S. Ballı, E.A. Sağbaş, and M. Peker, “A mobile solution based on soft computing for fall detection”. In Mobile Solutions and Their Usefulness in Everyday Life, Ed. Sara Paiva, 2019, ch. 14, pp. 275-294.

9. M. Peker, S. Ballı, and E.A. Sağbaş, “Predicting human actions using a hybrid of ReliefF feature selection and kernel-based extreme learning machine”, In Cognitive Analytics: Concepts, Methodologies, Tools, and Applications, Ed.Mehdi Khosrow-Pour , 2020, ch. 17, pp. 307-325.
10. N. Ahmed, J.I. Rafiq, and M.R. Islam, “Enhanced human activity recognition based on smartphone sensor data using hybrid feature selection model”, Sensors, vol. 20, no. 1, pp. 317, 2020. https://doi.org/10.3390/s20010317
11. S.W. Yahaya, A. Lotfi, and M. Mahmud, “Detecting anomaly and its sources in activities of daily living”, SN Computer Science, vol. 2, no. 1, pp. 1-18, 2021. https://doi.org/10.1007/s42979-020-00418-2
12. Y. Chen, and C. Shen, “Performance analysis of smartphone-sensor behavior for human activity recognition” IEEE Access, vol. 5, pp. 3095-3110, 2017. https://doi.org/10.1109/ACCESS.2017.2676168
13. R. Li, H. Li, and W. Shi, “Human activity recognition based on LPA”, Multimedia Tools and Applications, vol. 79, no. 41, pp. 31069-31086, 2020. https://doi.org/10.1007/s11042-020-09150-8
14. M.O. Gani, T. Fayezeen, R.J. Povinelli, R.O. Smith, M. Arif, A.J. Kattan, and S.I. Ahamed, “A light weight smartphone based human activity recognition system with high accuracy”, Journal of Network and Computer Applications, vol. 141, pp. 59-72, 2019. https://doi.org/10.1016/j.jnca.2019.05.001
15. A. Elsts, N. Twomey, R. McConville, and I. Craddock, “Energy-efficient activity recognition framework using wearable accelerometers”, Journal of Network and Computer Applications, vol. 168, pp. 102770, 2020. https://doi.org/10.1016/j.jnca.2020.102770
16. H. Gjoreski, J. Bizjak, M. Gjoreski, and M. Gams, “Comparing deep and classical machine learning methods for human activity recognition using wrist accelerometer”, In Proceedings of the IJCAI 2016 Workshop on Deep Learning for Artificial Intelligence, vol. 10, pp. 970, 2016.
17. A. Ignatov, “Real-time human activity recognition from accelerometer data using Convolutional Neural Networks”, Applied Soft Computing, vol. 62, pp. 915-922, 2018. https://doi.org/10.1016/j.asoc.2017.09.027
18. B. Zhou, J. Yang, and Q. Li, “Smartphone-based activity recognition for indoor localization using a convolutional neural network”, Sensors, vol. 19, no. 3, pp. 621, 2019. https://doi.org/10.3390/s19030621
19. S. Wan, L. Qi, X. Xu, C. Tong, and Z. Gu, “Deep learning models for real-time human activity recognition with smartphones”, Mobile Networks and Applications, vol. 25, no. 2, pp. 743-755, 2020. https://doi.org/10.1007/s11036-019-01445-x
20. X. Zhou, W. Liang, I. Kevin, K. Wang, H. Wang, L.T. Yang, and Q. Jin, “Deep-learning-enhanced human activity recognition for Internet of healthcare things”, IEEE Internet of Things Journal, vol. 7, no. 7 , pp. 6429-6438, 2020. https://doi.org/10.1109/JIOT.2020.2985082
21. M. Altuve, P. Lizarazo, and J. Villamizar, “Human activity recognition using improved complete ensemble EMD with adaptive noise and long short-term memory neural networks”, Biocybernetics and Biomedical Engineering, vol. 40, no. 3, pp. 901-909, 2020. https://doi.org/10.1016/j.bbe.2020.04.007
22. D. Mukherjee, R. Mondal, P.K. Singh, R. Sarkar, and D. Bhattacharjee, “EnsemConvNet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications”, Multimedia Tools and Applications, vol. 79, no. 41, pp. 31663-31690, 2020. https://doi.org/10.1007/s11042-020-09537-7
23. C. Catal, S. Tufekci, E. Pirmit, and G. Kocabag, “On the use of ensemble of classifiers for accelerometer-based activity recognition”, Applied Soft Computing, vol. 37, pp. 1018-1022, 2015. https://doi.org/10.1016/j.asoc.2015.01.025
24. I. Elamvazuthi, L.I. Izhar, and G. Capi, “Classification of human daily activities using ensemble methods based on smartphone inertial sensors”, Sensors, vol. 18, no. 12, pp. 4132, 2018. https://doi.org/10.3390/s18124132
25. S. Balli, E.A. Sağbaş, and M. Peker, “Human activity recognition from smart watch sensor data using a hybrid of principal component analysis and random forest algorithm”, Measurement and Control, vol. 52, no. 1-2, pp. 37-45, 2019. https://doi.org/10.1177/0020294018813692
26. O. Herrera-Alcántara, A.Y. Barrera-Animas, M. González-Mendoza, and F. Castro-Espinoza, “Monitoring student activities with smartwatches: On the academic performance enhancement”, Sensors, vol. 19, no. 7, pp. 1605, 2019. https://doi.org/10.3390/s19071605
27. N. Irvine, C. Nugent, S. Zhang, H. Wang, and W.W. Ng, “Neural network ensembles for sensor-based human activity recognition within smart environments”, Sensors, vol. 20, no. 1, pp. 216, 2020. https://doi.org/10.3390/s20010216
28. S. Brajesh, and I. Ray, “Ensemble approach for sensor-based human activity recognition”, In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pp. 296-300, 2020. https://doi.org/10.1145/3410530.3414352
29. R. Sekiguchi, K. Abe, T. Yokoyama, M. Kumano, and M. Kawakatsu, “Ensemble learning for human activity recognition”, In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pp. 335-339, 2020. https://doi.org/10.1145/3410530.3414346
30. A. Subasi, D.H. Dammas, R.D. Alghamdi, R.A. Makawi, E.A., Albiety, T. Brahimi, and A. Sarirete, “Sensor based human activity recognition using adaboost ensemble classifier”, procedia computer science, vol. 140, pp. 104-111, 2018. https://doi.org/10.1016/j.procs.2018.10.298
31. N. Dwivedi, D.K. Singh, and D.S. Kushwaha, “Orientation invariant skeleton feature (oisf): a new feature for human activity recognition”, Multimedia Tools and Applications, vol. 79, no. 29, pp. 21037-21072, 2020. https://doi.org/10.1007/s11042-020-08902-w
32. S. Balli, and E.A. Sağbaş, “The usage of statistical learning methods on wearable devices and a case study: activity recognition on smartwatches” Advances in statistical methodologies and their application to real problems, Ed. Tsukasa Hakimoto, 2017, ch. 13, pp. 259-277.
33. G.H. John, P. Langley, “Estimating continuous distributions in Bayesian classifiers”, In Proceedings of the Eleventh conference on Uncertainty in artificial intelligence, pp. 338-345, August 1995.
34. B.K. Alsberg, R. Goodacre, J.J. Rowland, and D.B. Kell, “Classification of pyrolysis mass spectra by fuzzy multivariate rule induction-comparison with regression, K-nearest neighbour, neural and decision-tree methods”, Analytica Chimica Acta, vol. 348, no. 1-3, pp. 389-407, 1997. https://doi.org/10.1016/S0003-2670(97)00064-0
35. E.A. Sağbaş, and S. Ballı, “Estimation of Human Activities by Using Wrist Movements”, 3. International Management Information Systems Conference, pp. 52-58, October 2016.
36. S. Ballı, E.A. Sağbaş, “Diagnosis of transportation modes on mobile phone using logistic regression classification”, IET Software, vol. 12, no. 2, pp.142-151, 2018. https://doi.org/10.1049/iet-sen.2017.0035
37. L. Breiman, “Random forests”, Machine learning, vol. 45, no. 1, pp. 5-32, 2001. https://doi.org/10.1023/a:1010933404324
38. S. Ballı, and E.A. Sağbaş, “Classification of Human Motions with Smartwatch Sensors”, Süleyman Demirel University Journal of Natural and Applied Sciences, vol. 21, no. 3, pp. 980-990, 2017. https://doi.org/10.19113/sdufbed.32689
39. L. Rokach, “Ensemble-based classifiers”, Artificial intelligence review, vol. 33, no. 1, pp. 1-39, 2010. https://doi.org/10.1007/s10462-009-9124-7
40. L. Breiman, “Bagging predictors”, Machine learning, vol. 24, no. 2, pp. 123-140, 1996. https://doi.org/10.1007/BF00058655
41. L.I. Kuncheva, “Combining pattern classifiers: methods and algorithms”, John Wiley & Sons, 2014.
42. A. Onan, S. Korukoğlu, and H. Bulut, “Ensemble of keyword extraction methods and classifiers in text classification”, Expert Systems with Applications, vol. 57, pp. 232-247, 2016. https://doi.org/10.1016/j.eswa.2016.03.045
43. E.A. Sağbaş, and S. Balli, “Activity recognition by voting method via motion sensors”, In 2017 International Conference on Computer Science and Engineering (UBMK), 2017. https://doi.org/10.1109/UBMK.2017.8093557
44. E.A. Sağbaş, and S. Ballı, “Usage of the smartphone sensors and accessing raw sensor data”, Academic Computing Conferences, pp. 180-186, 2015.
45. P.C. Mahalanobis, “On tests and measures of groups divergence”, IJ Asiatic Soc. Bengal, vol. 26, pp. 541, 1930.
46. C. Leys, O. Klein, Y. Dominicy, and C. Ley, “Detecting multivariate outliers: Use a robust variant of the Mahalanobis distance”, Journal of Experimental Social Psychology, vol. 74, pp. 150-156, 2018. https://doi.org/10.1016/j.jesp.2017.09.011
47. K. Polat, and S. Güneş, “Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform”, Applied Mathematics and Computation, vol. 187, no. 2, pp. 1017-1026, 2007. https://doi.org/10.1016/j.amc.2006.09.022
48. I.H. Witten, and E. Frank, “Data mining: practical machine learning tools and techniques with Java implementations”, Acm Sigmod Record, vol. 31, no.1, pp. 76-77, 2002.