Development of a Smart Activity Recognition System with Transfer Learning Based Deep Learning Models for Elderly Care

Abstract

In recent years, smart healthcare services have become popular in scientific research trends. Elderly care is a major topic in this services. Fall detection and activity recognition of elderly person living alone in their house or in a nursing home are vitally important. Because, falls are primary cause of most of the injuries, traumas, need of care and even deaths. To find a solution to this issue, scientists are start to use Artificial Intelligence. In this study, an intelligent activity recognition and fall detection system based on Convolutional Neural Network was developed. To develop this system an original dataset was created. By the proposed system, time distributions and classes of the activities are observed. When a fall is detected, the system gives an alert and warns relevant persons. The performances of used different models were compared using the dataset we created. To evaluate the performance of the systems, accuracy, precision, recall and F1 score metrics was used. For the ResNet101, these metrics are obtained as 98.66%, 98.54%, 98.78%, 98.66% respectively, that is the best of all scores. This results show that trained ResNet101 system can be used to help elderly persons and can be integrated to the other IoT systems.

Keywords

• Deep Learning
• Elderly Care
• Fall Detection
• Human Activity Recognition

Supporting Institution

Scientific Research Projects Coordination Unit of Fırat University (FÜBAP) under the project with protocol number ADEP.22.06

Project Number

Scientific Research Projects Coordination Unit of Fırat University (FÜBAP) under the project with protocol number ADEP.22.06

Ethical Statement

The authors declare that they have no conflicts of interest.

Thanks

Scientific Research Projects Coordination Unit of Fırat University (FÜBAP) under the project with protocol number ADEP.22.06

References

1. [1] S. B. Atitallah, M. Driss, W. Boulila, and H. B. Ghézala, “Leveraging Deep Learning and IoT big data analytics to support the smart cities development: Review and future directions,” Comput. Sci. Rev., vol. 38, p. 100303, Nov. 2020, doi: 10.1016/j.cosrev.2020.100303.
2. [2] “Statista, UK: people living alone, 2019, https://www.statista.com/statistics/281616/people-living-alone-in-the-united-kingdom-uk-by- age-and-gender/ (Accessed 5 December 2023).”.
3. [3] “World Health Organization, Falls.” [Online]. Available: World Health Organization https://www.who.int/news-room/fact-sheets/detail/falls
4. [4] “Global Pharma News & Resources.” [Online]. Available: https://www.pharmiweb.com/press-release/2020-05-05/fall-detection-system-market-to-register-cagr-4-growth-in-revenue-during-the-forecast-period-2019-t
5. [5] Bureau of Labor Statistics (2023) https://data.bls.gov/timeseries/FWU00X4XXXXX8EN00
6. [6] M. A. Khan, F. Algarni, and M. T. Quasim, Smart Cities: A Data Analytics Perspective. Springer, 2021.
7. [7] N. Zerrouki, F. Harrou, Y. Sun, A. Z. A. Djafer, and H. Amrane, “A Survey on Recent Advances in Fall Detection Systems Using Machine Learning Formalisms,” in 2022 7th International Conference on Frontiers of Signal Processing (ICFSP), IEEE, 2022, pp. 35–39.
8. [8] R. E, T. Perumal, and S. K, “A review on fall detection systems in bathrooms: challenges and opportunities,” Multimed. Tools Appl., Jan. 2024, doi: 10.1007/s11042-023-18088-6.

9. [9] Md. M. Islam et al., “Deep Learning Based Systems Developed for Fall Detection: A Review,” IEEE Access, vol. 8, pp. 166117–166137, 2020, doi: 10.1109/ACCESS.2020.3021943.
10. [10] A. Purwar and I. Chawla, “A systematic review on fall detection systems for elderly healthcare,” Multimed. Tools Appl., vol. 83, no. 14, pp. 43277–43302, Oct. 2023, doi: 10.1007/s11042-023-17190-z.
11. [11] X. Zhou, L.-C. Qian, P.-J. You, Z.-G. Ding, and Y.-Q. Han, “Fall detection using convolutional neural network with multi-sensor fusion,” in 2018 IEEE international conference on Multimedia & Expo Workshops (ICMEW), IEEE, 2018, pp. 1–5.
12. [12] J. Maitre, K. Bouchard, and S. Gaboury, “Fall Detection With UWB Radars and CNN-LSTM Architecture,” IEEE J. Biomed. Health Inform., vol. 25, no. 4, pp. 1273–1283, Apr. 2021, doi: 10.1109/JBHI.2020.3027967.
13. [13] G. L. Santos, P. T. Endo, K. H. de C. Monteiro, E. da S. Rocha, I. Silva, and T. Lynn, “Accelerometer-based human fall detection using convolutional neural networks,” Sensors, vol. 19, no. 7, p. 1644, 2019.
14. [14] E. Torti et al., “Embedded real-time fall detection with deep learning on wearable devices,” in 2018 21st euromicro conference on digital system design (DSD), IEEE, 2018, pp. 405–412.
15. [15] T. R. Mauldin, M. E. Canby, V. Metsis, A. H. Ngu, and C. C. Rivera, “SmartFall: A smartwatch-based fall detection system using deep learning,” Sensors, vol. 18, no. 10, p. 3363, 2018.
16. [16] A. Nait Aicha, G. Englebienne, K. S. Van Schooten, M. Pijnappels, and B. Kröse, “Deep learning to predict falls in older adults based on daily-life trunk accelerometry,” Sensors, vol. 18, no. 5, p. 1654, 2018.
17. [17] W.-N. Lie, A. T. Le, and G.-H. Lin, “Human fall-down event detection based on 2D skeletons and deep learning approach,” in 2018 International workshop on advanced image technology (IWAIT), IEEE, 2018, pp. 1–4.
18. [18] A. Shojaei-Hashemi, P. Nasiopoulos, J. J. Little, and M. T. Pourazad, “Video-based human fall detection in smart homes using deep learning,” in 2018 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2018, pp. 1–5.
19. [19] N. Lu, Y. Wu, L. Feng, and J. Song, “Deep learning for fall detection: Three-dimensional CNN combined with LSTM on video kinematic data,” IEEE J. Biomed. Health Inform., vol. 23, no. 1, pp. 314–323, 2018.
20. [20] W. Min, H. Cui, H. Rao, Z. Li, and L. Yao, “Detection of human falls on furniture using scene analysis based on deep learning and activity characteristics,” IEEE Access, vol. 6, pp. 9324–9335, 2018.
21. [21] K. Adhikari, H. Bouchachia, and H. Nait-Charif, “Deep learning based fall detection using simplified human posture,” Int J Comput Syst Eng, vol. 13, no. 5, pp. 255–260, 2019.
22. [22] Y. Jiang, T. Gong, L. He, S. Yan, X. Wu, and J. Liu, “Fall detection on embedded platform using infrared array sensor for healthcare applications,” Neural Comput. Appl., vol. 36, no. 9, pp. 5093–5108, Mar. 2024, doi: 10.1007/s00521-023-09334-x.
23. [23] S. Mobsite, N. Alaoui, M. Boulmalf, and M. Ghogho, “Activity Classification and Fall Detection Using Monocular Depth and Motion Analysis,” IEEE Access, vol. 12, pp. 1525–1541, 2024, doi: 10.1109/ACCESS.2023.3348413.
24. [24] X. Zi, K. Chaturvedi, A. Braytee, J. Li, and M. Prasad, “Detecting Human Falls in Poor Lighting: Object Detection and Tracking Approach for Indoor Safety,” Electronics, vol. 12, no. 5, p. 1259, Mar. 2023, doi: 10.3390/electronics12051259.
25. [25] A. R. Khekan, H. S. Aghdasi, and P. Salehpour, “The Impact of YOLO Algorithms Within Fall Detection Application: A Review,” IEEE Access, vol. 13, pp. 6793–6809, 2025, doi: 10.1109/ACCESS.2024.3496823.
26. [26] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255.
27. [27] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
28. [28] F. Chollet, “Xception: Deep Learning with Depthwise Separable Convolutions,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1800–1807. doi: 10.1109/CVPR.2017.195.
29. [29] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.
30. [30] A. G. Howard et al., “MobileNets: efficient convolutional neural networks for mobile vision applications (2017),” ArXiv Prepr. ArXiv170404861, vol. 126, 2017.
31. [31] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510–4520.
32. [32] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708.
33. [33] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in International conference on machine learning, PMLR, 2019, pp. 6105–6114.
34. [34] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11976–11986.
35. [35] H. Fırat and H. Üzen, “Detection of Pneumonia Using A Hybrid Approach Consisting of MobileNetV2 and Squeeze-and-Excitation Network,” Türk Doğa Ve Fen Derg., vol. 13, no. 1, pp. 54–61, 2024.
36. [36] A. Bello, S.-C. Ng, and M.-F. Leung, “Skin Cancer Classification Using Fine-Tuned Transfer Learning of DENSENET-121,” Appl. Sci., vol. 14, no. 17, p. 7707, 2024.
37. [37] Ü. Hüseyin and H. FIRAT, “ÖZNİTELİK ENTEGRASYONUNA DAYALI ESA MİMARİSİ KULLANILARAK ENDOSKOPİK GÖRÜNTÜLERİN SINIFLANDIRILMASI,” Kahramanmaraş Sütçü İmam Üniversitesi Mühendis. Bilim. Derg., vol. 27, no. 1, pp. 121–132, 2024.
38. [38] A. R. KP and S. Gowrishankar, “Convnext-based mango leaf disease detection: Differentiating pathogens and pests for improved accuracy,” Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 6, 2023.