Investigation of the Validity and Reliability of Two Smart Bands Selected That Count Steps at Different Walking Speed

Abstract

Whether activity monitor smart wristbands that have become widespread can accurately estimate the step count while walking is a matter of curiosity. The current study aims to analyze the validity and reliability of step count (SC) estimation at normal walking and fast walking pace under controlled conditions of two selected smart wristbands of the leading wearable device vendors, Xiaomi (Mi4) and Huawei (H4). Twenty healthy adult male and twenty healthy adult female were included in the study and analyzed separately. The mean age of male and female participants was 22.25 and 21.62 years, with BMI values of 24.22 kg/m2 and 21.42 kg/m2, respectively. The above-ground walking protocol consisted of four separate five-minute tests: Normal Walking Test, Normal Walking Retest, Fast Walking Test, and Fast Walking Retest. In the study, the analyses were performed by using activity monitor measurements and criterion measurements (the number of steps determined from video recordings), compatibility of test-retest measurement values, error indicators (MPE and MAPE), Intraclass Correlation Coefficients (ICC), and Bland-Altman limits of agreement. According to the current study results, it was revealed that the MAPE values recorded for Mi4 and H4 smart wristbands for both normal and fast walking pace in male and female participants were <5%, which was deemed excellent. According to all analyses, the H4 device was found to be valid and reliable, but according to ICC and Pearson Correlation analyses, the Mi4 device was not found to be valid and reliable at fast walking pace.

Keywords

• Reliability
• Smart Band
• Step Counting
• Validity
• Walking

References

1. Amazon, (2023 Nov 29). "Smart Band" Results. https://www.amazon.com
2. An, H. S., Jones, G. C., Kang, S. K., Welk, G. J., & Lee, J. M. (2017). How valid are wearable physical activity trackers for measuring steps?. European journal of sport science, 17(3), 360–368. https://doi.org/10.1080/17461391.2016.1255261
3. Bassett, D. R., Jr., Toth, L. P., LaMunion, S. R., & Crouter, S. E. (2017). Step Counting: A Review of Measurement Considerations and Health-Related Applications. Sports medicine (Auckland, N.Z.), 47(7), 1303–1315. https://doi.org/10.1007/s40279-016-0663-1
4. Bunn, J. A., Navalta, J. W., Fountaine, C. J., & Reece, J. D. (2018). Current State of Commercial Wearable Technology in Physical Activity Monitoring 2015-2017. International journal of exercise science, 11(7), 503–515. PMC5841672
5. Carlin, T., & Vuillerme, N. (2021). Step and Distance Measurement From a Low-Cost Consumer-Based Hip and Wrist Activity Monitor: Protocol for a Validity and Reliability Assessment. JMIR research protocols, 10(1), e21262. https://doi.org/10.2196/21262
6. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284-290. https://doi.org/10.1037/1040-3590.6.4.284
7. Clemes, S. A., O'Connell, S., Rogan, L. M., & Griffiths, P. L. (2010). Evaluation of a commercially available pedometer used to promote physical activity as part of a national programme. British journal of sports medicine, 44(16), 1178–1183. https://doi.org/10.1136/bjsm.2009.061085
8. CTA (Consumer Technology Association), (2016). Physical Activity Monitoring for Step Counting. (2023 Nov 29) https://www.cta.tech/

9. de la Casa Pérez, A., Latorre Román, P. Á., Muñoz Jiménez, M., Lucena Zurita, M., Laredo Aguilera, J. A., Párraga Montilla, J. A., & Cabrera Linares, J. C. (2022). Is the Xiaomi Mi Band 4 an Accuracy Tool for Measuring Health-Related Parameters in Adults and Older People? An Original Validation Study. International journal of environmental research and public health, 19(3), 1593. https://doi.org/10.3390/ijerph19031593
10. Dobkin, B. H., & Martinez, C. (2018). Wearable Sensors to Monitor, Enable Feedback, and Measure Outcomes of Activity and Practice. Current neurology and neuroscience reports, 18(12), 87. https://doi.org/10.1007/s11910-018-0896-5
11. Dooley, E. E., Golaszewski, N. M., & Bartholomew, J. B. (2017). Estimating Accuracy at Exercise Intensities: A Comparative Study of Self-Monitoring Heart Rate and Physical Activity Wearable Devices. JMIR mHealth and uHealth, 5(3), e34. https://doi.org/10.2196/mhealth.7043
12. Evenson, K. R., Goto, M. M., & Furberg, R. D. (2015). Systematic review of the validity and reliability of consumer-wearable activity trackers. The international journal of behavioral nutrition and physical activity, 12, 159. https://doi.org/10.1186/s12966-015-0314-1
13. Hartung, V., Sarshar, M., Karle, V., Shammas, L., Rashid, A., Roullier, P., Eilers, C., Mäurer, M., Flachenecker, P., Pfeifer, K., & Tallner, A. (2020). Validity of Consumer Activity Monitors and an Algorithm Using Smartphone Data for Measuring Steps during Different Activity Types. International journal of environmental research and public health, 17(24), 9314. https://doi.org/10.3390/ijerph17249314
14. Henriksen, A., Haugen Mikalsen, M., Woldaregay, A. Z., Muzny, M., Hartvigsen, G., Hopstock, L. A., & Grimsgaard, S. (2018). Using Fitness Trackers and Smartwatches to Measure Physical Activity in Research: Analysis of Consumer Wrist-Worn Wearables. Journal of medical Internet research, 20(3), e110. https://doi.org/10.2196/jmir.9157
15. Huawei (2022, Aug 20). Wearables band 4. https://consumer.huawei.com/tr/wearables/band4/
16. IDC (2022, Aug 20). Wearable Devices in the Middle East, Turkey, and Africa: Market Analysis and Vendor Highlights, 2021. https://www.idc.com/research/viewtoc.jsp?containerId=META49036222
17. Johnston, W., Judice, P. B., Molina García, P., Mühlen, J. M., Lykke Skovgaard, E., Stang, J., Schumann, M., Cheng, S., Bloch, W., Brønd, J. C., Ekelund, U., Grøntved, A., Caulfield, B., Ortega, F. B., & Sardinha, L. B. (2020). Recommendations for determining the validity of consumer wearable and smartphone step count: expert statement and checklist of the INTERLIVE network. British journal of sports medicine, 55(14), 780–793. https://doi.org/10.1136/bjsports-2020-103147
18. Kastelic, K., Dobnik, M., Löfler, S., Hofer, C., & Šarabon, N. (2021). Validity, Reliability and Sensitivity to Change of Three Consumer-Grade Activity Trackers in Controlled and Free-Living Conditions among Older Adults. Sensors (Basel, Switzerland), 21(18), 6245. https://doi.org/10.3390/s21186245
19. Shin, G., Jarrahi, M. H., Fei, Y., Karami, A., Gafinowitz, N., Byun, A., & Lu, X. (2019). Wearable activity trackers, accuracy, adoption, acceptance and health impact: A systematic literature review. Journal of biomedical informatics, 93, 103153. https://doi.org/10.1016/j.jbi.2019.103153
20. Silfee, V. J., Haughton, C. F., Jake-Schoffman, D. E., Lopez-Cepero, A., May, C. N., Sreedhara, M., Rosal, M. C., & Lemon, S. C. (2018). Objective measurement of physical activity outcomes in lifestyle interventions among adults: A systematic review. Preventive medicine reports, 11, 74–80. https://doi.org/10.1016/j.pmedr.2018.05.003
21. St Fleur, R. G., St George, S. M., Leite, R., Kobayashi, M., Agosto, Y., & Jake-Schoffman, D. E. (2021). Use of Fitbit Devices in Physical Activity Intervention Studies Across the Life Course: Narrative Review. JMIR mHealth and uHealth, 9(5), e23411. https://doi.org/10.2196/23411
22. Statista (2022, Aug 20). Connected wearable devices worldwide 2016-2022. Published by Federica Laricchia, Feb 14, 2022. https://www.statista.com/statistics/487291/global-connected-wearable-devices/#statisticContainer
23. Steeves, J. A., Tyo, B. M., Connolly, C. P., Gregory, D. A., Stark, N. A., & Bassett, D. R. (2011). Validity and reliability of the Omron HJ-303 tri-axial accelerometer-based pedometer. Journal of physical activity & health, 8(7), 1014–1020. https://doi.org/10.1123/jpah.8.7.1014
24. Straiton, N., Alharbi, M., Bauman, A., Neubeck, L., Gullick, J., Bhindi, R., & Gallagher, R. (2018). The validity and reliability of consumer-grade activity trackers in older, community-dwelling adults: A systematic review. Maturitas, 112, 85–93. https://doi.org/10.1016/j.maturitas.2018.03.016
25. Svarre, F. R., Jensen, M. M., Nielsen, J., & Villumsen, M. (2020). The validity of activity trackers is affected by walking speed: the criterion validity of Garmin Vivosmart® HR and StepWatch™ 3 for measuring steps at various walking speeds under controlled conditions. PeerJ, 8, e9381. https://doi.org/10.7717/peerj.9381
26. Topalidis, P., Florea, C., Eigl, E. S., Kurapov, A., Leon, C. A. B., & Schabus, M. (2021). Evaluation of a Low-Cost Commercial Actigraph and Its Potential Use in Detecting Cultural Variations in Physical Activity and Sleep. Sensors (Basel, Switzerland), 21(11), 3774. https://doi.org/10.3390/s21113774
27. Xiaomi (2022, Aug 20). Mi Smart Band 4. https://www.mi.com/tr/mi-smart-band-4/
28. Xie, J., Wen, D., Liang, L., Jia, Y., Gao, L., & Lei, J. (2018). Evaluating the Validity of Current Mainstream Wearable Devices in Fitness Tracking Under Various Physical Activities: Comparative Study. JMIR mHealth and uHealth, 6(4), e94. https://doi.org/10.2196/mhealth.9754