A review of mobile health application rating scales

Abstract

Nowadays, mobile devices have evolved to solve almost every problem and have become an indispensable part of our daily lives. According to statistics, users spend an average of three to five hours on their smartphones each day, and approximately ninety percent of this time is spent on applications. As in other sectors, mobile health applications (mHealth apps.) are increasingly utilized. The high demand in this regard naturally causes the rapid proliferation of mHealth applications, and it is becoming more and more challenging for both patients and healthcare professionals to identify superior applications due to the vast selection available in stores, varying in quality, reliability, and adherence to best practices in healthcare. In this context, various surveys and frameworks such as MARS, uMARS, ORCHA-24, ENLIGHT, THESIS, and ACCU3RATE have been proposed to provide a systematic and standardized approach. In this study, a review of proposed rating scales will be conducted.

Keywords

• e-health
• mobile health (mHealth)
• health applications
• scale development
• app evaluation

References

1. Amor-García, M. Á., Collado-Borrell, R., Escudero-Vilaplana, V., Melgarejo-Ortuño, A., Herranz-Alonso, A., Arija, J. Á. A., and Sanjurjo-Sáez, M. (2020). Assessing apps for patients with genitourinary tumors using the mobile application rating scale (MARS): systematic search in app stores and content analysis. JMIR mHealth and uHealth, 8(7):e17609. https://doi.org/10.2196/17609
2. Azad-Khaneghah, P., Neubauer, N., Miguel Cruz, A., and Liu, L. (2021). Mobile health app usability and quality rating scales: a systematic review. Disability and Rehabilitation: Assistive Technology, 16(7):712–721. https://doi.org/10.1080/17483107.2019.1701103
3. Aydoğdu, Y.Ö., Durak, H.Y., And Akgün, E. (2023). Investigating the validity and reliability of the mobile application rating scale. Bartın University Journal of Faculty of Education, 12(3):570–578. https://doi.org/10.14686/buefad.1274394
4. Bardus, M., Awada, N., Ghandour, L. A., Fares, E.-J., Gherbal, T., Al-Zanati, T., and Stoyanov, S. R. (2020). The Arabic version of the mobile app rating scale: development and validation study. JMIR mHealth and uHealth, 8(3):e16956. https://doi.org/10.2196/16956
5. Bardus, M., Van Beurden, S. B., Smith, J. R., and Abraham, C. (2016). A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management. International Journal of Behavioral Nutrition and Physical Activity, 13(1):1–9. https://doi.org/10.1186/s12966-016-0359-9
6. Barzegari, S., Sharifi Kia, A., Barzegari, M., Stoyanov, S. R., GhaziSaeedi, M., and Rafizadeh, M. (2022). The Persian version of the mobile application rating scale (MARS-fa): Translation and validation study. JMIR Formative Research, 6(12):e42225. https://doi.org/10.2196/42225
7. Baumel, A., Faber, K., Mathur, N., Kane, J. M., and Muench, F. (2017). Enlight: a comprehensive quality and therapeutic potential evaluation tool for mobile and web-based ehealth interventions. Journal of Medical Internet Research, 19(3):e82. https://doi.org/10.2196/jmir.7270
8. Biswas, M., Tania, M. H., Kaiser, M. S., Kabir, R., Mahmud, M., and Kemal, A. A. (2021). Accu3rate: A mobile health application rating scale based on user reviews. PLOS ONE, 16(12):e0258050. https://doi.org/10.1371/journal.pone.0258050

9. Calik, G., Kartal, B. B., Stoyanov, S., Gravas, S., Othman, L., de la Rosette, J., Albayrak, S., and Laguna, P. (2022). Turkish validation of the user version of the mobile application rating scale. Turkish Journal of Urology, 48(3):236–242. https://doi.org/10.5152/tud.2022.21324
10. Creber, R. M. M., Maurer, M. S., Reading, M., Hiraldo, G., Hickey, K. T., and Iribarren, S. (2016). Review and analysis of existing mobile phone apps to support heart failure symptom monitoring and self-care management using the mobile application rating scale (MARS). JMIR mHealth and uHealth, 4(2):e5882. https://doi.org/10.2196/mhealth.5882
11. Davalbhakta, S., Advani, S., Kumar, S., Agarwal, V., Bhoyar, S., Fedirko, E., Misra, D. P., Goel, A., Gupta, L., and Agarwal, V. (2020). A systematic review of smartphone applications available for corona virus disease 2019 (COVID-19) and the assessment of their quality using the mobile application rating scale (MARS). Journal of Medical Systems, 44:1–15. https://doi.org/10.1007/s10916-020-01633-3
12. Dilek, U., and Arikan, G. (2023). Engelsiz sağlık iletişim merkezi mobil uygulamasının sistem kullanılabilirlik değerlendirilmesi. Ankara Hacı Bayram Veli Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 25(2):567–588. https://doi.org/10.30913/ahbvu.1190765
13. Domnich, A., Arata, L., Amicizia, D., Signori, A., Patrick, B., Stoyanov, S., Hides, L., Gasparini, R., and Panatto, D. (2016). Development and validation of the Italian version of the mobile application rating scale and its generalizability to apps targeting primary prevention. BMC Medical Informatics and Decision Making, 16(1):1–10. https://doi.org/10.1186/s12911-016-0373-0
14. Duman, S., Tanrıklulu, G., and Demirel, B. (2022). Mindfulness mobile app user quality evaluation: MARS scale adaptation. 11th International Conference on Culture and Civilization (pp. 134–139). Mardin, Turkey. EPI. (2023). EF EPI, The world’s largest ranking of countries and regions by English skills. Accessed: 2023-12-13.
15. Grainger, R., Townsley, H., White, B., Langlotz, T., Taylor, W. J., et al. (2017). Apps for people with rheumatoid arthritis to monitor their disease activity: a review of apps for best practice and quality. JMIR mHealth and uHealth, 5(2):e6956. https://doi.org/10.2196/mhealth.6956
16. Hee Ko, K. K., Kim, S. K., Lee, Y., Lee, J. Y., and Stoyanov, S. R. (2022). Validation of a Korean version of mobile app rating scale (MARS) for apps targeting disease management. Health Informatics Journal, 28(2):14604582221091975. https://doi.org/10.1177/14604582221091975
17. Kay, M., Santos, J., and Takane, M. (2011). mHealth: New horizons for health through mobile technologies. World Health Organization, 64(7):66–71. Knitza, J., Tascilar, K., Messner, E.-M., Meyer, M., Vossen, D., Pulla, A., Bosch, P., Kittler, J., Kleyer, A., Sewerin, P., et al. (2019). German mobile apps in rheumatology: review and
18. analysis using the mobile application rating scale (MARS). JMIR mHealth and uHealth, 7(8):e14991. https://doi.org/10.2196/14991
19. Korkmaz, S., and Arikan, G. (2021). e-nabız uygulamasını değerlendirmek için kullanılan yeni bir araç: Mobil uygulama derecelendirme ölçeği. Ankara Hacı Bayram Veli Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 23(3):625–636. https://doi.org/10.30913/ahbvu.982160
20. Leigh, S., Ouyang, J., and Mimnagh, C. (2017). Effective? Engaging? Secure? Applying the ORCHA-24 framework to evaluate apps for chronic insomnia disorder. BMJ Ment Health, 20(4):e20–e20. https://doi.org/10.1136/eb-2017-102767 Levine, D. M., Co, Z., Newmark, L. P., Groisser, A. R., Holmgren, A. J., Haas, J. S., and Bates, D. W. (2020). Design and testing of a mobile health application rating tool. NPJ Digital Medicine, 3(1):74. https://doi.org/10.1038/s41746-020-0276-0 Machado, G. C., Pinheiro, M. B., Lee, H., Ahmed, O. H., Hendrick, P., Williams, C., and Kamper, S. J. (2016). Smartphone apps for the self-management of low back pain: a systematic review. Best Practice & Research Clinical Rheumatology, 30(6):1098–1109. https://doi.org/10.1016/j.berh.2017.04.002
21. Mani, M., Kavanagh, D. J., Hides, L., and Stoyanov, S. R., et al. (2015). Review and evaluation of mindfulness-based iPhone apps. JMIR mHealth and uHealth, 3(3):e4328. https://doi.org/10.2196/mhealth.4328 Martin-Payo, R., Carrasco-Santos, S., Cuesta, M., Stoyan, S., Gonzalez-Mendez, X., and Fernandez-Alvarez, M. d. M. (2021). Spanish adaptation and validation of the user version of the mobile application rating scale (uMARS). Journal of the American Medical Informatics Association, 28(12):2681–2686. https://doi.org/10.1093/jamia/ocab201
22. Mendi, O., Sari, M. K., Stoyanov, S., and Mendi, B. (2022). Development and validation of the Turkish version of the mobile app rating scale MARS-TR. International Journal of Medical Informatics, 166:104843. https://doi.org/10.1016/j.ijmedinf.2022.104843 Messner, E.-M., Terhorst, Y., Barke, A., Baumeister, H., Stoyanov, S., Hides, L., Kavanagh, D., Pryss, R., Sander, L., Probst, T., et al. (2020). The German version of the mobile app rating scale (MARS-G): development and validation study. JMIR mHealth and uHealth, 8(3):e14479. https://doi.org/10.2196/14479
23. Miro, J., Llorens-Vernet, P., et al. (2021). Assessing the quality of mobile health-related apps: interrater reliability study of two guides. JMIR mHealth and uHealth, 9(4):e26471. https://doi.org/10.2196/26471
24. Payo, R. M., Álvarez, M. F., Díaz, M. B., Izquierdo, M. C., Stoyanov, S. R., and Suárez, E. L. (2019). Spanish adaptation and validation of the mobile application rating scale questionnaire. International Journal of Medical Informatics, 129:95–99. https://doi.org/10.1016/j.ijmedinf.2019.06.004
25. Salazar, A., de Sola, H., Failde, I., Moral-Munoz, J. A., et al. (2018). Measuring the quality of mobile apps for the management of pain: systematic search and evaluation using the mobile app rating scale. JMIR mHealth and uHealth, 6(10):e10718. https://doi.org/10.2196/10718
26. Shinohara, Y., Yamamoto, K., Ito, M., Sakata, M., Koizumi, S., Hashisako, M., Sato, M., Wannous, M., Stoyanov, S. R., Nakajima, J., et al. (2022). Development and validation of the Japanese version of the uMARS (user version of the mobile app rating system). International Journal of Medical Informatics, 165:104809. https://doi.org/10.1016/j.ijmedinf.2022.104809
27. Statista. (2024). Number of hours spent per day using apps worldwide from 2019 to 2023, by country (in hours). Retrieved 10.02.24 from https://www.statista.com/statistics/1269704/time-spent-mobile-apps-worldwide/.
28. Statista. (2024). Number of mHealth apps available in the Apple App Store from 1st quarter 2015 to 3rd quarter 2022. Retrieved 10.02.24 from https://www.statista.com/statistics/779910/health-apps-available-ios-worldwide/.
29. Statista. (2024). Number of mHealth apps available in the Google Play Store from 1st quarter 2015 to 3rd quarter 2022. Retrieved 10.02.24 from https://www.statista.com/statistics/779919/health-apps-available-google-play-worldwide/.
30. Stifani, B. M., Peters, M., French, K., and Gill, R. K. (2023). There’s an app for it: A systematic review of mobile apps providing information about abortion using a revised MARS scale. PLOS Digital Health, 2(7):e0000277. https://doi.org/10.1371/journal.pdig.0000277
31. Stoyanov, S. R., Hides, L., Kavanagh, D. J., and Wilson, H. (2016). Development and validation of the user version of the mobile application rating scale (uMARS). JMIR mHealth and uHealth, 4(2):e5849. https://doi.org/10.2196/mhealth.5849
32. Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., and Mani, M. (2015). Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth and uHealth, 3(1):e3422. https://doi.org/10.2196/mhealth.