DeepTherapy: A mobile platform for osteoarthritis rehabilitation utilizing chain-of-thought reasoning and deep learning

Abstract

Objectives: To develop and evaluate an AI-driven mobile platform that integrates deep learning-based exercise analysis with large language model (LLM) feedback for enhancing osteoarthritis (OA) rehabilitation accessibility and effectiveness.

Methods: A deep learning framework was developed using Long Short-Term Memory (LSTM) architecture to classify exercise phases from video data of 10 rehabilitation exercises. The dataset consisted of approximately 800,000 frames collected from 20 healthy volunteers. A feedback system utilizing chain-of-thought reasoning in LLMs (GPT-4o and Claude 3.5 Sonnet) was implemented to generate targeted corrective feedback. Evaluation was conducted with OA patients (n=2) and physiotherapists (n=7) using the Intraclass Correlation Coefficient (ICC) and Likert scales.

Results: The developed LSTM models achieved 97.8% accuracy in exercise phase classification. Strong agreement between system-generated scores and expert evaluations was demonstrated (ICC=0.85). Physiotherapists slightly preferred Claude's outputs (52.4% vs 47.6%) but rated GPT-4o higher on clinical relevance (4.57/5 vs 4.13/5), clarity (4.71/5 vs 4.38/5), and helpfulness (4.50/5 vs 4.29/5).

Conclusions: DeepTherapy effectively addresses critical limitations in rehabilitation monitoring by providing qualitative movement assessment, identifying incorrect movements, and offering detailed guidance on technique improvement, potentially increasing rehabilitation accessibility while maintaining quality of care.

Keywords

• Osteoarthritis
• rehabilitation
• deep learning
• large language models
• chain-of-thought reasoning
• mobile health

References

1. 1. World Health Organization. Osteoarthritis. 2023 Jul 14. Available at: https://www.who.int/news-room/fact-sheets/detail/osteoarthritis.
2. 2. Losina E, Paltiel AD, Weinstein AM, et al. Lifetime medical costs of knee osteoarthritis management in the United States: impact of extending indications for total knee arthroplasty. Arthritis Care Res (Hoboken). 2015;67(2):203-215. doi: 10.1002/acr.22412.
3. 3. World Health Organization. Rehabilitation Fact sheet. WHO Newsroom. 2024 Apr 22. Available at: https://www.who.int/news-room/fact-sheets/detail/rehabilitation.
4. 4. Mrklas KJ, Barber T, Campbell-Scherer D, et al. Co-Design in the Development of a Mobile Health App for the Management of Knee Osteoarthritis by Patients and Physicians: Qualitative Study. JMIR Mhealth Uhealth. 2020;8(7):e17893. doi: 10.2196/17893.
5. 5. Dieter V, Janssen P, Krauss I. Efficacy of the mHealth-Based Exercise Intervention re.flex for Patients With Knee Osteoarthritis: Pilot Randomized Controlled Trial. JMIR Mhealth Uhealth. 2024;12:e54356. doi: 10.2196/54356.
6. 6. Beresford L, Norwood T. The Effect of Mobile Care Delivery on Clinically Meaningful Outcomes, Satisfaction, and Engagement Among Physical Therapy Patients: Observational Retrospective Study. JMIR Rehabil Assist Technol. 2022;9(1):e31349. doi: 10.2196/31349.
7. 7. Sun S, Simonsson O, McGarvey S, Torous J, Goldberg SB. Mobile phone interventions to improve health outcomes among patients with chronic diseases: an umbrella review and evidence synthesis from 34 meta-analyses. Lancet Digit Health. 2024;6(11):e857-e870. doi: 10.1016/S2589-7500(24)00119-5.
8. 8. Goh SL, Persson MSM, Stocks J, et al. Efficacy and potential determinants of exercise therapy in knee and hip osteoarthritis: a systematic review and meta-analysis. Ann Phys Rehabil Med. 2019;62(5):356-365. doi: 10.1016/j.rehab.2019.04.006.

9. 9. Bevilacqua A, Ciampi G, Argent R, Caulfield B, Kechadi T. Combining real-time segmentation and classification of rehabilitation exercises with LSTM networks and pointwise boosting. Proceedings of the AAAI Conference on Artificial Intelligence. 2020;34(8):13229-13234. doi: 10.1609/aaai.v34i08.7028.
10. 10. Krutz P, Rehm M, Lang Z, Dix M, Patalas-Maliszewska J. Classification of Sports Exercises and Repetition Counting based on Inertial Measurement Data. Sensors and Electronic Instrumentation Advances: Proceedings of the 9th International Conference on Sensors and Electronic Instrumentation Advances, 20-22 September 2023 Funchal (Madeira Island), Portugal, 2023, pp. 35-39. https://sensorsportal.com/SEIA_2023/SEIA_2023_Proceedings.pdf
11. 11. Whittaker JL, Truong LK, Dhiman K, Beck C. Osteoarthritis year in review 2020: rehabilitation and outcomes. Osteoarthritis Cartilage. 2021;29(1):21-30. doi: 10.1016/j.joca.2020.10.005.
12. 12. Runhaar J, Holden MA, Hattle M, et al; STEER OA Patient Advisory Group; OA Trial Bank Exercise Collaborative. Mechanisms of action of therapeutic exercise for knee and hip OA remain a black box phenomenon: an individual patient data mediation study with the OA Trial Bank. RMD Open. 2023;9(3):e003220. doi: 10.1136/rmdopen-2023-003220.
13. 13. Adans-Dester C, Hankov N, O'Brien A, et al. Enabling precision rehabilitation interventions using wearable sensors and machine learning to track motor recovery. NPJ Digit Med. 2020;3(1):121. doi: 10.1038/s41746-020-00328-w.
14. 14. Zhu Y, Wang C, Li J, Zeng L, Zhang P. Effect of different modalities of artificial intelligence rehabilitation techniques on patients with upper limb dysfunction after stroke-A network meta-analysis of randomized controlled trials. Front Neurol. 2023;14:1125172. doi: 10.3389/fneur.2023.1125172.
15. 15. Chen J, Wang J, Yuan Q, Yang Z. CNN-LSTM Model for Recognizing Video-Recorded Actions Performed in a Traditional Chinese Exercise. IEEE J Transl Eng Health Med. 2023;11:351-359. doi: 10.1109/JTEHM.2023.3282245.
16. 16. Rahman ZU, Ullah SI, Salam A, Rahman T, Khan I, Niazi B. Automated Detection of Rehabilitation Exercise by Stroke Patients Using 3-Layer CNN-LSTM Model. J Healthc Eng. 2022;2022:1563707. doi: 10.1155/2022/1563707.
17. 17. Ren B, Zhang Z, Chi Z, Chen S. Motion Trajectories Prediction of Lower Limb Exoskeleton Based on Long Short-Term Memory (LSTM) Networks. Actuators 2022;11(3):73. doi: 10.3390/act11030073.
18. 18. Görgülü YE, Tasdelen K. Human Activity Recognition and Temporal Action Localization Based on Depth Sensor Skeletal Data. 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), Istanbul, Turkey, 2020, pp. 1-5, doi: 10.1109/ASYU50717.2020.9259886.
19. 19. He J, Guo Z, Shao Z, Zhao J, Dan G. An LSTM-Based Prediction Method for Lower Limb Intention Perception by Integrative Analysis of Kinect Visual Signal. J Healthc Eng. 2020;2020:8024789. doi: 10.1155/2020/8024789.
20. 20. Liang F-Y, Zhong CH, Zhao X, et al. Online Adaptive and LSTM-Based Trajectory Generation of Lower Limb Exoskeletons for Stroke Rehabilitation. 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia, 2018, pp. 27-32. doi: 10.1109/ROBIO.2018.8664778.
21. 21. J. Sun, Y. Gong, C. Lin, Y. Shen, J. Guo, and N. Duan, "Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models," arXiv.org, Apr. 2023, doi: 10.48550/arXiv.2304.11657.
22. 22. Zhang Z, Zhang A, Li M, Smola AJ. Automatic Chain of Thought Prompting in Large Language Models. arXiv: 2210.03493, Oct 2022. doi: 10.48550/arXiv.2210.03493.
23. 23. Nguyen HH, Liu Y, Zhang C, Zhang T, Yu PS. CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks. arXiv: 2310.14623, Oct 2023, doi: 10.48550/arxiv.2310.14623.
24. 24. Villagrán I, Hernández R, Schuit G, et al. Implementing Artificial Intelligence in Physiotherapy Education: A Case Study on the Use of Large Language Models (LLM) to Enhance Feedback. IEEE Trans Learn Technol. 2024;17:2025-2036. doi: 10.1109/TLT.2024.3450210.
25. 25. Nachane SS, Gramopadhye O, Chanda P, et al. Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering. arXiv:2403.04890, Mar 2024. doi: 10.48550/arxiv.2403.04890.
26. 26. Masikisiki B, Marivate V, Hlope Y. Investigating the Efficacy of Large Language Models in Reflective Assessment Methods through Chain of Thoughts Prompting. arXiv:2310.00272, Sep 2023, doi: 10.48550/arXiv.2310.00272.
27. 27. Zou A, Zhang Z, Zhao H, Tang X. Meta-CoT: Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models. arXiv: 2310.06692, Oct 2023. doi: 10.48550/arxiv.2310.06692.
28. 28. Cheng X, Li J, Zhao WX, Wen J-R. ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting. arXiv:2403.14312, Mar 2024. doi: 10.48550/arXiv.2403.14312.
29. 29. Mahmoud H, Aljaldi F, El-Fiky A, et al. Artificial Intelligence machine learning and conventional physical therapy for upper limb outcome in patients with stroke: a systematic review and meta-analysis. Eur Rev Med Pharmacol Sci. 2023;27(11):4812-4827. doi: 10.26355/eurrev_202306_32598.
30. 30. Senadheera I, Hettiarachchi P, Haslam B, et al. AI Applications in Adult Stroke Recovery and Rehabilitation: A Scoping Review Using AI. Sensors (Basel). 2024;24(20):6585. doi: 10.3390/s24206585.
31. 31. Maceira-Elvira P, Popa T, Schmid AC, Hummel FC. Wearable technology in stroke rehabilitation: towards improved diagnosis and treatment of upper-limb motor impairment. J Neuroeng Rehabil. 2019;16(1):142. doi: 10.1186/s12984-019-0612-y.
32. 32. Panwar M, Biswas D, Bajaj H, et al. Rehab-Net: Deep Learning Framework for Arm Movement Classification Using Wearable Sensors for Stroke Rehabilitation. IEEE Trans Biomed Eng. 2019;66(11):3026-3037. doi: 10.1109/TBME.2019.2899927.
33. 33. Argent R, Slevin P, Bevilacqua A, Neligan M, Daly A, Caulfield B. Wearable Sensor-Based Exercise Biofeedback for Orthopaedic Rehabilitation: A Mixed Methods User Evaluation of a Prototype System. Sensors (Basel). 2019;19(2):432. doi: 10.3390/s19020432.
34. 34. De Fazio R, Mastronardi VM, De Vittorio M, Visconti P. Wearable Sensors and Smart Devices to Monitor Rehabilitation Parameters and Sports Performance: An Overview. Sensors (Basel). 2023;23(4):1856. doi: 10.3390/s23041856.
35. 35. Kristoffersson A, Linden M. A Systematic Review of Wearable Sensors for Monitoring Physical Activity. Sensors (Basel). 2022;22(2):573. doi: 10.3390/s22020573.
36. 36. Pong C, Tseng RMWW, Tham YC, Lum E. Current Implementation of Digital Health in Chronic Disease Management: Scoping Review. J Med Internet Res. 2024;26:e53576. doi: 10.2196/53576.
37. 37. Kyaw TL, Ng N, Theocharaki M, Wennberg P, Sahlen KG. Cost-effectiveness of Digital Tools for Behavior Change Interventions Among People With Chronic Diseases: Systematic Review. Interact J Med Res. 2023;12:e42396. doi: 10.2196/42396.
38. 38. Papadopoulos P, Soflano M, Connolly T. A Digital Health Intervention Platform (Active and Independent Management System) to Enhance the Rehabilitation Experience for Orthopedic Joint Replacement Patients: Usability Evaluation Study. JMIR Hum Factors. 2024;11:e50430. doi: 10.2196/50430.
39. 39. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155-63. doi: 10.1016/j.jcm.2016.02.012.