Human Robot Interaction Network Design with Wearable Wireless MIMU Sensors for Upper Extremity Exoskeleton Robot

Year 2020, Volume: 20 Issue: 6, 1165 - 1177, 31.12.2020

Beyda Taşar , Oğuz Yakut , Sait Müftü , Şakir Furkan Yöndem Hakan Durmuş

https://doi.org/10.35414/akufemubid.606874

Abstract

Within the scope of this research, human robot interaction network design was carried out by means of wearable wireless sensors MIMU (accelerometer, gyroscope, magnetometer) for the control of a two-degree upper-extremity exoskeletal robot system compatible with human body and supporting human movements. Angular acceleration, gyroscope information was obtained from two MIMU sensors connected to the upper and lower limbs of the subject, and AHRS (Attitude and Heading Reference Systems) algorithm was integrated with these sensor data and the upper extremity movement (upper arm, lower arm) quaternion orientation matrix was calculated. Euler orientation angles (for x, y, z axes) of shoulder and elbow joints were calculated by using kinematic analysis. With the developed interaction network, real time motion control of two degrees of freedom prototype upper extremity exoskeleton robot arm which is designed and manufactured with laboratory facilities was realized. As a result, the user performs the same movement synchronously in the exoskeleton robot as the person moves the arm.

Keywords

MIMU, Upper Extremity Exoskeleton Robot, Human-Robot İnteraction, Kinematic Analysis, AHRS Algorithm

Üst Ekstremite Dış İskelet Robot İçin Giyilebilir Kablosuz MIMU Sensörler Vasıtası İle İnsan Robot Etkileşim Ağı Tasarımı

Abstract

Bu araştırma kapsamında insan vücuduna uyumlu, insan hareketlerini destekleyen iki serbestlik dereceli bir üst-ekstremite dış iskelet robot sisteminin kontrolü için giyilebilir kablosuz sensörler MIMU (ivmeölçer, jiroskop) vasıtası ile insan robot etkileşim ağı tasarımı gerçekleştirilmiştir. Kişinin üst ve alt kol uzuvlarına bağlı iki adet MIMU sensörden açısal ivmelenme, jiroskop ve manyetometre bilgileri alınıp, AHRS (Attitude and Heading Reference Systems) algoritması ile bu sensör verileri bütünleştirilip kişinin üst ekstremite hareketine ilişkilin (üst kol, alt kol) kuaternion yönelim matrisi hesaplanmıştır. Kinematik analiz ile de kuaternion matrisi verileri kullanılarak omuz ve dirsek eklemlerine ait Euler yönelim açıları (x, y, z eksenleri için) hesaplanmıştır. Geliştirilen etkileşim ağı ile laboratuvar olanakları ile tasarlanan ve imalatı yapılan iki serbestlik dereceli prototip üst ekstremite dış iskelet robot kolun gerçek zamanlı hareket kontrolü gerçekleştirilmiştir. Sonuç olarak, kullanıcı kişi kolunu hareket ettirirken, dış iskelet robotta senkronize olarak aynı hareketi gerçekleştirmektedir.

Keywords

MIMU, Üst Ekstermite Diş İskelet Robot, İnsan-Robot Etkileşimi, Kinematik Analiz, AHRS Algoritması

References

• Atia MGB. and Salah O, 2018. Fuzzy logic with load compensation for upper limb exoskeleton control based on IMU data fusion, Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetic, December 12-15, 2018, Kuala Lumpur, Malaysia
• Bergamasco M, Scattareggia MS, 1995. The Mechanical Design of the MARCUS Prosthetics hand, In Proceedings of the IEEE International Workshop on Robot and Human.
• Bleser G, Hendeby G, Miezal M, 2011. Using egocentric vision to achieve robust inertial body tracking under magnetic disturbances. In Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Basel, Switzerland, 26–29 October 2011, pp. 103–109.
• Cowanet RE, 2012. Recent trends in assistive technology for mobility, Journal of NeuroEngineering and Rehabilitation, 9: 20, pp. 1–8, Apr. 2012.
• Del-Ama AJ, Moreno JC, Gil-Agudo A, De-los-Reyes A, and Pons JL., 2012. Online assessment of human–robot interaction for hybrid control of walking, Sensors, 12: 1, pp. 215–225.
• De Rossi M, 2011. Sensing pressure distribution on a lower-limb exoskeleton physical human–machine interface, Sensors, 11: 1, pp. 207–227, Jan. 2011.
• Dollar AM and Herr H, 2008. Lower extremity exoskeletons and active orthoses: Challenges and state of the art, IEEE Transactions on Robotics, 24: 1, pp. 144–158, Feb. 2008.
• El-Gohary M, McNames J, 2012. Shoulder and elbow joint angle tracking with inertial sensors IEEE Transactions on Biomedical Engineering, 59, 2635–2641.
• El-Gohary M, McNames J, 2015. Human joint angle estimation with inertial sensors and validation with a robot arm. IEEE Transactions on Biomedical Engineering, 62, 1759–1767.
• El-Gohary M, Holmstrom L, Huisinga J, King E, McNames J, Horak F, 2011. Upper limb joint angle tracking with inertial sensors. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Boston, MA, USA, 30 August–3 September 2011, pp. 5629–5632
• Fleischer C. Wege, A, Kondak K. and Hommel G. 2006. Application of EMG signals for controlling exoskeleton robots, Biomed Tech. 51: 5/6, pp. 314–319
• Fleischer C, Reinicke C, and Hummel G. 2005. Predicting the intended motion with EMG signals for an exoskeleton orthosis controller, Proceedings of the IEEE International Conference on Autonomous Robot Systems, pp. 2029–2034.
• Fleischer C. and Hummel, G, 2006. Embedded control system for a powered leg exoskeleton, In Embedded Systems—Modeling, Technology, and Applications. New York, NY, USA: Springer-Verlag, pp. 177–185.
• Frisoli A1, Bergamasco M, Carboncini MC, Rossi B, 2009. Robotic assisted rehabilitation in Virtual Reality with the L-EXOS, Stud Health Technol Information, 145:40-54.
• Güzel S, Bingül, Z. 2005. Robot Tekniği 1, Birsen yayınevi, İstanbul
• Huang J, Tu X, and He J. 2015. Design and Evaluation of the RUPERT Wearable Upper Extremity Exoskeleton Robot for Clinical and In-Home Therapies, IEEE Transactions on Systems, Man, and Cybernetics: Systems, DOI: 10.1109/TSMC.2015.2497205
• Huo W, Mohammed S, Moreno J C, and Amirat Y, 2016. Lower Limb Wearable Robots for Assistance and Rehabilitation: A State of the Art, IEEE Systems Journal, 10: 3.
• Hussain S, Xie S. Q, Jamwal, P. K. 2013. Control of a robotic orthosis for gait rehabilitation, Robotics and Autonomous Systems, 61, 911–919.
• Jung, Y, Kang, D, Kim, J, 2010. Upper body motion tracking with inertial sensors. In Proceedings of the 2010 IEEE International Conference on Robotics and Biomimetics (ROBIO), Tianjin, China, 14–18 December 2010, pp. 1746–1751.
• Kawamoto H, Lee S, Kanbe S, and Sankai Y, 2003. Power assist method for HAL-3 using EMG-based feedback controller, Proccesing of IEEE International Conference on Systems, Man and Cybernetics, pp. 1648–1653.
• Kawamoto H. and Sankai, Y, 2005. Power assist method based on phase sequence and muscle force condition for HAL, Journal Advanced Robotics ,19: 7, pp. 717–734.
• Lenzo B. 2013. Design of Novel Robotic Exoskeletons with Hybrid Actuation for Human. Scuola Superiore Sant'anna-Pisa: Innovative Technologies
• Lew E, Chavarriaga R, Silvoni S, and Millán JDR. 2012.Detection of self-paced reaching movement intention from EEG signals, Frontiers in Neuroengineering., 5: 13, pp. 1–17.
• Luengas YR, Lopez-Gutierrez JR, Salazar S and Lozano R, Robust controls for upper limbexoskeleton, real-time results, Proceedings of the Institution of Mechanical Engineers Part I:J Systems and Control Engineering, 1–10, IMechE 2018, Reprints and permissions: sagepub.co.uk/journalsPermissions.nav, DOI: 10.1177/0959651818758866
• Luinge HJ, Veltink PH. 2005. Measuring orientation of human body segments using miniature gyroscopes and accelerometers, Medical and Biological Engineering & Computing. 43, 273–282.
• Luinge HJ, Veltink PH., Baten CT, 2007. Ambulatory measurement of arm orientation. Journal of Biomechanics. 40, 78–85.
• Marchal-Crespo L, and Reinkensmeyer DJ. 2009. Review of control strategies for robotic movement training after neurologic injury, Journal of NeuroEngineering and Rehabilitation, 6: 20, pp. 1–15.
• Melchiorri C. 2015. Kinematic Model of Robot Manuplator Lecturer Notes, Bologna Universty, Department of the Electrical and Electronig Engineering.
• Mihelj M. 2006. Inverse kinematics of human arm based on multisensor data integration. Journal of Intelligent and Robotic Systems. 47, 139–153
• Miezal M, Bleser G, Schmitz N, Stricker D. 2013. A generic approach to inertial tracking of arbitrary kinematic chains. In Proceedings of the 8th International Conference on Body Area Networks, Boston, MA, USA, 30 September– 2 October, 2013, pp. 189–192. 67.
• Miezal M, Taetz B, Bleser G, 2016. On Inertial Body Tracking in the Presence of Model Calibration Errors. Sensors 16, 1132.
• Mohammed S, Amirat Y, and Rifai H, 2012. Lower-limb movement assistance through wearable robots: State of the art and challenges, Advanced Robotic, 26: 1/2, pp. 1–22.
• Picerno P, Cereatti A, Cappozzo, A, 2011. A spot check for assessing static orientation consistency of inertial and magnetic sensing units. Gait Posture, 33, 373–378.
• Peppoloni L, Filippeschi A, Ruffaldi E, Avizzano CA, 2013. A novel 7 degrees of freedom model for upper limb kinematic reconstruction based on wearable sensors. In Proceedings of the 2013 IEEE 11th International Symposium on Intelligent Systems and Informatics (SISY), Subotica, Serbia, 26–28 September 2013, pp. 105–110.
• Peppoloni L, Filippeschi A. Ruffaldi E, Avizzano C. 2016. A novel wearable system for the online assessment of risk for biomechanical load in repetitive efforts, International Journal of Industrial Ergonomics, 52, 1–11.
• Peppoloni L, Brizzi F, Avizzano CA, Ruffaldi, E. 2015. Immersive ROS-integrated framework for robot teleoperation. In Proceedings of the 2015 IEEE Symposium on 3D User Interfaces (3DUI), Arles, France, 23–24 March 2015, pp. 177–178.
• Pons JL, 2010. Rehabilitation exoskeletal robotics, IEEE Eng. Med. Biol. Mag., 29: 3, pp. 57–63, May/Jun. 2010.
• Pons JL, Wearable Robots: Biomechatronic Exoskeletons. New York, NY, USA: Wiley, 2008.
• Roetenberg, D, Luinge HJ, Baten CT, Veltink PH. 2005. Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 13, 395–405
• Roetenberg D, Luinge, H, Slycke, P. 2009. Xsens MVN: Full 6 DOF Human Motion Tracking Using Miniature Inertial Sensors, Xsens Motion Technologies BV: Enschede, The Netherlands,
• Ruffaldi E, Peppoloni L, Filippeschi A, Avizzano C, 2014. A novel approach to motion tracking with wearable sensors based on Probabilistic Graphical Models. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014, pp. 1247–1252.
• Sanchez R, Liu J, Rao S, Shah P, Smith R, Rahman T and Reinkensmeyer D. 2006. Automating Arm Movement Training Following Severe Stroke: Functional Exercises with Quantitative Feedback İn A Gravity-Reduced Environment. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 378–389.
• Suzuki K, Mito G, Kawamoto H, Hasegawa Y, and Sankai, Y, 2007. Intention-based walking support for paraplegia patients with robot suit HAL, Advanced Robotics, 21: 12, pp. 1441–1469.
• Taunyazov T, Omarali, B, Shintemirov, A. 2016. A novel low-cost 4-DOF wireless human arm motion tracker. In Proceedings of 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), Singapore, 26–29 June 2016, pp. 157–162.
• Taetz B, Bleser, G, Miezal, M. 2016. Towards Self-Calibrating Inertial Body Motion Capture. In Proceedings of the19th International Conference on Information Fusion, Heidelberg, Germany, 5–8 July 2016.
• Vignais N, Miezal M, Bleser G, Mura K, Gorecky D, Marin F, 2013. Innovative system for real-time ergonomic feedback in industrial manufacturing. Appl. Ergon. 44, 566–574.
• Vukobratovic M. 2006. Humanoid robotics—Past, present state, future, In Proc. 4th Serbian-Hungarian Joint Symp. Intell. Syst., pp. 13–31.
• Valiente A, 2005. Design of a Quasi-Passive Parallel Leg Exoskeleton to Augment Load Carrying for Walking, M.S. thesis, Dept. Mech. Eng., Massachusetts Inst. Technol., Cambridge, U.K., Aug. 2005.
• Veneman JF, Kruidhof R, Hekman EEG, Ekkelenkamp R, Van Asseldonk EHF, Van der Kooij H, 2007. Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 15 379–386.
• Wang Y. and Makeig, S, 2009. Predicting Intended Movement Direction Using EEG from Human Posterior Parietal Cortex. In: Schmorrow DD, Estabrooke IV, Grootjen M. (eds) Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience. FAC 2009. Lecture Notes in Computer Science, 5638. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02812-0_52
• Yavuz A, Akdoğan E, Aktan ME, Koru A, 2019. Desıgn, Produce and Control of A 2-Dof Upper Lımb Exoskeletal Robot, Journal of Thermal Engineering, 5(2), Special Issue 9, pp. 119-130.
• Zhang ZQ, Wong WC, Wu JK, 2011. Ubiquitous human upper-limb motion estimation using wearable sensors. IEEE Transactıons on Informatıon Technology in Bıomedıcıne. 15, 513–521.