Machine Learning-Assisted Wearable Thermo-Haptic Device for Creating Tactile Sensation

Abstract

The tactile modality is an important source of human experience and emotional expression, either on its own or by intensifying and complementing other senses, influencing our interactions with objects, people, animals and other beings. Following this, developed haptic devices transmit information to the user using tactile stimuli to increase or change sensory input. Haptics are an important factor that makes virtual worlds and remote interpersonal interaction tangible. Haptic feedback consists of more components that make an experience physically perceptible and realistic. Haptic feedbacks are widely used in mobile and wearable devices to convey various types of notifications to users. In this study, it was aimed to develop a new generation of wearable gloves against the hypoesthesia problem by combining artificial intelligence and thermohaptic, which are popular in many fields.

Keywords

• Wearable Gloves
• Artificial Intelligence
• Thermo Haptics
• Hypoesthesia

References

1. [1] R. Z. Hill and D. M. Bautista, “Getting in Touch with Mechanical Pain Mechanisms,” Trends Neurosci, vol. 43, no. 5, pp. 311–325, May 2020, doi: 10.1016/J.TINS.2020.03.004.
2. [2] E. H. Lee, S. H. Kim, and K. S. Yun, “Three-Axis Pneumatic Haptic Display for the Mechanical and Thermal Stimulation of a Human Finger Pad,” Actuators, vol. 10, no. 3, p. 60, Mar. 2021, doi: 10.3390/ACT10030060.
3. [3] X. Lu, X. Yao, W. F. Thompson, and L. Hu, “Movement-induced hypoalgesia: behavioral characteristics and neural mechanisms,” Ann N Y Acad Sci, vol. 1497, no. 1, pp. 39–56, Aug. 2021, doi: 10.1111/NYAS.14587.
4. [4] P. B. Shull and D. D. Damian, “Haptic wearables as sensory replacement, sensory augmentation and trainer - A review,” J Neuroeng Rehabil, vol. 12, no. 1, pp. 1–13, Jul. 2015, doi: 10.1186/S12984-015-0055-Z/FIGURES/7.
5. [5] S. Biswas and Y. Visell, “Haptic Perception, Mechanics, and Material Technologies for Virtual Reality,” Adv Funct Mater, vol. 31, no. 39, Sep. 2021, doi: 10.1002/ADFM.202008186.
6. [6] L. A. Connell, N. B. Lincoln, and K. A. Radford, “Somatosensory impairment after stroke: frequency of different deficits and their recovery,” Clin. Rehabil., vol. 22, no. 8, pp. 758–767, Aug. 2008, doi: 10.1177/0269215508090674.
7. [7] M. A. Choukou, S. Mbabaali, J. B. Hani, and C. Cooke, “Haptic-Enabled Hand Rehabilitation in Stroke Patients: A Scoping Review,” Applied Sciences, vol. 11, no. 8, p.3712, Apr. 2021, doi: 10.3390/APP11083712.
8. [8] S. M. Lai, S. Studenski, P. W. Duncan, and S. Perera, “Persisting Consequences of Stroke Measured by the Stroke Impact Scale,” Stroke, vol. 33, no. 7, pp. 1840–1844, Jul. 2002, doi: 10.1161/01.STR.0000019289.15440.F2.

9. [9] C. E. Seim, B. Ritter, K. E. Flavin, M. G. Lansberg, and A. M. Okamura, “Affective Ratings of Vibrotactile Signals in Older Adults with and without History of Stroke,” 2021 IEEE World Haptics Conference, WHC 2021, pp. 457–462, Jul. 2021, doi: 10.1109/WHC49131.2021.9517216.
10. [10] B. Zhang and M. Sra, “PneuMod: A Modular Haptic Device with Localized Pressure and Thermal Feedback,” in Proceedings of the 27th ACM Symposium on Virtual Teality Software and Thechnology, 2021, doi: 10.1145/3489849.3489857.
11. [11] J. J. Huaroto, E. Suárez, and E. A. Vela, “Wearable mechatronic devices for upper-limb amputees,” in Control Theory in Biomedical Engineering, 2020, pp. 205–234, doi: 10.1016/B978-0-12-821350-6.00008-1.
12. [12] S. Günther, “Somatosensory Interaction: Investigating Mechanoreception, Thermoception, and Proprioception for On-Body Haptic Feedback,” 2022, doi: 10.26083/TUPRINTS-00021617.
13. [13] A. Nasser, K. Zheng, and K. Zhu, “ThermEarhook: Investigating Spatial Thermal Haptic Feedback on the Auricular Skin Area,” ICMI 2021 - Proceedings of the 2021 International Conference on Multimodal Interaction, pp. 662–672, Oct. 2021, doi: 10.1145/3462244.3479922.
14. [14] J. Lee et al., “Stretchable Skin-Like Cooling/Heating Device for Reconstruction of Artificial Thermal Sensation in Virtual Reality,” Adv Funct Mater, vol. 30, no. 29, p. 1909171, Jul. 2020, doi: 10.1002/ADFM.201909171.
15. [15] A. I. Abdullah, N. Mat Isa, M. A. Al-Mashhadani, O. M. Al-Moayed, and A. K. Kareem, “Design of a Hybrid Haptic Wearable Device for Upper Limb Amputees to Recover the Missing Sensation,” International Journal of Mechanical Engineering, vol. 7, no. 1, 2022.
16. [16] J. Lee, D. Kim, H. Sul, & S. H. Ko, “Thermo‐haptic materials and devices for wearable virtual and augmented reality,” Advanced Functional Materials, vol. 31, no.39, p. 2007376, 2021.
17. [17] C. Chen and S. Qiu, “Influence of temperature profiles on thermal tactile sensation,” Adv. Robot, vol. 28, no. 1, pp. 53–61, 2014, doi: 10.1080/01691864.2013.854460.
18. [18] H. N. Ho and L. A. Jones, “Contribution of thermal cues to material discrimination and localization,” Percept Psychophys, vol. 68, no. 1, pp. 118–128, 2006, doi: 10.3758/BF03193662/METRICS.
19. [19] R. L. Peiris, W. Peng, Z. Chen, L. Chan, and K. Minamizawa, “ThermoVR: Exploring integrated thermal haptic feedback with head mounted displays,” Conference on Human Factors in Computing Systems - Proceedings, vol. 2017-May, pp. 5452–5456, May 2017, doi: 10.1145/3025453.3025824.
20. [20] W. Peng, R. L. Peiris, and K. Minamizawa, “Exploring of simulating passing through feeling on the wrist: Using thermal feedback,” in Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology, pp. 187–188, Oct. 2017, doi: 10.1145/3131785.3131819.
21. [21] S. Li, X. Tong, J. Huang, X. Wu, W. Yang, and Z. Pan, “A Thermal and Vibrational Feedback Glove Based on the Tactile Characteristics of Human Hand Skin,” IEEE Access, vol. 8, pp. 226671-226684, 2020, doi: 10.1109/ACCESS.2020.3045614.
22. [22] Y. Celik, J. Moore, M. Durgun, S. Stuart, W. L. Woo, & A. Godfrey, “Gait on the edge: A proposed wearable for continuous real-time monitoring beyond the laboratory.” IEEE Sensors J., vol. 23, no. 23, pp. 29656-29666, 2023, doi: 10.1109/JSEN.2023.3328054
23. [23] C. Zhang and Y. Lu, “Study on artificial intelligence: The state of the art and future prospects,” J Ind Inf Integr, vol. 23, p. 100224, Sep. 2021, doi: 10.1016/J.JII.2021.100224.
24. [24] M. Kışlakçı and M. Durgun, “Anomaly Diagnosis Using Autoencoder in Edge Computing Systems”, ISVOS, vol. 6, no. 1, pp. 41–50, 2022, doi: 10.47897/bilmes.1132562.
25. [25] R. S. Dahiya and M. Valle, “Robotic tactile sensing: Technologies and system,” Robotic Tactile Sensing: Technologies and System, vol. 9789400705791, pp. 1–245, Jan. 2014, doi: 10.1007/978-94-007-0579-1/COVER.
26. [26] O. Ozioko and R. Dahiya, “Smart Tactile Gloves for Haptic Interaction, Communication, and Rehabilitation,” Advanced Intelligent Systems, vol. 4, no. 2, p. 2100091, Feb. 2022, doi: 10.1002/AISY.202100091.
27. [27] L. M. Davidovic, D. Laketic, J. Cumic, E. Jordanova, and I. Pantic, “Application of artificial intelligence for detection of chemico-biological interactions associated with oxidative stress and DNA damage,” Chem Biol Interact, vol. 345, p. 109533, Aug. 2021, doi: 10.1016/J.CBI.2021.109533.
28. [28] N. Maffulli et al., “Artificial intelligence and machine learning in orthopedic surgery: a systematic review protocol,” J Orthop Surg Res, vol. 15, no. 1, Dec. 2020, doi: 10.1186/S13018-020-02002-Z.
29. [29] J. Wang and X. Hu, “Convolutional Neural Networks With Gated Recurrent Connections,” IEEE Trans Pattern Anal Mach Intell, vol. 44, no. 7, pp. 3421–3435, Jul. 2022, doi: 10.1109/TPAMI.2021.3054614.
30. [30] T. M. Maddox, J. S. Rumsfeld, and P. R. O. Payne, “Questions for Artificial Intelligence in Health Care,” JAMA, vol. 321, no. 1, pp. 31–32, Jan. 2019, doi: 10.1001/JAMA.2018.18932.
31. [31] G. M. Minopoulos, V. A. Memos, K. D. Stergiou, C. L. Stergiou, and K. E. Psannis, “A Medical Image Visualization Technique Assisted with AI-Based Haptic Feedback for Robotic Surgery and Healthcare,” Applied Sciences, vol. 13, no. 6, p. 3592, Mar. 2023, doi: 10.3390/APP13063592.
32. [32] R. De Fazio, V. M. Mastronardi, M. Petruzzi, M. De Vittorio, and P. Visconti, “Human–Machine Interaction through Advanced Haptic Sensors: A Piezoelectric Sensory Glove with Edge Machine Learning for Gesture and Object Recognition,” Future Internet 2023, vol. 15, no. 1, p. 14, Dec. 2022, doi: 10.3390/FI15010014.
33. [33] D. Ravenscroft, I. Prattis, T. Kandukuri, Y. A. Samad, and L. G. Occhipinti, “A Wearable Graphene Strain Gauge Sensor with Haptic Feedback for Silent Communications,” FLEPS 2021 - IEEE International Conference on Flexible and Printable Sensors and Systems, Jun. 2021, doi: 10.1109/FLEPS51544.2021.9469728.
34. [34] D. Wang, K. Ohnishi, and W. Xu, “Novel emerging sensing, actuation, and control techniques for haptic interaction and teleoperation,” IEEE Transactions on Industrial Electronics, vol. 67, no. 1, pp. 624–626, Jan. 2020, doi: 10.1109/TIE.2019.2927784.
35. [35] Д. Л. Овчинников, А. Ю. Тычков, Д. С. Чернышов, А. Д. Сашина, and П. Государственный Университет, “РАЗРАБОТКА ПРОТОТИПА ‘УМНОЙ’ ОДЕЖДЫ С ДИСТАНЦИОННЫМ КОНТРОЛЕМ И УПРАВЛЕНИЕМ ТЕМПЕРАТУРОЙ ПОСРЕДСТВОМ МИКРОЭЛЕКТРОННОГО ИСПОЛНЕНИЯ,” Измерение. Мониторинг. Управление. Контроль, vol. 4, no.42, 2022, doi: 10.21685/2307-5538-2022-4-13.
36. [36] A. Puliafito, A. Celesti, M. Villari, and M. Fazio, “Towards the integration between IoT and cloud computing: An approach for the secure self-configuration of embedded devices,” Int J Distrib Sens Netw, vol. 2015, 2015, doi: 10.1155/2015/286860.
37. [37] P. F. Folty´nek, M. Babiuch, and P. S. ˇ Uránek, “Measurement and data processing from Internet of Things modules by dual-core application using ESP32 board,” Measurement and Control, vol. 52, no. 8, pp. 970–984, 2019, doi: 10.1177/0020294019857748.
38. [38] J. Kaur, N. Batra, and S. Goyal, “SmartEmoDetect: An Internet of Things Based Emotion Monitoring Wearable Technology for Drivers Lightweight Operating System Framework for IoT Devices View project Integrated platform for innovation and smart product designs View project SmartEmoDetect: An IoT Based Emotion Monitoring Wearable Technology for Drivers,” Article in Journal of Computational and Theoretical Nanoscience, vol. 16, pp. 1–5, 2019, doi: 10.1166/jctn.2019.8279.
39. [39] M. Fezari and A. Al Dahoud, “Exploring One-wire Temperature sensor ‘DS18B20’ with Microcontrollers”, Accessed: Jun. 02, 2023. [Online]. Available: https://www.researchgate.net/publication/330854061
40. [40] H. K. Naji, N. Goga, A. J. M. Karkar, H. A. Ali, M. Falahi, and A. T. H. Al-Rawi, “Design and Development of a Bracelet to Monitor Vital signs of COVID-19 Patients,” IICETA 2022 - 5th International Conference on Engineering Technology and its Applications, pp. 109–114, 2022, doi: 10.1109/IICETA54559.2022.9888696.
41. [41] S. Miah, G. M. Jahedul Islam, S. K. Das, S. Islam, M. Islam, and K. K. Islam, “Internet of Things (IoT) based automatic electrical energy meter billing system,” vol. 14, no. 4, pp. 39–50, doi: 10.9790/2834-1404013950.
42. [42] H. Hanan, A. A. N. Gunawan, and M. Sumadiyasa, “Water level detection system based on ultrasonic sensors HC-SR04 and ESP8266-12 modules with telegram and buzzer communication media,” Instrum. mes. métrol., vol. 18, no. 3, pp. 305–309, 2019, doi: 10.18280/i2m.180311.
43. [43] K. N. Swaroop, K. Chandu, R. Gorrepotu, and S. Deb, “A health monitoring system for vital signs using IoT,” Internet of Things, vol. 5, pp. 116–129, Mar. 2019, doi: 10.1016/J.IOT.2019.01.004.
44. [44] E.-B. Park, S. J. M. Yazdi, and J.-H. Lee, “Development of wearable temperature sensor based on Peltier thermoelectric device to change human body temperature,” Sens. Mater., vol. 32, no. 9, p. 2959, 2020, doi: 10.18494/SAM.2020.2741.
45. [45] R. Li and H. Fariborzi, “Design and Demonstration of MEM Relay-Based Arithmetic and Sequential Circuit Blocks,” IEEE Trans Electron Devices, vol. 68, no. 12, pp. 6415–6421, Dec. 2021, doi: 10.1109/TED.2021.3118663
46. [46] M. O. Alsumady, Y. K. Alturk, A. Dagamseh, and M. Tantawi, “Controlling of DC-DC buck converters using microcontrollers,” Int. J. Circuits Syst. Signal Process., vol. 15, pp. 197–202, 202, doi: 10.46300/9106.2021.15.22.
47. [47] C. Du, “Empirical study on college students’ debugging abilities in computer programming,” in 2009 First International Conference on Information Science and Engineering, 2009, doi: 10.1109/ICISE.2009.550.
48. [48] R. K. Kodali and K. S. Mahesh, “Low cost ambient monitoring using ESP8266,” in 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I), 2016, doi: 10.1109/IC3I.2016.7918788.
49. [49] M. S. Gowri R, R. R. Gowsika S, and M. D. Assistant Professor, “Prepaid Energy Meter with Automatic Billing and Theft Detection,” International Journal of New Innovations in Engineering and Technology, vol. 22, p. 619, 2023.
50. [50] H. A. Sonar, J.-L. Huang, and J. Paik, “Soft Touch using Soft Pneumatic Actuator–Skin as a Wearable Haptic Feedback Device,” Advanced Intelligent Systems, vol. 3, no. 3, Mar. 2021, doi: 10.1002/AISY.202000168.
51. [51] J. I. C. Lindsay, I. Jiang, E. Larson, R. Adams, S. N. Patel, and B. Hannaford, “Good vibrations: An evaluation of vibrotactile impedance matching for low power wearable applications,” in Proceedings of the 26th annual ACM symposium on User interface software and technology, 2013, doi: 10.1145/2501988.2502051.
52. [52] K. Shimada, “Estimation of Fast and Slow Adaptions in the Tactile Sensation of Mechanoreceptors Mimicked by Hybrid Fluid (HF) Rubber with Equivalent Electric Circuits and Properties,” Sensors, vol. 23, no. 3, p. 1327, Jan. 2023, doi: 10.3390/S23031327.
53. [53] M. Chikai and H. Miyake, “Development of a Haptic Performance Diagnostics Device for Children with Developmental Disorder,” Iryou kikigaku (The Japanese journal of medical instrumentation), vol. 84, no. 1, pp. 18–22, 2014, doi: 10.4286/JJMI.84.18.
54. [54] J. Wheeler, K. Bark, J. Savall, and M. Cutkosky, “Investigation of rotational skin stretch for proprioceptive feedback with application to myoelectric systems,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 18, no. 1, pp. 58–66, Feb. 2010, doi: 10.1109/TNSRE.2009.2039602.
55. [55] T. Maeda and T. Kurahashi, “TherModule: Wearable and modular thermal feedback system based on a wireless platform,” in Proceedings of the 10th Augmented Human International Conference 2019, 2019., doi: 10.1145/3311823.3311826.