A decision-making mechanism based on EMG signals and adaptive neural fuzzy inference system (ANFIS) for hand gesture prediction

Öz

Artificial intelligence (AI)-based technologies assist users in applying the intended action when upper extremity movement cannot be fully provided. Electromyography (EMG), a depiction of muscle activity, offers various advantages when employed with AI-based systems like virtual reality applications and prosthetics controls. In this paper, a fuzzy logic (FL)-based decision-making mechanism is presented in order to provide effective control and improve the prediction performance of the stated systems. In this regard, EMG signals were collected from 30 participants when imitating different seven hand gestures. After the necessary preprocessing and segmentation processes, the Empirical Mode Decomposition (EMD) method which is the first stage of the Hilbert-Huang Transform (HHT) was applied and Intrinsic Mode Functions (IMF) were obtained. High-resolution time-frequency (TF) images were obtained by applying HHT to the IMFs determined by a statistical selection method. Various distinctive features were extracted from the visualized TF images based on the joint representation of the time and frequency domain. The Adaptive Neuro-Fuzzy Inference System (ANFIS) was then fed these features, which used two alternative clustering approaches. For seven hand gesture classifications, the average accuracy scores for the Subtractive Clustering (SC) and Fuzzy C-mean (FCM) clustering methods were obtained as 93.88% and 92.10%, respectively. The proposed feature extraction method based on TF representation combined with FL techniques yielded encouraging results for the classification of nonstationary and nonlinear biological signals such as EMG.

Anahtar Kelimeler

• Fuzzy logic
• empirical mode decomposition
• EMG
• Hilbert-Huang Transform
• time-frequency analysis

El hareketi tahmini için EMG sinyalleri ve uyarlamalı sinirsel bulanık çıkarım sistemine (ANFIS) dayalı bir karar verme mekanizması

Öz

Üst ekstremite hareketi tam olarak sağlanamadığında, yapay zeka (artificial intelligence/AI) sistemleri kullanıcılara amaçlanan hareketin uygulanması konusunda yardımcı olurlar. Kas aktivitesinin temsili olan elektromiyografi (EMG), sanal gerçeklik uygulamaları ve protez kontrolleri gibi AI-tabanlı sistemlerde kullanıldığında çeşitli faydalar sağlar. Bu çalışmada, bahsedilen sistemlere etkin kontrol sunmak ve tahmin performanslarını iyileştirmek amacıyla bulanık mantık (Fuzzy Logic/FL)-tabanlı bir karar verme mekanizması sunulmuştur. Bu bağlamda, 30 katılımcıdan yedi farklı el hareketini taklit etmesi sonucunda oluşan EMG sinyalleri toplandı. Gerekli ön işleme ve bölütleme işlemlerinin ardından elde edilen sinyallere Hilbert-Huang Dönüşümü'nün (HHD) ilk aşaması Görgül Kip Ayrışımı (GKA) metodu uygulandı ve İçsel Mod Fonksiyonları (İMF) elde edildi. İstatistiksel İMF seçim yöntemi ile belirlenen İMF’lere HHD uygulanmasıyla iyi çözünürlüklü zaman-frekans (time-frequency/TF) imgeleri elde edildi. Zaman ve frekans uzayının ortak temsiline dayalı görselleştirilmiş TF imgelerinden çeşitli ayırt edici öznitelikler çıkartıldı. İki farklı kümeleme tekniği uygulanan öznitelik veri seti, Uyarlamalı Sinirsel Bulanık Çıkarım Sistemi'ne (ANFIS) girdi olarak verildi. Yedi el hareketi sınıflandırması için Azaltımlı (Subtractive Clustering/SC) ve Bulanık C-ortalama (Fuzzy C-mean/FCM) kümeleme yöntemleri için ortalama doğruluk değerleri sırasıyla %93,88 ve %92,10 olarak elde edilmiştir. TF temsiline dayalı özniteliklerin FL yaklaşımlarıyla sınıflandırılması sonucu elde edilen bulgular, EMG gibi durağan ve doğrusal olmayan biyolojik sinyallerin sınıflandırılması için umut verici olduğunu göstermiştir.

Anahtar Kelimeler

• Bulanık mantık
• görgül kip ayrışımı
• EMG
• Hilbert-Huang Dönüşümü
• zaman-frekans analizi

Kaynakça

1. Balbinot A., Favieiro G., A Neuro-Fuzzy System for Characterization of Arm Movements, Sensors, 13 (2), 2613–2630, 2013.
2. Fajardo J.M., Gomez O., Prieto F., EMG hand gesture classification using handcrafted and deep features, Biomed. Signal Process. Control, 63 (March 2020), 102210, 2021.
3. Zahak M., Signal Acquisition Using Surface EMG and Circuit Design Considerations for Robotic Prosthesis, Computational Intelligence in Electromyography Analysis - A Perspective on Current Applications and Future Challenges, Naik G.R., IntechOpen, London, United Kingdom: IntechOpen, (2012).
4. Kılıç E., Başer Ö., Kızılhan H., EMG-based stiffness estimation of ankle joint and real-time implementation on a variable stiffness ankle exoskeleton robot, J. Fac. Eng. Archit. Gazi Univ., 36 (1), 225–240, 2020.
5. Ozdemir M.A., Kisa D.H., Guren O., Onan A., Akan A., EMG based Hand Gesture Recognition using Deep Learning, 2020 Med. Technol. Congr., 1–4, 2020.
6. Arozi M., Ariyanto M., Kristianto A., Munadi, Setiawan J.D., EMG Signal Processing of Myo Armband Sensor for Prosthetic Hand Input using RMS and ANFIS, 2020 7th Int. Conf. Inf. Technol. Comput. Electr. Eng., 36–40, 2020.
7. De la Cruz-Sánchez B.A., Arias-Montiel M., Lugo-González E., EMG-controlled hand exoskeleton for assisted bilateral rehabilitation, Biocybern. Biomed. Eng., 42 (2), 596–614, 2022.
8. Kisa D.H., Ozdemir M.A., Guren O., Akan A., EMG based Hand Gesture Classification using Empirical Mode Decomposition Time-Series and Deep Learning, 2020 Med. Technol. Congr., 1–4, 2020.

9. Ozdemir M.A., Kisa D.H., Guren O., Akan A., Hand gesture classification using time–frequency images and transfer learning based on CNN, Biomed. Signal Process. Control, 77 103787, 2022.
10. Lee K.H., Min J.Y., Byun S., Electromyogram-Based Classification of Hand and Finger Gestures Using Artificial Neural Networks, Sensors, 22 (1), 225, 2021.
11. Narayan Y., Hb vsEMG signal classification with time domain and Frequency domain features using LDA and ANN classifier, Mater. Today Proc., 37 (Part 2), 3226–3230, 2021.
12. Zhai X., Jelfs B., Chan R.H.M., Tin C., Short latency hand movement classification based on surface EMG spectrogram with PCA, 2016 38th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., 327–330, 2016.
13. Phinyomark A., Phukpattaranont P., Limsakul C., Feature reduction and selection for EMG signal classification, Expert Syst. Appl., 39 (8), 7420–7431, 2012.
14. Srhoj-Egekher V., Cifrek M., Medved V., The application of Hilbert-Huang transform in the analysis of muscle fatigue during cyclic dynamic contractions., Med. Biol. Eng. Comput., 49 (6), 659–69, 2011.
15. Revilla L.M., Delis A.L., Olaya A.F.R., Evaluation of the Hilbert-Huang Transform for myoelectric pattern classification: Towards a method to detect movement intention, 2013 Pan Am. Heal. Care Exch., 1–6, 2013.
16. Huang N.E., Shen Z., Long S.R., Wu M.C., Shih H.H., Zheng Q., Yen N., Tung C.C., Liu H.H., Mathematical S.P., Sciences E., The Empirical Mode Decomposition and the Hilbert Spectrum for Nonlinear and Non- Stationary Time Series Analysis, Proc. Math. Phys. Eng. Sci., 454 (1971), 903–995, 1998.
17. Lin C.-F., Zhu J.-D., Hilbert–Huang transformation-based time-frequency analysis methods in biomedical signal applications, Proc. Inst. Mech. Eng. Part H J. Eng. Med., 226 (3), 208–216, 2012.
18. Wahid M.F., Tafreshi R., Al-Sowaidi M., Langari R., Subject-independent hand gesture recognition using normalization and machine learning algorithms, J. Comput. Sci., 27 69–76, 2018.
19. Gadekallu T.R., Srivastava G., Liyanage M., M. I., Chowdhary C.L., Koppu S., Maddikunta P.K.R., Hand gesture recognition based on a Harris Hawks optimized Convolution Neural Network, Comput. Electr. Eng., 100 107836, 2022.
20. Jahani Fariman H., Ahmad S.A., Hamiruce Marhaban M., Ali Jan Ghasab M., Chappell P.H., Simple and Computationally Efficient Movement Classification Approach for EMG-controlled Prosthetic Hand: ANFIS vs. Artificial Neural Network, Intell. Autom. Soft Comput., 21 (4), 559–573, 2015.
21. Karlik B., Tokhi M.O., Alci M., A fuzzy clustering neural network architecture for multifunction upper-limb prosthesis, IEEE Trans. Biomed. Eng., 50 (11), 1255–1261, 2003.
22. Subasi A., Classification of EMG signals using combined features and soft computing techniques, Appl. Soft Comput., 12 (8), 2188–2198, 2012.
23. Khezri M., Jahed M., Real-time intelligent pattern recognition algorithm for surface EMG signals, Biomed. Eng. Online, 6 (1), 45, 2007.
24. Ouyang G., Zhu X., Ju Z., Liu H., Dynamical Characteristics of Surface EMG Signals of Hand Grasps via Recurrence Plot, IEEE J. Biomed. Heal. Informatics, 18 (1), 257–265, 2014.
25. Khezri M., Jahed M., A Neuro–Fuzzy Inference System for sEMG-Based Identification of Hand Motion Commands, IEEE Trans. Ind. Electron., 58 (5), 1952–1960, 2011.
26. Kaiser M.S., Chowdhury Z.I., Mamun S. Al, Hussain A., Mahmud M., A Neuro-Fuzzy Control System Based on Feature Extraction of Surface Electromyogram Signal for Solar-Powered Wheelchair, Cognit. Comput., 8 (5), 946–954, 2016.
27. Ulkir O., Gokmen G., Kaplanoglu E., Emg Signal Classification Using Fuzzy Logic, Balk. J. Electr. Comput. Eng., 5 (2), 97–101, 2017.
28. Caesarendra W., Tjahjowidodo T., Nico Y., Wahyudati S., Nurhasanah L., EMG finger movement classification based on ANFIS, J. Phys. Conf. Ser., 1007 (1), 012005, 2018.
29. Andrade A.O., Nasuto S., Kyberd P., Sweeney-Reed C.M., Van Kanijn F.R., EMG signal filtering based on Empirical Mode Decomposition, Biomed. Signal Process. Control, 1 (1), 44–55, 2006.
30. Lingling Chen, Peng Yang, Linan Zu, Xin Guo, Movement recognition by electromyography signal for transfemoral prosthesis control, 2009 4th IEEE Conf. Ind. Electron. Appl., 1127–1132, 2009.
31. Küçük H., Eminoğlu İ., Balcı K., Classification of neuromuscular diseases with artificial intelligence methods, J. Fac. Eng. Archit. Gazi Univ., 34 (4), 1725–1741, 2019.
32. Karlsson S., Yu J., Akay M., Time-frequency analysis of myoelectric signals during dynamic contractions: A comparative study, IEEE Trans. Biomed. Eng., 47 (2), 228–238, 2000.
33. Xie H., Wang Z., Mean frequency derived via Hilbert-Huang transform with application to fatigue EMG signal analysis, Comput. Methods Programs Biomed., 82 (2), 114–120, 2006.
34. Huang N.E., Introduction to the Hilbert–Huang Transform and its related mathematical problems, Hilbert–Huang Transform and Its Applications, Shen S.S., World Scientific, 1-26, 2014.
35. Zahra A., Kanwal N., ur Rehman N., Ehsan S., McDonald-Maier K.D., Seizure detection from EEG signals using Multivariate Empirical Mode Decomposition, Comput. Biol. Med., 88 132–141, 2017.
36. Peng Z.K., Tse P.W., Chu F.L., A comparison study of improved Hilbert–Huang transform and wavelet transform: Application to fault diagnosis for rolling bearing, Mech. Syst. Signal Process., 19 (5), 974–988, 2005.
37. Komaty A., Boudraa A.-O., Augier B., Dare-Emzivat D., EMD-Based Filtering Using Similarity Measure Between Probability Density Functions of IMFs, IEEE Trans. Instrum. Meas., 63 (1), 27–34, 2014.
38. Andrade A., Kyberd P., Nasuto S., The application of the Hilbert spectrum to the analysis of electromyographic signals, Inf. Sci. (Ny)., 178 (9), 2176–2193, 2008.
39. Hafizah W.M., Supriyanto E., Yunus J., Feature Extraction of Kidney Ultrasound Images Based on Intensity Histogram and Gray Level Co-occurrence Matrix, 2012 Sixth Asia Model. Symp., 115–120, 2012.
40. Ozdemir M.A., Ozdemir G.D., Guren O., Classification of COVID-19 electrocardiograms by using hexaxial feature mapping and deep learning, BMC Med. Inform. Decis. Mak., 21 (1), 170, 2021.
41. Vasantha M., Bharathi D.V.S., Dhamodharan R., Medical Image Feature, Extraction, Selection And Classification, Int. J. Eng. Sci. Technol., 2 (6), 2071–2076, 2010.
42. Jia G., Lam H.-K., Ma S., Yang Z., Xu Y., Xiao B., Classification of Electromyographic Hand Gesture Signals Using Modified Fuzzy C-Means Clustering and Two-Step Machine Learning Approach, IEEE Trans. Neural Syst. Rehabil. Eng., 28 (6), 1428–1435, 2020.
43. Zadeh L.A., Fuzzy Sets, Inf. Control, 8 338–353, 1965.
44. Jang J.-S.R., ANFIS: adaptive-network-based fuzzy inference system, IEEE Trans. Syst. Man. Cybern., 23 (3), 665–685, 1993.
45. Ozdemir M.A., Kisa D.H., Guren O., Akan A., Dataset for multi-channel surface electromyography (sEMG) signals of hand gestures, Data Br., 41 107921, 2022.
46. Peldek S., Becerikli Y., Recognition of human action in motion detected images with GMACA, J. Fac. Eng. Archit. Gazi Univ., 2018 (18–2), 1025–1044, 2018.
47. Khushaba R.N., Kodagoda S., Takruri M., Dissanayake G., Toward improved control of prosthetic fingers using surface electromyogram (EMG) signals, Expert Syst. Appl., 39 (12), 10731–10738, 2012.
48. Palmer H., Cohen K., Genetic Fuzzy Hand Gesture Classifier, Explainable AI and Other Applications of Fuzzy Techniques, Rayz, j., Raskin, V., Dick, S., and Kreinovich, V., Springer, Cham, 332-342, 2022.