El Sıkma Hareketinin İşlevsel Yakın Kızılaltı Spektroskopisi ve Elektromiyografi Sinyalleri Kullanılarak Sınıflandırılması

Öz

El hareketinin sınıflandırılması, özellikle inme rahatsızlığı geçiren kişilerde nörorehabilitasyon amaçlı beyin bilgisayar arayüzü (BBA) modellerinin geliştirilmesinde büyük önem arz etmektedir. Ancak, el hareketi odaklı BBA modellerinin geliştirilmesinde kullanılan kas ve beyin aktivitesi ölçüm modalitelerinin tek başlarına kullanılmasında, nörolojik adaptasyon ve bazı hasta gruplarının nöromusküler hastalık barındırması gibi çeşitli problemler bulunmaktadır. Bu çalışmada bir kavrama kuvveti görevi aracılığı ile gerçekleştirilen el hareketinin sonucu elde edilen işlevsel yakın kızılaltı spektroskopisi (iYKAS) ve elektromiyografi (EMG) sinyalleri kullanılarak el hareketinin sınıflandırılması gerçekleştirilmiştir. Bu sinyallerden çıkartılan öznitelikler, L1 norm tabanlı bir destek vektör makinesi (DVM) ile seçildikten sonra, K-en yakın komşuluk, doğrusal ve radyal temelli DVM, Gradyan Artırma, Adaboost, Naive Bayes, Doğrusal Diskriminant, Kuadratik Diskriminant ve Lojistik regresyon sınıflandırıcılarına verilmiştir. Sınıflandırıcıların başarımı, bir katılımcıyı dışarıda bırak (leave-one-subject-out) çapraz geçerliliği uygulanarak gerçekleştirilmiştir. Sınıflandırıcılar arasında en yüksek doğruluk yüzdesi, iYKAS ve EMG odaklı özniteliklerden faydalanılarak, Doğrusal Diskriminant metodu ile %84 olarak bulunmuştur. Sonuçlarımız bize işlevsel yakın kızılaltı spektroskopisi ve elektromiyografi verilerinin el hareketinin sınıflandırılmasında kullanılabileceğini ve bunun BBA sistemlerine de entegre edilebileceğini ortaya koymaktadır.

Anahtar Kelimeler

• El sıkma
• iYKAS
• EMG
• Makine Öğrenmesi

Classification of Hand-Grip Movement Using Functional Near-Infrared Spectroscopy and Electromyography

Abstract

Classification of hand movement is of great importance in the development of brain-computer interface (BCI) models for neurorehabilitation, particularly in stroke patients. However, there are various problems in the application of muscle and brain activity measurement modalities used in the performance of hand movement-oriented BCI systems, such as neurological adaptation and neuromuscular disease in some patient groups. In this study, classification of hand movement was performed using functional near infrared spectroscopy (iYKAS) and electromyography (EMG) signals obtained as a result of hand movement performed through a grip strength task. Features extracted from these signals are given to K-nearest neighbor, linear and radial basis SVM, Gradient Boost, Adaboost, Naive Bayes, Linear Discriminant, Quadratic Discriminant and Logistic regression classifiers after they are selected with an L1 norm-based support vector machine (SVM). The performance of the classifiers was achieved by applying the leave-one-subject-out cross-validation. Among the classifiers, the highest percentage of accuracy was found to be 84% with the Linear Discriminant method, using iYKAS and EMG focused features. Our results reveal that functional near-infrared spectroscopy and electromyography data can be used to classify hand movement and can be integrated into BCI systems.

Keywords

• Hand-Grip
• fNIRS
• EMG
• Machine Learning

References

1. [1] L. F. Nicolas-Alonso and J. Gomez-Gil, "Brain computer interfaces, a review," Sensors (Basel), vol. 12, no. 2, pp. 1211-79, 2012, doi: 10.3390/s120201211.
2. [2] J. J. Shih, D. J. Krusienski, and J. R. Wolpaw, "Brain-computer interfaces in medicine," Mayo Clinic proceedings, vol. 87, no. 3, pp. 268-79, Mar 2012, doi: 10.1016/j.mayocp.2011.12.008.
3. [3] C. L. Pulliam, S. R. Stanslaski, and T. J. Denison, "Industrial perspectives on brain-computer interface technology," Handb Clin Neurol, vol. 168, pp. 341-352, 2020, doi: 10.1016/B978-0-444-63934-9.00025-1.
4. [4] J. Kogel, J. R. Schmid, R. J. Jox, and O. Friedrich, "Using brain-computer interfaces: a scoping review of studies employing social research methods," BMC Med Ethics, vol. 20, no. 1, p. 18, Mar 7 2019, doi: 10.1186/s12910-019-0354-1.
5. [5] R. Carabalona, P. Castiglioni, and F. Gramatica, "Brain-computer interfaces and neurorehabilitation," Studies in health technology and informatics, vol. 145, pp. 160-76, 2009. [Online]. Available: https://www.ncbi.nlm.nih.gov/pubmed/19592793 https://ebooks.iospress.nl/publication/12265.
6. [6] S. Sreedharan, R. Sitaram, J. S. Paul, and C. Kesavadas, "Brain-computer interfaces for neurorehabilitation," Critical reviews in biomedical engineering, vol. 41, no. 3, pp. 269-79, 2013, doi: 10.1615/critrevbiomedeng.2014010697.
7. [7] M. J. Young, D. J. Lin, and L. R. Hochberg, "Brain-Computer Interfaces in Neurorecovery and Neurorehabilitation," Semin Neurol, vol. 41, no. 2, pp. 206-216, Apr 2021, doi: 10.1055/s-0041-1725137.
8. [8] R. Sitaram et al., "FMRI brain-computer interface: a tool for neuroscientific research and treatment," Comput Intell Neurosci, p. 25487, 2007, doi: 10.1155/2007/25487.

9. [9] S. Machado et al., "EEG-based brain-computer interfaces: an overview of basic concepts and clinical applications in neurorehabilitation," Rev Neurosci, vol. 21, no. 6, pp. 451-68, 2010, doi: 10.1515/revneuro.2010.21.6.451.
10. [10] N. Naseer and K. S. Hong, "fNIRS-based brain-computer interfaces: a review," (in eng), Front Hum Neurosci, vol. 9, p. 3, 2015, doi: 10.3389/fnhum.2015.00003.
11. [11] J. Mellinger et al., "An MEG-based brain-computer interface (BCI)," Neuroimage, vol. 36, no. 3, pp. 581-93, Jul 1 2007, doi: 10.1016/j.neuroimage.2007.03.019.
12. [12] S. H. Kohl, D. M. A. Mehler, M. Luhrs, R. T. Thibault, K. Konrad, and B. Sorger, "The Potential of Functional Near-Infrared Spectroscopy-Based Neurofeedback-A Systematic Review and Recommendations for Best Practice," Front Neurosci, vol. 14, p. 594, 2020, doi: 10.3389/fnins.2020.00594.
13. [13] S. Balasubramanian, E. Garcia-Cossio, N. Birbaumer, E. Burdet, and A. Ramos-Murguialday, "Is EMG a Viable Alternative to BCI for Detecting Movement Intention in Severe Stroke?," IEEE transactions on bio-medical engineering, vol. 65, no. 12, pp. 2790-2797, Dec 2018, doi: 10.1109/TBME.2018.2817688.
14. [14] J. Rouillard et al., "Hybrid BCI Coupling EEG and EMG for Severe Motor Disabilities," Procedia Manufacturing, vol. 3, pp. 29-36, 2015/01/01/ 2015, doi: https://doi.org/10.1016/j.promfg.2015.07.104.
15. [15] V. de Seta et al., "Cortico-muscular coupling to control a hybrid brain-computer interface for upper limb motor rehabilitation: A pseudo-online study on stroke patients," Frontiers in Human Neuroscience, Original Research vol. 16, 2022. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fnhum.2022.1016862.
16. [16] A. Chowdhury, H. Raza, Y. K. Meena, A. Dutta, and G. Prasad, "Online Covariate Shift Detection-Based Adaptive Brain–Computer Interface to Trigger Hand Exoskeleton Feedback for Neuro-Rehabilitation," IEEE Transactions on Cognitive and Developmental Systems, vol. 10, no. 4, pp. 1070-1080, 2018, doi: 10.1109/TCDS.2017.2787040.
17. [17] A. Chowdhury, H. Raza, Y. K. Meena, A. Dutta, and G. Prasad, "An EEG-EMG correlation-based brain-computer interface for hand orthosis supported neuro-rehabilitation," J Neurosci Methods, vol. 312, pp. 1-11, Jan 15 2019, doi: 10.1016/j.jneumeth.2018.11.010.
18. [18] T. Sadoyama, T. Masuda, and H. Miyano, "Relationships between muscle fibre conduction velocity and frequency parameters of surface EMG during sustained contraction," European journal of applied physiology and occupational physiology, vol. 51, no. 2, pp. 247-256, 1983/08/01 1983, doi: 10.1007/BF00455188.
19. [19] C. Byung Chan and S. Bo Hyeok, "Development of new brain computer interface based on EEG and EMG," in 2008 IEEE International Conference on Robotics and Biomimetics, 22-25 Feb. 2009 2009, pp. 1665-1670, doi: 10.1109/ROBIO.2009.4913251. [Online]. Available: https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?tp=&arnumber=4913251&ref=
20. [20] R. Leeb, H. Sagha, R. Chavarriaga, and R. Millan Jdel, "A hybrid brain-computer interface based on the fusion of electroencephalographic and electromyographic activities," J Neural Eng, vol. 8, no. 2, p. 025011, Apr 2011, doi: 10.1088/1741-2560/8/2/025011.
21. [21] N. A. Bhagat et al., "Design and Optimization of an EEG-Based Brain Machine Interface (BMI) to an Upper-Limb Exoskeleton for Stroke Survivors," Front Neurosci, vol. 10, p. 122, 2016, doi: 10.3389/fnins.2016.00122.
22. [22] J. Zhang, B. Wang, C. Zhang, Y. Xiao, and M. Y. Wang, "An EEG/EMG/EOG-Based Multimodal Human-Machine Interface to Real-Time Control of a Soft Robot Hand," Front Neurorobot, vol. 13, p. 7, 2019, doi: 10.3389/fnbot.2019.00007.
23. [23] Z. Wang et al., "Incorporating EEG and EMG Patterns to Evaluate BCI-Based Long-Term Motor Training," IEEE Transactions on Human-Machine Systems, vol. 52, no. 4, pp. 648-657, 2022, doi: 10.1109/THMS.2022.3168425.
24. [24] A. Scano et al., "NIRS-EMG for Clinical Applications: A Systematic Review," Applied Sciences, vol. 9, no. 15, doi: 10.3390/app9152952.
25. [25] P. Ortega, T. Zhao, and A. A. Faisal, "HYGRIP: Full-Stack Characterization of Neurobehavioral Signals (fNIRS, EEG, EMG, Force, and Breathing) During a Bimanual Grip Force Control Task," Front Neurosci, vol. 14, p. 919, 2020, doi: 10.3389/fnins.2020.00919.
26. [26] F. Pedregosa et al., "Scikit-learn: Machine Learning in Python," J. Mach. Learn. Res., vol. 12, no. null, pp. 2825–2830, 2011.
27. [27] M. Cope and D. T. Delpy, "System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination," Med Biol Eng Comput, vol. 26, no. 3, pp. 289-94, May 1988. [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/2855531.
28. [28] L. Koessler et al., "Automated cortical projection of EEG sensors: anatomical correlation via the international 10-10 system," Neuroimage, vol. 46, no. 1, pp. 64-72, May 15 2009, doi: 10.1016/j.neuroimage.2009.02.006.
29. [29] M. Okamoto et al., "Three-dimensional probabilistic anatomical cranio-cerebral correlation via the international 10-20 system oriented for transcranial functional brain mapping," Neuroimage, vol. 21, no. 1, pp. 99-111, Jan 2004. [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/14741647 http://ac.els-cdn.com/S1053811903005366/1-s2.0-S1053811903005366-main.pdf?_tid=b4eff368-3e5d-11e4-8c7e-00000aab0f27&acdnat=1410953471_c5a2dba1d0acdd9b6ef748fc21a01acd.
30. [30] H. Ayaz et al., "Optical imaging and spectroscopy for the study of the human brain: status report," Neurophotonics, vol. 9, no. Suppl 2, p. S24001, Aug 2022, doi: 10.1117/1.NPh.9.S2.S24001.
31. [31] M. A. Yucel et al., "Best practices for fNIRS publications," Neurophotonics, vol. 8, no. 1, p. 012101, Jan 2021, doi: 10.1117/1.NPh.8.1.012101.
32. [32] B. Molavi and G. A. Dumont, "Wavelet-based motion artifact removal for functional near-infrared spectroscopy," Physiological measurement, vol. 33, no. 2, pp. 259-70, Feb 2012, doi: 10.1088/0967-3334/33/2/259.
33. [33] V. N. Vapnik, "The Nature of Statistical Learning," Theory, 1995 1995. [Online]. Available: https://ci.nii.ac.jp/naid/10020951890/en/.
34. [34] J. Ying, Q. Wei, and X. Zhou, "Riemannian geometry-based transfer learning for reducing training time in c-VEP BCIs," Scientific reports, vol. 12, no. 1, p. 9818, Jun 14 2022, doi: 10.1038/s41598-022-14026-y.
35. [35] K. Won, M. Kwon, M. Ahn, and S. C. Jun, "Selective Subject Pooling Strategy to Improve Model Generalization for a Motor Imagery BCI," Sensors (Basel), vol. 21, no. 16, Aug 12 2021, doi: 10.3390/s21165436.
36. [36] A. M. Ray et al., "A subject-independent pattern-based Brain-Computer Interface," Frontiers in behavioral neuroscience, vol. 9, p. 269, 2015, doi: 10.3389/fnbeh.2015.00269.
37. [37] S. Montero-Hernandez et al., "Estimating Functional Connectivity Symmetry between Oxy- and Deoxy-Haemoglobin: Implications for fNIRS Connectivity Analysis," Algorithms, vol. 11, no. 5, 2018, doi: 10.3390/a11050070.
38. [38] H. Niu et al., "Resting-state functional connectivity assessed with two diffuse optical tomographic systems," Journal of biomedical optics, vol. 16, no. 4, p. 046006, Apr 2011, doi: 10.1117/1.3561687.
39. [39] Y. J. Zhang, C. M. Lu, B. B. Biswal, Y. F. Zang, D. L. Peng, and C. Z. Zhu, "Detecting resting-state functional connectivity in the language system using functional near-infrared spectroscopy," Journal of biomedical optics, vol. 15, no. 4, p. 047003, Jul-Aug 2010, doi: 10.1117/1.3462973.
40. [40] F. Homae et al., "Development of global cortical networks in early infancy," The Journal of neuroscience : the official journal of the Society for Neuroscience, vol. 30, no. 14, pp. 4877-82, Apr 7 2010, doi: 10.1523/JNEUROSCI.5618-09.2010.
41. [41] E. Kirilina et al., "The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy," Neuroimage, vol. 61, no. 1, pp. 70-81, May 15 2012, doi: 10.1016/j.neuroimage.2012.02.074.
42. [42] A. Jaramillo-Yanez, M. E. Benalcazar, and E. Mena-Maldonado, "Real-Time Hand Gesture Recognition Using Surface Electromyography and Machine Learning: A Systematic Literature Review," Sensors (Basel), vol. 20, no. 9, Apr 27 2020, doi: 10.3390/s20092467.
43. [43] J. Shin and J. Jeong, "Multiclass classification of hemodynamic responses for performance improvement of functional near-infrared spectroscopy-based brain-computer interface," Journal of biomedical optics, vol. 19, no. 6, p. 067009, Jun 2014, doi: 10.1117/1.JBO.19.6.067009.
44. [44] M. Stangl, G. Bauernfeind, J. Kurzmann, R. Scherer, and C. Neuper, "A Haemodynamic Brain–Computer Interface Based on Real-Time Classification of near Infrared Spectroscopy Signals during Motor Imagery and Mental Arithmetic," Journal of Near Infrared Spectroscopy, vol. 21, no. 3, pp. 157-171, 2013/06/01 2013, doi: 10.1255/jnirs.1048.
45. [45] R. Zimmermann et al., "Detection of motor execution using a hybrid fNIRS-biosignal BCI: a feasibility study," J Neuroeng Rehabil, vol. 10, p. 4, Jan 21 2013, doi: 10.1186/1743-0003-10-4.
46. [46] K. S. Hong, N. Naseer, and Y. H. Kim, "Classification of prefrontal and motor cortex signals for three-class fNIRS-BCI," Neurosci Lett, vol. 587, pp. 87-92, Feb 5 2015, doi: 10.1016/j.neulet.2014.12.029.
47. [47] P. Ortega and A. Faisal, "HemCNN: Deep Learning enables decoding of fNIRS cortical signals in hand grip motor tasks," in 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER), 4-6 May 2021 2021, pp. 718-721, doi: 10.1109/NER49283.2021.9441323.
48. [48] G. Derosiere, F. Alexandre, N. Bourdillon, K. Mandrick, T. E. Ward, and S. Perrey, "Similar scaling of contralateral and ipsilateral cortical responses during graded unimanual force generation," Neuroimage, vol. 85 Pt 1, pp. 471-7, Jan 15 2014, doi: 10.1016/j.neuroimage.2013.02.006.
49. [49] A. Abdalmalak et al., "Assessing Time-Resolved fNIRS for Brain-Computer Interface Applications of Mental Communication," Front Neurosci, vol. 14, p. 105, 2020, doi: 10.3389/fnins.2020.00105.