EEG+YKS Tabanlı Beyin Bilgisayar Arayüzü Sistemlerinde Makine Öğrenme Tekniklerinin Kullanımına İlişkin İnceleme

Yıl 2024, Cilt: 14 Sayı: 1, 39 - 49, 30.01.2024

Ebru Ergün Önder Aydemir Onur Erdem Korkmaz

Öz

Beyin-bilgisayar arayüzü (BBA), kişinin beyin aktivitesini algılayan ve bu aktiviteyi bir bilgisayar veya diğer dış cihazlarla iletişim kurmak için kullanılabilir bir formata çeviren sistemdir. BBA sistemleri için çeşitli beyin görüntüleme teknikleri, giriş işareti olarak kullanılmaktadır. Diğer alternatiflere göre birçok avantajı olmasından dolayı elektroensefalografi (EEG), BBA sistemlerinde sıkça tercih edilmektedir. Ancak, sadece EEG kullanarak yapılan çalışmalar, BBA sistemlerinin performansını tatmin edici bir seviyeye yükseltememiştir. Son yıllarda, beyin görüntüleme yöntemlerinin kendilerine özgü avantaj ve dezavantajlarından ötürü farklı sinyal kayıt yöntemlerini bir araya getirerek kullanma eğilimi artmıştır. Bu çalışmada, EEG+ yakın kızılötesi spektroskopisi (YKS) tabanlı BBA sistemlerinde makine öğrenme tekniklerinin kullanımına ilişkin yapılan mevcut araştırmaların bir incelemesi sunulmaktadır. Öncelikle, EEG ve YKS sinyallerinin tekli nasıl kullanıldığı ve daha sonra bu iki sinyalin nasıl hibrit BBA sistemlerinde bir araya getirildiği ilgili literatürle sunulmuştur. Çalışmalar incelendiğinde, hibrit BBA ile EEG’ye göre sınıflandırma doğruluğu ortalama %7.58, YKS’ye göre ise %13.04 artış hesaplanmıştır. Ulaşılan sonuç, hibrit BBA sistemlerinin insan-makine etkileşimini iyileştirmeye önemli katkılar sağlayacağını göstermektedir.

Anahtar Kelimeler

Hibrit, Elektroensefalografi, Yakın Kızılötesi Spektroskopisi, Beyin Bilgisayar Arayüzü, Sınıflandırma, Öznitelik Çıkarma

Etik Beyan

Veri setleri proje için Karadeniz Teknik Üniversitesi Etik Kurulu tarafından onaylandı ve Helsinki Deklarasyonu kapsamında yürütüldü.

Destekleyen Kurum

Karadeniz Teknik Üniversitesi

Proje Numarası

FHD-2020-9166

Teşekkür

Bu çalışmada kullanılan veriler Atatürk Üniversitesi Spor Bilimleri Uygulama ve Araştırma Merkezi'nde kayıt altına alınmıştır. Bu çalışma Karadeniz Teknik Üniversitesi Bilimsel Araştırma Projeleri Koordinatörlüğü tarafından FHD-2020-9166 proje numarasıyla desteklenmiştir. Ebru Ergun, 2211-C Yurt İçi Öncelikli Alanlar Doktora Burs Programı kapsamında TÜBİTAK bursuyla desteklendi.

Kaynakça

• [1] S. Aggarwal, and N. Chugh, “Review of machine learning techniques for EEG based brain computer interface”, Archives of Computational Methods in Engineering, vol. 1, no. 20, 2022.
• [2] A. Naser, and O. Aydemir, “Classification of pleasant and unpleasant odor imagery EEG signals”, Neural Computing and Applications, vol. 35, no. 12, pp. 9105-9114, 2023.
• [3] O. E. Korkmaz, O. Aydemir, E. A. Oral, and I. Y. Ozbek, “A novel probabilistic and 3D column P300 stimulus presentation paradigm for EEG-based spelling systems”, Neural Computing and Applications, pp. 1-15, 2023.
• [4] E. Ergün, and O. Aydemir, “A new evolutionary preprocessing approach for classification of mental arithmetic based EEG signals”, Cognitive Neurodynamics, vol. 14, pp. 609-617, 2020.
• [5] Z. Liu, J. Shore, M. Wang, F. Yuan, A. Buss, and X. Zhao, “A systematic review on hybrid EEG/fNIRS in brain-computer interface”, Biomedical Signal Processing and Control, vol. 68, no. 102595, 2021.
• [6] D. Pirrone, E. Weitschek, P. Di Paolo, S. De Salvo, and M. C. De Cola, “ Eeg signal processing and supervised machine learning to early diagnose alzheimer’s disease”, Applied sciences, vol. 12, no. 11, 2021.
• [7] P. C. Sharma, R. Raja, S. K. Vishwakarma, S. Sharma, P. K. Mishra, and V. S. Kushwah, “Analysis of brain signal processing and real-time EEG signal enhancement”, Multimedia Tools and Applications, vol. 81, no. 28, pp. 41013-41033, 2022.
• [8] Ö. Aydemir, “Beyin Bilgisayar Arayüzü ve Geleceği”, Journal of Investigations on Engineering and Technology, vol. 5, no. 2, pp. 135-141, 2022.
• [9] N. Panigrahi, and S. P. Mohanty, “Brain Computer Interface: EEG Signal Processing”, CRC Press, 2022.
• [10] R. Chai, S.H. Ling, G.P. Hunter, H.T. Nguyen, “Mental non-motor imagery tasks classifications of brain computer interface for wheelchair commands using genetic algorithm-based neural network”, Proc. Int. Joint Conf, Neural Networks, pp. 10–15, 2012.
• [11] N.K. Verma, L.S.V.S. Rao, S.K. Sharma, “Motor imagery EEG signal classification on DWT and crosscorrelated signal features”, 9th Int. Conf. Indust. Inform. Syst., 2015.
• [12] M.H. Bhatti, et al., “Soft computing-based EEG classification by optimal feature selection and neural networks”, IEEE Trans. Ind. Inf, vol. 15, no. 10, pp. 5747–5754, 2019.
• [13] N. Bagh, M.R. Reddy, “Hilbert Transform-Based Event-Related Patterns for Motor Imagery Brain Computer Interface”, Biomedical Signal Processing and Control, vol. 62, 2020.
• [14] Q. Wang ve O. Sourina, “Real-time mental arithmetic task recognition from EEG signals”, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 21 no. 2, pp. 225-232, 2013.
• [15] W. Yi, S. Qiu, H. Qi, L. Zhang, B. Wan, D. Ming, “EEG feature comparison and classification of simple and compound limb motor imagery”, J. NeuroEng. Rehabil, vol. 10, no. 1, pp. 1–12, 2013.
• [16] I. Dokare, N. Kant, “Performance analysis of SVM, k-NN and BPNN classifiers for motor imagery”, Int. J. Eng. Trends Technol, vol. 10, no. 1, pp. 9–23, 2014.
• [17] C. Lindig-Leon, L. Bougrain, “A multi-label classification method for detection of combined motor imageries, in: Proceedings”, IEEE International Conference on Systems, Man, and Cybernetics, 2016.
• [18] A., Koven, and S. Herschorn, “NIRS: past, present, and future in functional urology”, Current Bladder Dysfunction Reports, vol. 17, no. 4, pp. 241-249, 2022.
• [19] T. W. L. Scheeren, P. Schober, and L. A. Schwarte, “Monitoring tissue oxygenation by near infrared spectroscopy (NIRS): background and current applications”, Journal of clinical monitoring and computing, vol. 26, pp. 279-287, 2012.
• [20] E. Ergün, and Ö. Aydemir, “Decoding of binary mental arithmetic based near-infrared spectroscopy signals”, In 2018 3rd International Conference on Computer Science and Engineering (UBMK), pp. 201-204, 2018.
• [21] N. Naseer, and K. S., Hong, “Decoding answers to four-choice questions using functional near infrared spectroscopy”, Journal of Near Infrared Spectroscopy, vol. 23, no. 1, pp. 23-31, 2015.
• [22] R. Li, T. Potter, W. Huang, and Y. Zhang, “Enhancing performance of a hybrid eeg-fnirs system using channel selection and early temporal features”, Frontiers in Human Neuroscience, pp. 11-462, 2015.
• [23] Y. Tomita, F. B. Vialatte, G. Dreyfus, Y. Mitsukura, H. Bakardjian, and A. Cichocki, “Bimodal BCI using simultaneously NIRS and EEG”, IEEE Transactions on Biomedical Engineering, vol. 61, no. 4, pp. 1274-1284, 2014.
• [24] Y. Blokland, L. Spyrou, D. Thijssen, T. Eijsvogels, W. Colier, M. Floor-Westerdijk, vd., “Combined EEG-fNIRS decoding of motor attempt and imagery for brain switch control: an offline study in patients with tetraplegia” IEEE transactions on neural systems and rehabilitation engineering, vol. 22, no. 2, pp. 222-229, 2013.
• [25] P. A. Cicalese, R. Li, M. B. Ahmadi, C. Wang, J. T. Francis, S. Selvaraj, ve Y. Zhang, “An EEG-fNIRS hybridization technique in the four-class classification of alzheimer’s disease”, Journal of neuroscience methods, vol. 336, pp. 108-618, 2020.
• [26] M. A. Hasan, M. U. Khan, and D., Mishra, “A computationally efficient method for hybrid EEG-fNIRS BCI based on the Pearson correlation”, BioMed Research International, pp. 1-13, 2020.
• [27] E. Ergün, “Elektroensefalografi ve Yakın Kızılötesi Spektroskopi Tabanlı Hibrit Model Kullanarak Beyin Bilgisayar Arayüzü Sisteminin Performansının Arttırılması”, Doktora Tezi, Karadeniz Teknik Üniversitesi, Fen Bilimleri Enstitüsü, Trabzon, 2023.
• [28] F. Putze, S. Hesslinger, C. Y. Tse, Y. Y. Huang, C. Herff, C. T. Guan, vd., “Hybrid fNIRS-EEG based classification of auditory and visual perception processes”, Frontiers in Neuroscience, vol. 8, no. 373, 2014.
• [29] B. Koo, H. G. Lee, Y. Nam, H. Kang, C. S. Koh, H. C. Shin, vd., “A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery”, Journal of Neuroscience Methods, vol. 244, pp. 26–32, 2015.
• [30] H. Aghajani, M. Garbey, and A. Omurtag, “Measuring mental workload with EEG plus fNIRS”, Frontiers in human neuroscience, vol. 11, no. 359, pp. 1-20, 2017.
• [31] J. Shin, J. Kwon, and C. H. Im, “A ternary hybrid EEG-NIRS brain-computer interface for the classification of brain activation patterns during mental arithmetic, motor imagery, and idle state”, Frontiers in neuroinformatics, vol. 12, no. 5, pp. 1-9, 2018.
• [32] M. Nour, Ş. Öztürk, and K. Polat, “A novel classification framework using multiple bandwidth method with optimized CNN for brain–computer interfaces with EEG-fNIRS signals”, Neural Computing and Applications, vol. 33, no. 22, pp. 15815-15829, 2021.
• [33] Y. Zhang, Y. Li, L. Kong, Q. Niu, ve Y. Bai, “Improved DBSCAN Spindle Bearing Condition Monitoring Method Based on Kurtosis and Sample Entropy”, Machines, vol. 10, no. 5, 2022.
• [34] A. M. Chiarelli, P. Croce, A. Merla ve F. Zappasodi, “Deep learning for hybrid EEG-fNIRS brain–computer interface: application to motor imagery classification”, Journal of neural engineering, 15, 3, 036028, 2018.
• [35] M. A. Rahman, M. S. Uddin ve M. Ahmad, “Modeling and classification of voluntary and imagery movements for brain–computer interface from fNIR and EEG signals through convolutional neural network”, Health Information Science and Systems, 7, 1, 22, 2019.
• [36] M. Nour, Ş. Öztürk ve K. Polat, “A novel classification framework using multiple bandwidth method with optimized CNN for brain–computer interfaces with EEG-fNIRS signals”, Neural Computing and Applications, 33, 15815-15829, 2021.
• [37] M. H. R. Rabbani ve S. M. R. Islam, “Deep learning networks based decision fusion model of EEG and fNIRS for classification of cognitive tasks”, Cognitive Neurodynamics, 1-18, 2023.
• [38] J. Shin, A. von Lühmann, B. Blankertz, D. W. Kim, J. Jeong, H. J. Hwang, and K. R. Müller, “Open access dataset for EEG+ NIRS single-trial classification”, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25, 10, 1735-1745, 2016.
• [39] X. X. Yin, B. L. Xu, C. H. Jiang, Y. F. Fu, Z. D.Wang, H. Y. Li, vd., “A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching”, Journal of Neural Engineering, vol. 12, no. 036004, 2015.
• [40] M. S. Al-Quraishi, I. Elamvazuthi, T. B. Tang, M. Al-Qurishi, S. H. Adil, ve M. Ebrahim, “Bimodal data fusion of simultaneous measurements of EEG and fNIRS during lower limb movements”, Brain Sciences, vol. 11, no. 6, pp. 1-713, 2021.
• [41] E. Ergün ve Ö. Aydemir, “A hybrid BCI using singular value decomposition values of the fast walsh hadamard transform coefficients”, IEEE Transactions on Cognitive and Developmental Systems, pp. 1-10, 2022.
• [42] A. P. Buccino, H. O. Keles ve A. Omurtag, “Hybrid EEG-fNIRS asynchronous brain–computer interface for multiple motor tasks”, Plos One, vol. 11, no. 1, pp. 1-8, 2016.
• [43] Q. He, L. Feng, G. Jiang, and P. Xie, “Multimodal Multitask Neural Network for Motor Imagery Classification With EEG and fNIRS Signals”, IEEE Sensors Journal, vol. 22, no. 21, pp. 20695-20706, 2021.
• [44] S. Fazli, J. Mehnert, J. Steinbrink, G. Curio, A. Villringer, K. R. Müller, vd., “Enhanced performance by a hybrid NIRS-EEG brain computer interface”, Neuroimage, vol. 59, pp. 519–529, 2012.
• [45] F. Putze, S. Hesslinger, C. Y. Tse, Y. Y. Huang, C. Herff, C. T. Guan, vd., “Hybrid fNIRS-EEG based classification of auditory and visual perception processes, Frontiers in Neuroscience”, vol. 8, no. 373, 2014.
• [46] S. Ge, Q. Yang, R. M. Wang, P. Lin, J. F. Gao, Y. Leng, vd., “A brain computer interface based on a few-channel EEG-fNIRS bimodal system”, IEEE Access, vol. 5, pp. 208–218, 2017.
• [47] S. Ahn, T. Nguyen, H. Jang, J. G. Kim, ve S. C. Jun, “Exploring neuro- physiological correlates of drivers’ mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data”, Frontiers in human neuroscience, vol. 10, no. 219, pp. 1-14, 2016.
• [48] O. Aydemir, and E. Ergün, “A robust and subject-specific sequential forward search method for effective channel selection in brain computer interfaces”, Journal of neuroscience methods, vol. 313, pp. 60-67, 2019.