Balkan Journal of Electrical and Computer Engineering

2147-284X 2147-284X

Balkan Yayın

10.17694/bajece.1144279

Artificial Intelligence

Yapay Zeka

Brain Decoding over the MEG Signals Using Riemannian Approach and Machine Learning

https://orcid.org/0000-0001-8654-0902

Özer

Zeynep

BANDIRMA ONYEDİ EYLÜL ÜNİVERSİTESİ, ÖMER SEYFETTİN UYGULAMALI BİLİMLER FAKÜLTESİ

https://orcid.org/0000-0001-5220-3959

Çetin

Onursal

BANDIRMA ONYEDİ EYLÜL ÜNİVERSİTESİ

https://orcid.org/0000-0003-3578-0150

Görür

Kutlucan

BANDIRMA ONYEDİ EYLÜL ÜNİVERSİTESİ

https://orcid.org/0000-0002-3158-4032

Temurtaş

Feyzullah

BANDIRMA ONYEDİ EYLÜL ÜNİVERSİTESİ

08 21 2023

11 3 207 218 07 17 2022 08 18 2022

2013

Balkan Journal of Electrical and Computer Engineering

Brain decoding is an emerging approach for understanding the face perception mechanism in the human brain. Face visual stimuli and perception mechanism are considered as a challenging ongoing research of the neuroscience field. In this study, face/scrambled face visual stimulations were implemented over the sixteen participants to be decoded the face or scrambled face classification using machine learning (ML) algorithms via magnetoencephalography (MEG) signals. This noninvasive and high spatial/temporal resolution signal is a neurophysiological technique which measures the magnetic fields generated by the neuronal activity of the brain. The Riemannian approach was used as a highly promising feature extraction technique. Then Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN) were employed as deep learning algorithms, Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) were implemented as shallow algorithms. The improved classification performances are very encouraging, especially for deep learning algorithms. The LSTM and GRU have achieved 92.99% and 91.66% accuracy and 0.977 and 0.973 of the area under the curve (AUC) scores, respectively. Moreover, CNN has yielded 90.62% accuracy. As our best knowledge, the improved outcomes and the usage of the deep learning on the MEG dataset signals from 16 participants are critical to expand the literature of brain decoding after visual stimuli. And this study is the first attempt with these methods in systematic comparison. Moreover, MEG-based Brain-Computer Interface (BCI) approaches may also be implemented for Internet of Things (IoT) applications, including biometric authentication, thanks to the specific stimuli of individual’s brainwaves.

Magnetoencephalography Brain Decoding Riemannian Approach Deep Learning.

1. Zarief, C. N., & Hussein, W. (2019). Decoding the Human Brain Activity and Predicting the Visual Stimuli from Magnetoencephalography (MEG) Recordings. In Proceedings of the 2019 International Conference on Intelligent Medicine and Image Processing - IMIP ’19 (pp. 35–42). New York, New York, USA: ACM Press. https://doi.org/10.1145/3332340.3332352

2. Lin, J.-F. L., Silva-Pereyra, J., Chou, C.-C., & Lin, F.-H. (2018). The sequence of cortical activity inferred by response latency variability in the human ventral pathway of face processing. Scientific Reports, 8(1), 5836. https://doi.org/10.1038/s41598-018-23942-x

3. Watanabe, S., Miki, K., & Kakigi, R. (2005). Mechanisms of face perception in humans: A magneto- and electro-encephalographic study. Neuropathology, 25(1), 8–20. https://doi.org/10.1111/j.1440-1789.2004.00603.x

4. Tadel, F., Bock, E., Niso, G., Mosher, J. C., Cousineau, M., Pantazis, D., … Baillet, S. (2019). MEG/EEG Group Analysis With Brainstorm. Frontiers in Neuroscience, 13(FEB), 1–21. https://doi.org/10.3389/fnins.2019.00076

5. Caliskan, A., Yuksel, M. E., Badem, H., & Basturk, A. (2017). A deep neural network classifier for decoding human brain activity based on magnetoencephalography. Elektronika Ir Elektrotechnika, 23(2), 63–67. https://doi.org/10.5755/j01.eie.23.2.18002

6. Özkaya, Ş. N., & Yıldırım, T. (2018). Assessment of Components and Methods Used to Identify Responses and Regions of Brain Related with Face Recognition and Perception. Procedia Computer Science, 131, 38–44. https://doi.org/10.1016/j.procs.2018.04.183

7. http://web.mit.edu/kitmitmeg/whatis.html. (n.d.). Retrieved from http://web.mit.edu/kitmitmeg/whatis.html

8. Kia, S. M., Vega Pons, S., Weisz, N., & Passerini, A. (2017). Interpretability of Multivariate Brain Maps in Linear Brain Decoding: Definition, and Heuristic Quantification in Multivariate Analysis of MEG Time-Locked Effects. Frontiers in Neuroscience, 10. https://doi.org/10.3389/fnins.2016.00619.

9. Cetin, O., & Temurtas, F. (2020). A comparative study on classification of magnetoencephalography signals using probabilistic neural network and multilayer neural network. Soft Computing. https://doi.org/10.1007/s00500-020-05296-7

10. Çetin, O., & Temurtaş, F. (2018). A Study on Brain Computer Interface using Learning Vector Quantization. Sakarya University Journal of Computer and Information Sciences, 1(2), 1–7.

11. Çetin, O., & Temurtaş, F. (2019). Classification of Magnetoencephalography Signals Regarding Visual Stimuli by Generalized Regression Neural Network. Dicle Tıp Dergisi, 45(3), 19–25. https://doi.org/10.5798/dicletip.534819.

12. Gorur, K., Bozkurt, M., Bascil, M., & Temurtas, F. (2019). GKP Signal Processing Using Deep CNN and SVM for Tongue-Machine Interface. Traitement Du Signal, 36(4), 319–329. https://doi.org/10.18280/ts.360404

13. Yeom, H. G., Kim, J. S., & Chung, C. K. (2020). LSTM Improves Accuracy of Reaching Trajectory Prediction From Magnetoencephalography Signals. IEEE Access, 8, 20146–20150. https://doi.org/10.1109/ACCESS.2020.2969720

14. Alom, M. Z., Moody, A. T., Maruyama, N., Van Essen, B. C., & Taha, T. M. (2018). Effective Quantization Approaches for Recurrent Neural Networks. In Proceedings of the International Joint Conference on Neural Networks (Vol. 2018-July). https://doi.org/10.1109/IJCNN.2018.8489341

15. Alpaydın, E. (2010). Introduction to Machine Learning. MIT Press, Cambridge, Massachusetts.

16. Ghojogh, B., & Crowley, M. (2019). Linear and Quadratic Discriminant Analysis: Tutorial, (4), 1–16. Retrieved from http://arxiv.org/abs/1906.02590

17. Barachant, A. (2014). MEG decoding using Riemannian Geometry and Unsupervised classification. Notes on the winner of the Kaggle “DecMeg2014 - Decoding the Human Brain” competition., 1–8.

18. Henson, R. N., Wakeman, D. G., Litvak, V., & Friston, K. J. (2011). A Parametric Empirical Bayesian Framework for the EEG/MEG Inverse Problem: Generative Models for Multi-Subject and Multi-Modal Integration. Frontiers in Human Neuroscience, 5(August), 1–16. https://doi.org/10.3389/fnhum.2011.00076

19. “DecMeg2014-Decoding the Human Brain”. [Online]. Available: https://www.kaggle.com/c/decoding- the-human-brain. (n.d.), 2014.

20. Olivetti, E., Kia, S. M., & Avesani, P. (2014). MEG decoding across subjects. Proceedings - 2014 International Workshop on Pattern Recognition in Neuroimaging, PRNI 2014. https://doi.org/10.1109/PRNI.2014.6858538

21. Roy, P. P., Kumar, P., & Chang, V. (2020). A hybrid classifier combination for home automation using EEG signals. Neural Computing and Applications, 32(14), 1–19. https://doi.org/10.1007/s00521-020-04804-y

22. Yger, F., Berar, M., & Lotte, F. (2017). Riemannian Approaches in Brain-Computer Interfaces: A Review. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(10), 1753–1762. https://doi.org/10.1109/TNSRE.2016.2627016

23. Barachant, A., Bonnet, S., Congedo, M., & Jutten, C. (2012). Multiclass brain-computer interface classification by Riemannian geometry. IEEE Transactions on Biomedical Engineering, 59(4), 920–928. https://doi.org/10.1109/TBME.2011.2172210

24. Uçar, M. K. (2020). Classification Performance-Based Feature Selection Algorithm for Machine Learning: P-Score. IRBM, 41(4), 229–239. https://doi.org/10.1016/j.irbm.2020.01.006

25. Bascil, M. S., Tesneli, A. Y., & Temurtas, F. (2015). Multi-channel EEG signal feature extraction and pattern recognition on horizontal mental imagination task of 1-D cursor movement for brain computer interface. Australasian Physical and Engineering Sciences in Medicine, 38(2), 229–239. https://doi.org/10.1007/s13246-015-0345-6

26. Hossin, M., & Sulaiman, M. . (2015). A Review on Evaluation Metrics for Data Classification Evaluations. International Journal of Data Mining & Knowledge Management Process, 5(2), 01–11. https://doi.org/10.5121/ijdkp.2015.5201

27. Toğaçar, M., Ergen, B., Cömert, Z., & Özyurt, F. (2020). A Deep Feature Learning Model for Pneumonia Detection Applying a Combination of mRMR Feature Selection and Machine Learning Models. IRBM, 41(4), 212–222. https://doi.org/10.1016/j.irbm.2019.10.006

28. Yang, S., & Berdine, G. (2017). The receiver operating characteristic (ROC) curve. The Southwest Respiratory and Critical Care Chronicles, 5(19), 34. https://doi.org/10.12746/swrccc.v5i19.391

29. Muller, K.-R., Mika, S., Ratsch, G., Tsuda, K., & Scholkopf, B. (2001). An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 12(2), 181–201. https://doi.org/10.1109/72.914517

30. Koudjonou, K. M., & Rout, M. (2020). A stateless deep learning framework to predict net asset value. Neural Computing and Applications, 32(14), 1–19. https://doi.org/10.1007/s00521-019-04525-x

31. Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791

32. Ozer, I., Ozer, Z., & Findik, O. (2018). Noise robust sound event classification with convolutional neural network. Neurocomputing, 272, 505–512. https://doi.org/10.1016/j.neucom.2017.07.021

33. Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology, 195(1), 215–243. https://doi.org/10.1113/jphysiol.1968.sp008455

34. Ma, J., Wu, F., Zhu, J., Xu, D., & Kong, D. (2017). A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics, 73, 221–230. https://doi.org/10.1016/j.ultras.2016.09.011

35. Golmohammadi, M., Ziyabari, S., Shah, V., Von Weltin, E., Campbell, C., Obeid, I., & Picone, J. (2017). Gated recurrent networks for seizure detection. In 2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB) (Vol. 2018-Janua, pp. 1–5). IEEE. https://doi.org/10.1109/SPMB.2017.8257020

36. Hadjikhani, N., Kveraga, K., Naik, P., & Ahlfors, S. P. (2009). Early (M170) activation of face-specific cortex by face-like objects. NeuroReport, 20(4), 403–407. https://doi.org/10.1097/WNR.0b013e328325a8e1

37. Wakeman, D. G., & Henson, R. N. (2015). A multi-subject, multi-modal human neuroimaging dataset. Scientific Data, 2(1), 150001. https://doi.org/10.1038/sdata.2015.1

38. Dash, D., Ferrari, P., & Wang, J. (2020). Decoding Imagined and Spoken Phrases From Non-invasive Neural (MEG) Signals. Frontiers in Neuroscience, 14(September 2004), 8–20. https://doi.org/10.3389/fnins.2020.00290

39. Gupta, S., & Gandhi, T. (2020). Identification of Neural Correlates of Face Recognition Using Machine Learning Approach. In Computer Vision and Machine Intelligence in Medical Image Analysis (pp. 13–20). https://doi.org/10.1007/978-981-13-8798-2_2

40. Li, J., Pan, J., Wang, F., & Yu, Z. (2021). Inter-Subject MEG Decoding for Visual Information with Hybrid Gated Recurrent Network. Applied Sciences, 11(3), 1215. https://doi.org/10.3390/app11031215

41. Liu, C., Kang, Y., Zhang, L., & Zhang, J. (2021). Rapidly Decoding Image Categories From MEG Data Using a Multivariate Short-Time FC Pattern Analysis Approach. IEEE Journal of Biomedical and Health Informatics, 25(4), 1139–1150. https://doi.org/10.1109/JBHI.2020.3008731

42. Huang, H., Hu, L., Xiao, F., Du, A., Ye, N., & He, F. (2019). An EEG-Based Identity Authentication System with Audiovisual Paradigm in IoT. Sensors, 19(7), 1664. https://doi.org/10.3390/s19071664

43. Sooriyaarachchi, J., Seneviratne, S., Thilakarathna, K., & Zomaya, A. Y. (2021). MusicID: A Brainwave-Based User Authentication System for Internet of Things. IEEE Internet of Things Journal, 8(10), 8304–8313. https://doi.org/10.1109/JIOT.2020.3044726