Brain Decoding over the MEG Signals Using Riemannian Approach and Machine Learning

Year 2023, , 207 - 218, 21.08.2023

Zeynep Özer , Onursal Çetin , Kutlucan Görür , Feyzullah Temurtaş

https://doi.org/10.17694/bajece.1144279

Abstract

Brain decoding is an emerging approach for understanding the face perception mechanism in the human brain. Face visual stimuli and perception mechanism are considered as a challenging ongoing research of the neuroscience field. In this study, face/scrambled face visual stimulations were implemented over the sixteen participants to be decoded the face or scrambled face classification using machine learning (ML) algorithms via magnetoencephalography (MEG) signals. This noninvasive and high spatial/temporal resolution signal is a neurophysiological technique which measures the magnetic fields generated by the neuronal activity of the brain. The Riemannian approach was used as a highly promising feature extraction technique. Then Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN) were employed as deep learning algorithms, Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) were implemented as shallow algorithms. The improved classification performances are very encouraging, especially for deep learning algorithms. The LSTM and GRU have achieved 92.99% and 91.66% accuracy and 0.977 and 0.973 of the area under the curve (AUC) scores, respectively. Moreover, CNN has yielded 90.62% accuracy. As our best knowledge, the improved outcomes and the usage of the deep learning on the MEG dataset signals from 16 participants are critical to expand the literature of brain decoding after visual stimuli. And this study is the first attempt with these methods in systematic comparison. Moreover, MEG-based Brain-Computer Interface (BCI) approaches may also be implemented for Internet of Things (IoT) applications, including biometric authentication, thanks to the specific stimuli of individual’s brainwaves.

Keywords

Magnetoencephalography, Brain Decoding, Riemannian Approach, Deep Learning.

References

• 1. Zarief, C. N., & Hussein, W. (2019). Decoding the Human Brain Activity and Predicting the Visual Stimuli from Magnetoencephalography (MEG) Recordings. In Proceedings of the 2019 International Conference on Intelligent Medicine and Image Processing - IMIP ’19 (pp. 35–42). New York, New York, USA: ACM Press. https://doi.org/10.1145/3332340.3332352
• 2. Lin, J.-F. L., Silva-Pereyra, J., Chou, C.-C., & Lin, F.-H. (2018). The sequence of cortical activity inferred by response latency variability in the human ventral pathway of face processing. Scientific Reports, 8(1), 5836. https://doi.org/10.1038/s41598-018-23942-x
• 3. Watanabe, S., Miki, K., & Kakigi, R. (2005). Mechanisms of face perception in humans: A magneto- and electro-encephalographic study. Neuropathology, 25(1), 8–20. https://doi.org/10.1111/j.1440-1789.2004.00603.x
• 4. Tadel, F., Bock, E., Niso, G., Mosher, J. C., Cousineau, M., Pantazis, D., … Baillet, S. (2019). MEG/EEG Group Analysis With Brainstorm. Frontiers in Neuroscience, 13(FEB), 1–21. https://doi.org/10.3389/fnins.2019.00076
• 5. Caliskan, A., Yuksel, M. E., Badem, H., & Basturk, A. (2017). A deep neural network classifier for decoding human brain activity based on magnetoencephalography. Elektronika Ir Elektrotechnika, 23(2), 63–67. https://doi.org/10.5755/j01.eie.23.2.18002
• 6. Özkaya, Ş. N., & Yıldırım, T. (2018). Assessment of Components and Methods Used to Identify Responses and Regions of Brain Related with Face Recognition and Perception. Procedia Computer Science, 131, 38–44. https://doi.org/10.1016/j.procs.2018.04.183
• 7. http://web.mit.edu/kitmitmeg/whatis.html. (n.d.). Retrieved from http://web.mit.edu/kitmitmeg/whatis.html
• 8. Kia, S. M., Vega Pons, S., Weisz, N., & Passerini, A. (2017). Interpretability of Multivariate Brain Maps in Linear Brain Decoding: Definition, and Heuristic Quantification in Multivariate Analysis of MEG Time-Locked Effects. Frontiers in Neuroscience, 10. https://doi.org/10.3389/fnins.2016.00619.
• 9. Cetin, O., & Temurtas, F. (2020). A comparative study on classification of magnetoencephalography signals using probabilistic neural network and multilayer neural network. Soft Computing. https://doi.org/10.1007/s00500-020-05296-7
• 10. Çetin, O., & Temurtaş, F. (2018). A Study on Brain Computer Interface using Learning Vector Quantization. Sakarya University Journal of Computer and Information Sciences, 1(2), 1–7.
• 11. Çetin, O., & Temurtaş, F. (2019). Classification of Magnetoencephalography Signals Regarding Visual Stimuli by Generalized Regression Neural Network. Dicle Tıp Dergisi, 45(3), 19–25. https://doi.org/10.5798/dicletip.534819.
• 12. Gorur, K., Bozkurt, M., Bascil, M., & Temurtas, F. (2019). GKP Signal Processing Using Deep CNN and SVM for Tongue-Machine Interface. Traitement Du Signal, 36(4), 319–329. https://doi.org/10.18280/ts.360404
• 13. Yeom, H. G., Kim, J. S., & Chung, C. K. (2020). LSTM Improves Accuracy of Reaching Trajectory Prediction From Magnetoencephalography Signals. IEEE Access, 8, 20146–20150. https://doi.org/10.1109/ACCESS.2020.2969720
• 14. Alom, M. Z., Moody, A. T., Maruyama, N., Van Essen, B. C., & Taha, T. M. (2018). Effective Quantization Approaches for Recurrent Neural Networks. In Proceedings of the International Joint Conference on Neural Networks (Vol. 2018-July). https://doi.org/10.1109/IJCNN.2018.8489341
• 15. Alpaydın, E. (2010). Introduction to Machine Learning. MIT Press, Cambridge, Massachusetts.
• 16. Ghojogh, B., & Crowley, M. (2019). Linear and Quadratic Discriminant Analysis: Tutorial, (4), 1–16. Retrieved from http://arxiv.org/abs/1906.02590
• 17. Barachant, A. (2014). MEG decoding using Riemannian Geometry and Unsupervised classification. Notes on the winner of the Kaggle “DecMeg2014 - Decoding the Human Brain” competition., 1–8.
• 18. Henson, R. N., Wakeman, D. G., Litvak, V., & Friston, K. J. (2011). A Parametric Empirical Bayesian Framework for the EEG/MEG Inverse Problem: Generative Models for Multi-Subject and Multi-Modal Integration. Frontiers in Human Neuroscience, 5(August), 1–16. https://doi.org/10.3389/fnhum.2011.00076
• 19. “DecMeg2014-Decoding the Human Brain”. [Online]. Available: https://www.kaggle.com/c/decoding- the-human-brain. (n.d.), 2014.
• 20. Olivetti, E., Kia, S. M., & Avesani, P. (2014). MEG decoding across subjects. Proceedings - 2014 International Workshop on Pattern Recognition in Neuroimaging, PRNI 2014. https://doi.org/10.1109/PRNI.2014.6858538
• 21. Roy, P. P., Kumar, P., & Chang, V. (2020). A hybrid classifier combination for home automation using EEG signals. Neural Computing and Applications, 32(14), 1–19. https://doi.org/10.1007/s00521-020-04804-y
• 22. Yger, F., Berar, M., & Lotte, F. (2017). Riemannian Approaches in Brain-Computer Interfaces: A Review. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(10), 1753–1762. https://doi.org/10.1109/TNSRE.2016.2627016
• 23. Barachant, A., Bonnet, S., Congedo, M., & Jutten, C. (2012). Multiclass brain-computer interface classification by Riemannian geometry. IEEE Transactions on Biomedical Engineering, 59(4), 920–928. https://doi.org/10.1109/TBME.2011.2172210
• 24. Uçar, M. K. (2020). Classification Performance-Based Feature Selection Algorithm for Machine Learning: P-Score. IRBM, 41(4), 229–239. https://doi.org/10.1016/j.irbm.2020.01.006
• 25. Bascil, M. S., Tesneli, A. Y., & Temurtas, F. (2015). Multi-channel EEG signal feature extraction and pattern recognition on horizontal mental imagination task of 1-D cursor movement for brain computer interface. Australasian Physical and Engineering Sciences in Medicine, 38(2), 229–239. https://doi.org/10.1007/s13246-015-0345-6
• 26. Hossin, M., & Sulaiman, M. . (2015). A Review on Evaluation Metrics for Data Classification Evaluations. International Journal of Data Mining & Knowledge Management Process, 5(2), 01–11. https://doi.org/10.5121/ijdkp.2015.5201
• 27. Toğaçar, M., Ergen, B., Cömert, Z., & Özyurt, F. (2020). A Deep Feature Learning Model for Pneumonia Detection Applying a Combination of mRMR Feature Selection and Machine Learning Models. IRBM, 41(4), 212–222. https://doi.org/10.1016/j.irbm.2019.10.006
• 28. Yang, S., & Berdine, G. (2017). The receiver operating characteristic (ROC) curve. The Southwest Respiratory and Critical Care Chronicles, 5(19), 34. https://doi.org/10.12746/swrccc.v5i19.391
• 29. Muller, K.-R., Mika, S., Ratsch, G., Tsuda, K., & Scholkopf, B. (2001). An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 12(2), 181–201. https://doi.org/10.1109/72.914517
• 30. Koudjonou, K. M., & Rout, M. (2020). A stateless deep learning framework to predict net asset value. Neural Computing and Applications, 32(14), 1–19. https://doi.org/10.1007/s00521-019-04525-x
• 31. Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791
• 32. Ozer, I., Ozer, Z., & Findik, O. (2018). Noise robust sound event classification with convolutional neural network. Neurocomputing, 272, 505–512. https://doi.org/10.1016/j.neucom.2017.07.021
• 33. Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology, 195(1), 215–243. https://doi.org/10.1113/jphysiol.1968.sp008455
• 34. Ma, J., Wu, F., Zhu, J., Xu, D., & Kong, D. (2017). A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics, 73, 221–230. https://doi.org/10.1016/j.ultras.2016.09.011
• 35. Golmohammadi, M., Ziyabari, S., Shah, V., Von Weltin, E., Campbell, C., Obeid, I., & Picone, J. (2017). Gated recurrent networks for seizure detection. In 2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB) (Vol. 2018-Janua, pp. 1–5). IEEE. https://doi.org/10.1109/SPMB.2017.8257020
• 36. Hadjikhani, N., Kveraga, K., Naik, P., & Ahlfors, S. P. (2009). Early (M170) activation of face-specific cortex by face-like objects. NeuroReport, 20(4), 403–407. https://doi.org/10.1097/WNR.0b013e328325a8e1
• 37. Wakeman, D. G., & Henson, R. N. (2015). A multi-subject, multi-modal human neuroimaging dataset. Scientific Data, 2(1), 150001. https://doi.org/10.1038/sdata.2015.1
• 38. Dash, D., Ferrari, P., & Wang, J. (2020). Decoding Imagined and Spoken Phrases From Non-invasive Neural (MEG) Signals. Frontiers in Neuroscience, 14(September 2004), 8–20. https://doi.org/10.3389/fnins.2020.00290
• 39. Gupta, S., & Gandhi, T. (2020). Identification of Neural Correlates of Face Recognition Using Machine Learning Approach. In Computer Vision and Machine Intelligence in Medical Image Analysis (pp. 13–20). https://doi.org/10.1007/978-981-13-8798-2_2
• 40. Li, J., Pan, J., Wang, F., & Yu, Z. (2021). Inter-Subject MEG Decoding for Visual Information with Hybrid Gated Recurrent Network. Applied Sciences, 11(3), 1215. https://doi.org/10.3390/app11031215
• 41. Liu, C., Kang, Y., Zhang, L., & Zhang, J. (2021). Rapidly Decoding Image Categories From MEG Data Using a Multivariate Short-Time FC Pattern Analysis Approach. IEEE Journal of Biomedical and Health Informatics, 25(4), 1139–1150. https://doi.org/10.1109/JBHI.2020.3008731
• 42. Huang, H., Hu, L., Xiao, F., Du, A., Ye, N., & He, F. (2019). An EEG-Based Identity Authentication System with Audiovisual Paradigm in IoT. Sensors, 19(7), 1664. https://doi.org/10.3390/s19071664
• 43. Sooriyaarachchi, J., Seneviratne, S., Thilakarathna, K., & Zomaya, A. Y. (2021). MusicID: A Brainwave-Based User Authentication System for Internet of Things. IEEE Internet of Things Journal, 8(10), 8304–8313. https://doi.org/10.1109/JIOT.2020.3044726