Brain Decoding over the MEG Signals Using Riemannian Approach and Machine Learning

Abstract

Brain decoding is an emerging approach for understanding the face perception mechanism in the human brain. Face visual stimuli and perception mechanism are considered as a challenging ongoing research of the neuroscience field. In this study, face/scrambled face visual stimulations were implemented over the sixteen participants to be decoded the face or scrambled face classification using machine learning (ML) algorithms via magnetoencephalography (MEG) signals. This noninvasive and high spatial/temporal resolution signal is a neurophysiological technique which measures the magnetic fields generated by the neuronal activity of the brain. The Riemannian approach was used as a highly promising feature extraction technique. Then Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN) were employed as deep learning algorithms, Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) were implemented as shallow algorithms. The improved classification performances are very encouraging, especially for deep learning algorithms. The LSTM and GRU have achieved 92.99% and 91.66% accuracy and 0.977 and 0.973 of the area under the curve (AUC) scores, respectively. Moreover, CNN has yielded 90.62% accuracy. As our best knowledge, the improved outcomes and the usage of the deep learning on the MEG dataset signals from 16 participants are critical to expand the literature of brain decoding after visual stimuli. And this study is the first attempt with these methods in systematic comparison. Moreover, MEG-based Brain-Computer Interface (BCI) approaches may also be implemented for Internet of Things (IoT) applications, including biometric authentication, thanks to the specific stimuli of individual’s brainwaves.

Keywords

• Magnetoencephalography
• Brain Decoding
• Riemannian Approach
• Deep Learning.

References

1. 1. Zarief, C. N., & Hussein, W. (2019). Decoding the Human Brain Activity and Predicting the Visual Stimuli from Magnetoencephalography (MEG) Recordings. In Proceedings of the 2019 International Conference on Intelligent Medicine and Image Processing - IMIP ’19 (pp. 35–42). New York, New York, USA: ACM Press. https://doi.org/10.1145/3332340.3332352
2. 2. Lin, J.-F. L., Silva-Pereyra, J., Chou, C.-C., & Lin, F.-H. (2018). The sequence of cortical activity inferred by response latency variability in the human ventral pathway of face processing. Scientific Reports, 8(1), 5836. https://doi.org/10.1038/s41598-018-23942-x
3. 3. Watanabe, S., Miki, K., & Kakigi, R. (2005). Mechanisms of face perception in humans: A magneto- and electro-encephalographic study. Neuropathology, 25(1), 8–20. https://doi.org/10.1111/j.1440-1789.2004.00603.x
4. 4. Tadel, F., Bock, E., Niso, G., Mosher, J. C., Cousineau, M., Pantazis, D., … Baillet, S. (2019). MEG/EEG Group Analysis With Brainstorm. Frontiers in Neuroscience, 13(FEB), 1–21. https://doi.org/10.3389/fnins.2019.00076
5. 5. Caliskan, A., Yuksel, M. E., Badem, H., & Basturk, A. (2017). A deep neural network classifier for decoding human brain activity based on magnetoencephalography. Elektronika Ir Elektrotechnika, 23(2), 63–67. https://doi.org/10.5755/j01.eie.23.2.18002
6. 6. Özkaya, Ş. N., & Yıldırım, T. (2018). Assessment of Components and Methods Used to Identify Responses and Regions of Brain Related with Face Recognition and Perception. Procedia Computer Science, 131, 38–44. https://doi.org/10.1016/j.procs.2018.04.183
7. 7. http://web.mit.edu/kitmitmeg/whatis.html. (n.d.). Retrieved from http://web.mit.edu/kitmitmeg/whatis.html
8. 8. Kia, S. M., Vega Pons, S., Weisz, N., & Passerini, A. (2017). Interpretability of Multivariate Brain Maps in Linear Brain Decoding: Definition, and Heuristic Quantification in Multivariate Analysis of MEG Time-Locked Effects. Frontiers in Neuroscience, 10. https://doi.org/10.3389/fnins.2016.00619.

9. 9. Cetin, O., & Temurtas, F. (2020). A comparative study on classification of magnetoencephalography signals using probabilistic neural network and multilayer neural network. Soft Computing. https://doi.org/10.1007/s00500-020-05296-7
10. 10. Çetin, O., & Temurtaş, F. (2018). A Study on Brain Computer Interface using Learning Vector Quantization. Sakarya University Journal of Computer and Information Sciences, 1(2), 1–7.
11. 11. Çetin, O., & Temurtaş, F. (2019). Classification of Magnetoencephalography Signals Regarding Visual Stimuli by Generalized Regression Neural Network. Dicle Tıp Dergisi, 45(3), 19–25. https://doi.org/10.5798/dicletip.534819.
12. 12. Gorur, K., Bozkurt, M., Bascil, M., & Temurtas, F. (2019). GKP Signal Processing Using Deep CNN and SVM for Tongue-Machine Interface. Traitement Du Signal, 36(4), 319–329. https://doi.org/10.18280/ts.360404
13. 13. Yeom, H. G., Kim, J. S., & Chung, C. K. (2020). LSTM Improves Accuracy of Reaching Trajectory Prediction From Magnetoencephalography Signals. IEEE Access, 8, 20146–20150. https://doi.org/10.1109/ACCESS.2020.2969720
14. 14. Alom, M. Z., Moody, A. T., Maruyama, N., Van Essen, B. C., & Taha, T. M. (2018). Effective Quantization Approaches for Recurrent Neural Networks. In Proceedings of the International Joint Conference on Neural Networks (Vol. 2018-July). https://doi.org/10.1109/IJCNN.2018.8489341
15. 15. Alpaydın, E. (2010). Introduction to Machine Learning. MIT Press, Cambridge, Massachusetts.
16. 16. Ghojogh, B., & Crowley, M. (2019). Linear and Quadratic Discriminant Analysis: Tutorial, (4), 1–16. Retrieved from http://arxiv.org/abs/1906.02590
17. 17. Barachant, A. (2014). MEG decoding using Riemannian Geometry and Unsupervised classification. Notes on the winner of the Kaggle “DecMeg2014 - Decoding the Human Brain” competition., 1–8.
18. 18. Henson, R. N., Wakeman, D. G., Litvak, V., & Friston, K. J. (2011). A Parametric Empirical Bayesian Framework for the EEG/MEG Inverse Problem: Generative Models for Multi-Subject and Multi-Modal Integration. Frontiers in Human Neuroscience, 5(August), 1–16. https://doi.org/10.3389/fnhum.2011.00076
19. 19. “DecMeg2014-Decoding the Human Brain”. [Online]. Available: https://www.kaggle.com/c/decoding- the-human-brain. (n.d.), 2014.
20. 20. Olivetti, E., Kia, S. M., & Avesani, P. (2014). MEG decoding across subjects. Proceedings - 2014 International Workshop on Pattern Recognition in Neuroimaging, PRNI 2014. https://doi.org/10.1109/PRNI.2014.6858538
21. 21. Roy, P. P., Kumar, P., & Chang, V. (2020). A hybrid classifier combination for home automation using EEG signals. Neural Computing and Applications, 32(14), 1–19. https://doi.org/10.1007/s00521-020-04804-y
22. 22. Yger, F., Berar, M., & Lotte, F. (2017). Riemannian Approaches in Brain-Computer Interfaces: A Review. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(10), 1753–1762. https://doi.org/10.1109/TNSRE.2016.2627016
23. 23. Barachant, A., Bonnet, S., Congedo, M., & Jutten, C. (2012). Multiclass brain-computer interface classification by Riemannian geometry. IEEE Transactions on Biomedical Engineering, 59(4), 920–928. https://doi.org/10.1109/TBME.2011.2172210
24. 24. Uçar, M. K. (2020). Classification Performance-Based Feature Selection Algorithm for Machine Learning: P-Score. IRBM, 41(4), 229–239. https://doi.org/10.1016/j.irbm.2020.01.006
25. 25. Bascil, M. S., Tesneli, A. Y., & Temurtas, F. (2015). Multi-channel EEG signal feature extraction and pattern recognition on horizontal mental imagination task of 1-D cursor movement for brain computer interface. Australasian Physical and Engineering Sciences in Medicine, 38(2), 229–239. https://doi.org/10.1007/s13246-015-0345-6
26. 26. Hossin, M., & Sulaiman, M. . (2015). A Review on Evaluation Metrics for Data Classification Evaluations. International Journal of Data Mining & Knowledge Management Process, 5(2), 01–11. https://doi.org/10.5121/ijdkp.2015.5201
27. 27. Toğaçar, M., Ergen, B., Cömert, Z., & Özyurt, F. (2020). A Deep Feature Learning Model for Pneumonia Detection Applying a Combination of mRMR Feature Selection and Machine Learning Models. IRBM, 41(4), 212–222. https://doi.org/10.1016/j.irbm.2019.10.006
28. 28. Yang, S., & Berdine, G. (2017). The receiver operating characteristic (ROC) curve. The Southwest Respiratory and Critical Care Chronicles, 5(19), 34. https://doi.org/10.12746/swrccc.v5i19.391
29. 29. Muller, K.-R., Mika, S., Ratsch, G., Tsuda, K., & Scholkopf, B. (2001). An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 12(2), 181–201. https://doi.org/10.1109/72.914517
30. 30. Koudjonou, K. M., & Rout, M. (2020). A stateless deep learning framework to predict net asset value. Neural Computing and Applications, 32(14), 1–19. https://doi.org/10.1007/s00521-019-04525-x
31. 31. Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791
32. 32. Ozer, I., Ozer, Z., & Findik, O. (2018). Noise robust sound event classification with convolutional neural network. Neurocomputing, 272, 505–512. https://doi.org/10.1016/j.neucom.2017.07.021
33. 33. Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology, 195(1), 215–243. https://doi.org/10.1113/jphysiol.1968.sp008455
34. 34. Ma, J., Wu, F., Zhu, J., Xu, D., & Kong, D. (2017). A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics, 73, 221–230. https://doi.org/10.1016/j.ultras.2016.09.011
35. 35. Golmohammadi, M., Ziyabari, S., Shah, V., Von Weltin, E., Campbell, C., Obeid, I., & Picone, J. (2017). Gated recurrent networks for seizure detection. In 2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB) (Vol. 2018-Janua, pp. 1–5). IEEE. https://doi.org/10.1109/SPMB.2017.8257020
36. 36. Hadjikhani, N., Kveraga, K., Naik, P., & Ahlfors, S. P. (2009). Early (M170) activation of face-specific cortex by face-like objects. NeuroReport, 20(4), 403–407. https://doi.org/10.1097/WNR.0b013e328325a8e1
37. 37. Wakeman, D. G., & Henson, R. N. (2015). A multi-subject, multi-modal human neuroimaging dataset. Scientific Data, 2(1), 150001. https://doi.org/10.1038/sdata.2015.1
38. 38. Dash, D., Ferrari, P., & Wang, J. (2020). Decoding Imagined and Spoken Phrases From Non-invasive Neural (MEG) Signals. Frontiers in Neuroscience, 14(September 2004), 8–20. https://doi.org/10.3389/fnins.2020.00290
39. 39. Gupta, S., & Gandhi, T. (2020). Identification of Neural Correlates of Face Recognition Using Machine Learning Approach. In Computer Vision and Machine Intelligence in Medical Image Analysis (pp. 13–20). https://doi.org/10.1007/978-981-13-8798-2_2
40. 40. Li, J., Pan, J., Wang, F., & Yu, Z. (2021). Inter-Subject MEG Decoding for Visual Information with Hybrid Gated Recurrent Network. Applied Sciences, 11(3), 1215. https://doi.org/10.3390/app11031215
41. 41. Liu, C., Kang, Y., Zhang, L., & Zhang, J. (2021). Rapidly Decoding Image Categories From MEG Data Using a Multivariate Short-Time FC Pattern Analysis Approach. IEEE Journal of Biomedical and Health Informatics, 25(4), 1139–1150. https://doi.org/10.1109/JBHI.2020.3008731
42. 42. Huang, H., Hu, L., Xiao, F., Du, A., Ye, N., & He, F. (2019). An EEG-Based Identity Authentication System with Audiovisual Paradigm in IoT. Sensors, 19(7), 1664. https://doi.org/10.3390/s19071664
43. 43. Sooriyaarachchi, J., Seneviratne, S., Thilakarathna, K., & Zomaya, A. Y. (2021). MusicID: A Brainwave-Based User Authentication System for Internet of Things. IEEE Internet of Things Journal, 8(10), 8304–8313. https://doi.org/10.1109/JIOT.2020.3044726