Stockwell Dönüşümü ve Derin Öğrenme Modelleri Kullanarak Motor Hareket Hayali EEG Sinyallerinin Dönüştürülmesi ve Sınıflandırılması için Yeni Bir Yaklaşım

Yıl 2025, Cilt: 15 Sayı: 1, 152 - 170, 15.03.2025

Çağatay Murat Yılmaz

https://doi.org/10.31466/kfbd.1509850

Öz

EEG sinyalleri kullanılarak motor hareket hayali (MHH) görevlerinin sınıflandırılması, beyin-bilgisayar arayüzleri (BBA) gibi teknolojilerinin gelişiminde önemli rol oynayarak popülerlik kazanmıştır. Bu çalışmada, EEG sinyallerini zaman-frekans uzayında görüntülere kodlamak için Stockwell dönüşümünü kullanan ve görüntüleri önceden eğitilmiş Inception-ResNet-V2, AlexNet ve SqueezeNet evrişimli sinir ağlarına (ESA) vererek sınıflandıran yaklaşımlar önerilmiştir. Denekten-deneğe ve oturumdan-oturuma değişkenliğin fazla olması MHH görevlerinin tanınmasını zorlaştırmaktadır. Literatür çalışmalarının çoğu denek içi performansı incelemiştir. Bu çalışmada ise bir katılımcıyı dışarıda bırak çapraz doğrulama stratejisi kullanılmış, denekler arası MHH varyasyonun etkisi araştırılmış, modellerin performansı ve genelleme yeteneğini değerlendirerek literatüre katkıda bulunulmaya çalışılmıştır. Aynı zamanda farklı oturumlar ve geri besleme olup olmama durumları da değerlendirilmiştir. MHH görevlerini sınıflandırmanın zorluğu ve denekler arası farklılıklar göz önüne alındığında sonuçlar ümit vericidir. İpucu tabanlı gösterim paradigması ve geri bildirimsiz sinyaller için sonuçlar %62,1-%80,8 arasında; gülen yüz geri bildirimi içeren sinyaller için %57,1-%96,3 arasında; geri bildirim içeren ve içermeyen sinyaller için ise %56,8-%91,4 arasındadır. Bu bulgular, MHH görevleri için Stockwell dönüşümü ile ESA’larla birleştirmenin potansiyelini vurgulamakta ve EEG tabanlı BBA uygulamalarında denekler arası değişkenlik hakkında bilgi sunmaktadır.

Anahtar Kelimeler

Stockwell Dönüşümü, Evrişimli Sinir Ağları, Transfer Öğrenme, Motor Hareket Hayali, Beyin Bilgisayar Arayüzleri, Yapay Zeka Yazılımı

A Novel Approach to Motor Imagery EEG Signal Transformation and Classification Using Stockwell Transform and Deep Learning Models

Öz

Motor imagery (MI) classification using EEG signals has gained popularity, playing an essential role in developing technologies such as brain-computer interfaces (BCIs). This paper proposes novel approaches using the Stockwell transform (S-transform) to encode signals into images in time-frequency space and classify them by feeding them to pre-trained Inception-ResNet-V2, AlexNet, and SqueezeNet CNNs. High subject-to-subject and session-to-session signal variability hinder the recognition of MI tasks. Most literature has studied within-subject performance. This study conducted experiments using a leave-one-subject-out cross-validation strategy, investigated inter-subject variation's effect and contributed by evaluating the model's performance and generalization ability. At the same time, different sessions and the presence or absence of feedback were assessed, and the results were analyzed. The results are encouraging, considering the difficulty of classifying MI and inter-subject differences. For a cue-based paradigm and non-feedback signals, the results are between 62.1-80.8%; for signals with smiley feedback, the results are between 57.1-96.3%; and for signals with and without feedback are between 56.8-91.4%. These findings highlight the potential of combining the S-transform with CNNs, offering valuable insights into inter-subject variability in EEG-based BCI applications.

Anahtar Kelimeler

Stockwell Transform, Convolutional Neural Networks, Transfer Learning, Motor Imagery, Brain-Computer Interfaces, Artificial Intelligence Software

Kaynakça

• Alwasiti, H., Yusoff, M. Z., Raza, K. (2020). Motor imagery classification for brain computer interface using deep metric learning. IEEE Access, 8, 109949-109963. doi.org/10.1109/ACCESS.2020.3002459
• Chacon-Murguia, M. I., Rivas-Posada, E. (2020, July). Feature extraction evaluation for two motor imagery recognition based on common spatial patterns, time-frequency transformations and SVM. In 2020 International Joint Conference on Neural Networks (IJCNN) (pp. 1-7). IEEE. https://doi.org/10.1109/IJCNN48605.2020.9206638
• Chang, H., Yang, J. (2018). Genetic-based feature selection for efficient motion imaging of a brain–computer interface framework. Journal of Neural Engineering, 15(5), 056020. https://doi.org/10.1088/1741-2552/aad567
• Das, M. K., Ari, S. (2013). Analysis of ECG signal denoising method based on S-transform. IRBM, 34(6), 362-370. https://doi.org/10.1016/j.irbm.2013.07.012
• Ferdi, A. Y., Ghazli, A. (2024). Authentication with a one-dimensional CNN model using EEG-based brain-computer interface. Computer Methods in Biomechanics and Biomedical Engineering, 1-12. https://doi.org/10.1080/10255842.2024.2355490
• Gibson, P., Lamoureux, M. Margrave, G. (2006) Letter to the Editor: Stockwell and Wavelet Transforms. J Fourier Anal Appl 12, 713–721. https://doi.org/10.1007/s00041-006-6087-9
• Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360. https://arxiv.org/abs/1602.07360
• Kalbkhani, H., Shayesteh, M. G. (2017). Stockwell transform for epileptic seizure detection from EEG signals. Biomedical Signal Processing and Control, 38, 108-118. https://doi.org/10.1016/j.bspc.2017.05.008
• Krizhevsky, A., Sutskever, I., Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. https://doi.org/10.1145/3065386
• Luo, T. -J. (2022) Dual regularized feature extraction and adaptation for cross-subject motor imagery EEG classification. IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 1092-1099) IEEE. https://doi.org/10.1109/BIBM55620.2022.9995282
• Luo, T. J. (2023). Dual selections based knowledge transfer learning for cross-subject motor imagery EEG classification. Frontiers in Neuroscience, 17, 1274320. https://doi.org/10.3389/fnins.2023.1274320
• Ortiz, M., Ferrero, L., Iáñez, E., Azorín, J. M., Contreras-Vidal, J. L. (2020). Sensory integration in human movement: A new brain-machine interface based on gamma band and attention level for controlling a lower-limb exoskeleton. Frontiers in Bioengineering and Biotechnology, 8, 735. https://doi.org/10.3389/fbioe.2020.00735
• Phan, D. T. (2024). Reduce Computational Complexity for Continuous Wavelet Transform in Acoustic Recognition Using Hop Size. arXiv preprint arXiv:2408.14302.
• Reddy, A. K. G., Sharma, R. (2024). Enhancing motor imagery classification: a novel CNN with self-attention using local and global features of filtered EEG data. Connection Science, 36(1). https://doi.org/10.1080/09540091.2024.2426812
• Qian, L., Feng, Z., Hu, H., & Sun, Y. (2020, October). A novel scheme for classification of motor imagery signal using Stockwell transform of CSP and CNN model. IEEE International Conference on Systems, Man, and Cybernetics (pp. 3673-3677). IEEE. https://doi.org/10.1109/SMC42975.2020.9282917
• Salimpour, S., Kalbkhani, H., Seyyedi, S., Solouk, V. (2022). Stockwell transform and semi-supervised feature selection from deep features for classification of BCI signals. Scientific Reports, 12(1), 11773. https://doi.org/10.1038/s41598-022-15813-3
• Stockwell, R. G., Mansinha, L., Lowe, R. P. (1996). Localization of the complex spectrum: the S transform. IEEE Transactions on Signal Processing, 44(4), 998-1001. https:/doi.org/10.1109/78.492555
• Sundar, A. (2024). Time frequency distribution of a signal using S-transform (Stockwell transform). Retrieved from https://www.mathworks.com/matlabcentral/fileexchange/51808-time-frequency-distribution-of-a-signal-using-s-transform-stockwell-transform. MATLAB Central File Exchange.
• Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A. (2017, February). Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 31, No. 1). https://doi.org/10.1609/aaai.v31i1.11231
• URL-1: https://www.bbci.de/competition/iv/#datasets, (Date Accessed: 02 July 2024).
• Talukder, M. A., Islam, M. M., Uddin, M. A., Akhter, A., Pramanik, M. A. J., Aryal, S., ... & Moni, M. A. (2023). An efficient deep learning model to categorize brain tumor using reconstruction and fine-tuning. Expert systems with applications, 230, 120534. https://doi.org/10.1016/j.eswa.2023.120534
• Tangermann, M., Müller, K. R., Aertsen, A., Birbaumer, N., Braun, C., Brunner, C., ... & Blankertz, B. (2012). Review of the BCI competition IV. Frontiers in neuroscience, 6, 55. https://doi.org/10.3389/fnins.2012.00055
• Wang, W., Li, B., Wang, H., Wang, X., Qin, Y., Shi, X., Liu, S. (2024) EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification. Med Biol Eng Comput 62, 107–120. https://doi.org/10.1007/s11517-023-02931-x
• Yilmaz, C. M., (2021). Classification of EEG-based motor imagery tasks using 2-D features and quasi-probabilistic distribution models, Ph.D. Thesis, Karadeniz Technical University, Graduate Institute of Natural and Applied Sciences, Türkiye, 2021.
• Yilmaz, C. M., Hatipoglu Yilmaz, B. (2023). Advancements in image feature-based classification of motor imagery EEG data: A comprehensive review. Traitement du Signal, 40(5). https://doi.org/10.18280/ts.400507
• Zhu, X., Li, P., Li, C., Yao, D., Zhang, R., Xu, P. (2019). Separated channel convolutional neural network to realize the training free motor imagery BCI systems. Biomedical Signal Processing and Control, 49, 396-403. https://doi.org/10.1016/j.bspc.2018.12.027
• Zanini, P., Congedo, M., Jutten, C., Said, S., Berthoumieu, Y. (2018) Transfer Learning: A Riemannian Geometry Framework with Applications to Brain–Computer Interfaces. IEEE Transactions on Biomedical Engineering, 65 (5), 1107-1116. https://doi.org/10.1109/TBME.2017.2742541
• Zidelmal, Z., Amirou, A., Ould-Abdeslam, D., Moukadem, A., Dieterlen, A. (2014). QRS detection using S-Transform and Shannon energy. Computer methods and programs in biomedicine, 116(1), 1-9. https://doi.org/10.1016/j.cmpb.2014.04.008