EEG-induced Fear-type Emotion Classification Through Wavelet Packet Decomposition, Wavelet Entropy, and SVM

Year 2022, , 241 - 251, 31.12.2022

Çağlar Uyulan , Ahmet Ergun Gümüş , Zozan Güleken

https://doi.org/10.17350/HJSE19030000277

Abstract

Among the most significant characteristics of human beings is their ability to feel emotions. In recent years, human-machine interface (HM) research has centered on ways to empower the classification of emotions. Mainly, human-computer interaction (HCI) research concentrates on methods that enable computers to reveal the emotional states of humans. In this research, an emotion detection system based on visual IAPPS pictures through EMOTIV EPOC EEG signals was proposed. We employed EEG signals acquired from channels (AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4) for individuals in a visual induced setting (IAPS fear and neutral aroused pictures). The wavelet packet transform (WPT) combined with the wavelet entropy algorithm was applied to the EEG signals. The entropy values were extracted for every two classes. Finally, these feature matrices were fed into the SVM (Support Vector Machine) type classifier to generate the classification model. Also, we evaluated the proposed algorithm as area under the ROC (Receiver Operating Characteristic) curve, or simply AUC (Area under the curve) was utilized as an alternative single-number measure. Overall classification accuracy was obtained at 91.0%. For classification, the AUC value given for SVM was 0.97. The calculations confirmed that the proposed approaches are successful for the detection of the emotion of fear stimuli via EMOTIV EPOC EEG signals and that the accuracy of the classification is acceptable.

Keywords

EMOTIV EPOC EEG, Fear emotion, Wavelet Entropy, SVM, ROC

Supporting Institution

-

Project Number

-

Thanks

-

References

• [1] E. Osuna, L. F. Rodríguez, J. O. Gutierrez-Garcia, and L. A. Castro, “Development of computational models of emotions: A software engineering perspective,” Cogn. Syst. Res., vol. 60, 2020, doi: 10.1016/j.cogsys.2019.11.001.
• [2] A. Hassouneh, A. M. Mutawa, and M. Murugappan, “Development of a Real-Time Emotion Recognition System Using Facial Expressions and EEG based on machine learning and deep neural network methods,” Informatics Med. Unlocked, vol. 20, p. 100372, 2020, doi: 10.1016/j.imu.2020.100372.
• [3] F. Balducci, C. Grana, and R. Cucchiara, “Affective level design for a role-playing videogame evaluated by a brain–computer interface and machine learning methods,” Vis. Comput., vol. 33, no. 4, 2017, doi: 10.1007/s00371-016-1320-2.
• [4] N. S. Suhaimi, J. Mountstephens, and J. Teo, “EEG-Based Emotion Recognition: A State-of-the-Art Review of Current Trends and Opportunities,” Computational Intelligence and Neuroscience, vol. 2020. 2020, doi: 10.1155/2020/8875426.
• [5] A. Mert and A. Akan, “Emotion recognition from EEG signals by using multivariate empirical mode decomposition,” Pattern Anal. Appl., vol. 21, no. 1, 2018, doi: 10.1007/s10044-016-0567-6.
• [6] M. M. Bradley and P. J. Lang, “Measuring emotion: The self-assessment manikin and the semantic differential,” J. Behav. Ther. Exp. Psychiatry, vol. 25, no. 1, 1994, doi: 10.1016/0005-7916(94)90063-9.
• [7] J. D. Morris, “OBSERVATIONS: SAM: The Self-Assessment Manikin - An Efficient Cross-Cultural Measurement of Emotional Response,” J. Advert. Res., vol. 35, no. 6, pp. 63–68, 1995.
• [8] C. E. Izard, “Emotion theory and research: Highlights, unanswered questions, and emerging issues,” Annu. Rev. Psychol., vol. 60, pp. 1–25, 2009, doi: 10.1146/annurev.psych.60.110707.163539.
• [9] G. Di Flumeri, P. Aricò, G. Borghini, N. Sciaraffa, A. Di Florio, and F. Babiloni, “The dry revolution: Evaluation of three different eeg dry electrode types in terms of signal spectral features, mental states classification and usability,” Sensors (Switzerland), vol. 19, no. 6, 2019, doi: 10.3390/s19061365.
• [10] S. Jeon, J. Chien, C. Song, and J. Hong, “A Preliminary Study on Precision Image Guidance for Electrode Placement in an EEG Study,” Brain Topogr., vol. 31, no. 2, 2018, doi: 10.1007/s10548-017-0610-y.
• [11] J. Fan, J. W. Wade, A. P. Key, Z. E. Warren, and N. Sarkar, “EEG-based affect and workload recognition in a virtual driving environment for ASD intervention,” IEEE Trans. Biomed. Eng., vol. 65, no. 1, 2018, doi: 10.1109/TBME.2017.2693157.
• [12] A. D. Bigirimana, N. Siddique, and D. Coyle, “A hybrid ICA-wavelet transform for automated artefact removal in EEG-based emotion recognition,” 2017, doi: 10.1109/SMC.2016.7844928.
• [13] A. Tandle, N. Jog, P. D’cunha, and M. Chheta, “Classification of Artefacts in EEG Signal Recordings and EOG Artefact Removal using EOG Subtraction,” Commun. Appl. Electron., vol. 4, no. 1, 2016, doi: 10.5120/cae2016651997.
• [14] M. Murugappan and S. Murugappan, “Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT),” 2013, doi: 10.1109/CSPA.2013.6530058.
• [15] M. Horvat, M. Dobrinic, M. Novosel, and P. Jercic, “Assessing emotional responses induced in virtual reality using a consumer EEG headset: A preliminary report,” 2018, doi: 10.23919/MIPRO.2018.8400184.
• [16] Y. Liu et al., “Multi-channel EEG-based emotion recognition via a multi-level features guided capsule network,” Comput. Biol. Med., vol. 123, 2020, doi: 10.1016/j.compbiomed.2020.103927.
• [17] K. Schaaff and T. Schultz, “Towards emotion recognition from electroencephalographic signals,” 2009, doi: 10.1109/ACII.2009.5349316.
• [18] T. T. Erguzel et al., “Entropy: A Promising EEG Biomarker Dichotomizing Subjects With Opioid Use Disorder and Healthy Controls,” Clin. EEG Neurosci., vol. 51, no. 6, 2020, doi: 10.1177/1550059420905724.
• [19] M. A. Asghar, M. J. Khan, M. Rizwan, M. Shorfuzzaman, and R. M. Mehmood, “AI inspired EEG-based spatial feature selection method using multivariate empirical mode decomposition for emotion classification,” in Multimedia Systems, 2022, vol. 28, no. 4, doi: 10.1007/s00530-021-00782-w.
• [20] M. Z. I. Ahmed, N. Sinha, S. Phadikar, and E. Ghaderpour, “Automated Feature Extraction on AsMap for Emotion Classification Using EEG,” Sensors, vol. 22, no. 6, 2022, doi: 10.3390/s22062346.
• [21] X. Zhu et al., “EEG Emotion Classification Network Based on Attention Fusion of Multi-Channel Band Features,” Sensors, vol. 22, no. 14, 2022, doi: 10.3390/s22145252.
• [22] A. Anuragi, D. Singh Sisodia, and R. Bilas Pachori, “EEG-based cross-subject emotion recognition using Fourier-Bessel series expansion based empirical wavelet transform and NCA feature selection method,” Inf. Sci. (Ny)., vol. 610, pp. 508–524, 2022, doi: 10.1016/j.ins.2022.07.121.
• [23] C. C. N. Network, J. Dai, X. Xi, G. Li, and T. Wang, “brain sciences EEG-Based Emotion Classification Using Improved,” 2022.
• [24] Q. Gao, Y. Yang, Q. Kang, Z. Tian, and Y. Song, “EEG-based Emotion Recognition with Feature Fusion Networks,” Int. J. Mach. Learn. Cybern., vol. 13, no. 2, 2022, doi: 10.1007/s13042-021-01414-5.
• [25] W. Kan, Y. Li, 阚威, and 李云, “Emotion recognition from EEG signals by using LSTM recurrent neural networks,” J. Nanjing Univ. Nat. Sci., vol. 55, no. 1, pp. 110–116, 2019.
• [26] V. Padhmashree and A. Bhattacharyya, “Human emotion recognition based on time–frequency analysis of multivariate EEG signal,” Knowledge-Based Syst., vol. 238, 2022, doi: 10.1016/j.knosys.2021.107867.
• [27] H. Liu, J. Zhang, Q. Liu, and J. Cao, “Minimum spanning tree based graph neural network for emotion classification using EEG,” Neural Networks, vol. 145, 2022, doi: 10.1016/j.neunet.2021.10.023.
• [28] P. J. Lang, M. M. Bradley, and B. N. Cuthbert, “International affective picture system (IAPS): Technical manual and affective ratings,” NIMH Cent. Study Emot. Atten., pp. 39–58, 1997.
• [29] N. Kumar, K. Khaund, and S. M. Hazarika, “Bispectral Analysis of EEG for Emotion Recognition,” Procedia Comput. Sci., vol. 84, pp. 31–35, 2016, doi: 10.1016/j.procs.2016.04.062.
• [30] R. Yuvaraj et al., “Detection of emotions in Parkinson’s disease using higher order spectral features from brain’s electrical activity,” Biomed. Signal Process. Control, vol. 14, no. 1, pp. 108–116, 2014, doi: 10.1016/j.bspc.2014.07.005.
• [31] N. A. Badcock, P. Mousikou, Y. Mahajan, P. De Lissa, J. Thie, and G. McArthur, “Validation of the Emotiv EPOC® EEG gaming systemfor measuring research quality auditory ERPs,” PeerJ, vol. 2013, no. 1, pp. 1–17, 2013, doi: 10.7717/peerj.38.
• [32] M. Klug and K. Gramann, “Identifying key factors for improving ICA-based decomposition of EEG data in mobile and stationary experiments,” Eur. J. Neurosci., no. May, pp. 1–15, 2020, doi: 10.1111/ejn.14992.
• [33] G. Strang and T. Nguyen, Wavelets and filter banks. Wellesley, MA: Cambridge Press, 1997.
• [34] A. N. Akansu and R. A. Haddad, Multiresolution Signal Decomposition: Transforms, Subbands, and Wavelets. 2001.
• [35] J. Gomes and L. Velho, “The Fast Wavelet Transform,” in From Fourier Analysis to Wavelets, 2015.
• [36] A. Abbate, C. M. DeCusatis, and P. K. Das, “Discrete Wavelet Transform: From Frames to Fast Wavelet Transform,” in Wavelets and Subbands, 2002.
• [37] H. Heidari Bafroui and A. Ohadi, “Application of wavelet energy and Shannon entropy for feature extraction in gearbox fault detection under varying speed conditions,” Neurocomputing, vol. 133, 2014, doi: 10.1016/j.neucom.2013.12.018.
• [38] O. A. Rosso et al., “Wavelet entropy: A new tool for analysis of short duration brain electrical signals,” J. Neurosci. Methods, vol. 105, no. 1, 2001, doi: 10.1016/S0165-0270(00)00356-3.
• [39] F. Salo, A. B. Nassif, and A. Essex, “Dimensionality reduction with IG-PCA and ensemble classifier for network intrusion detection,” Comput. Networks, vol. 148, 2019, doi: 10.1016/j.comnet.2018.11.010.
• [40] C. Uyulan and T. T. Erguzel, “Analysis of Time – Frequency EEG Feature Extraction Methods for Mental Task Classification,” Int. J. Comput. Intell. Syst., vol. 10, no. 1, 2017, doi: 10.2991/ijcis.10.1.87.
• [41] C. Uyulan, T. T. Ergüzel, and N. Tarhan, “Entropy-based feature extraction technique in conjunction with wavelet packet transform for multi-mental task classification,” Biomed. Tech., 2019, doi: 10.1515/bmt-2018-0105.
• [42] R. N. Khushaba, A. Al-Jumaily, and A. Al-Ani, “Novel feature extraction method based on fuzzy entropy and wavelet packet transform for myoelectric control,” 2007, doi: 10.1109/ISCIT.2007.4392044.
• [43] R. N. Khushaba, S. Kodagoda, S. Lal, and G. Dissanayake, “Driver drowsiness classification using fuzzy wavelet-packet-based feature-extraction algorithm,” IEEE Trans. Biomed. Eng., vol. 58, no. 1, 2011, doi: 10.1109/TBME.2010.2077291.
• [44] C. Uyulan and T. Erguzel, “Comparison of Wavelet Families for Mental Task Classification,” J. Neurobehav. Sci., vol. 3, no. 2, 2016, doi: 10.5455/jnbs.1454666348.
• [45] G. I. Webb and Z. Zheng, “Multistrategy ensemble learning: Reducing error by combining ensemble learning techniques,” IEEE Trans. Knowl. Data Eng., vol. 16, no. 8, 2004, doi: 10.1109/TKDE.2004.29.
• [46] M. Majnik and Z. Bosnić, “ROC analysis of classifiers in machine learning: A survey,” Intelligent Data Analysis, vol. 17, no. 3. 2013, doi: 10.3233/IDA-130592.
• [47] D. J. Hand, “Measuring classifier performance: A coherent alternative to the area under the ROC curve,” Mach. Learn., vol. 77, no. 1, 2009, doi: 10.1007/s10994-009-5119-5.
• [48] M. Hamada, B. B. Zaidan, and A. A. Zaidan, “A Systematic Review for Human EEG Brain Signals Based Emotion Classification, Feature Extraction, Brain Condition, Group Comparison,” Journal of Medical Systems, vol. 42, no. 9. 2018, doi: 10.1007/s10916-018-1020-8.
• [49] K. Guo, H. Candra, H. Yu, H. Li, H. T. Nguyen, and S. W. Su, “EEG-based emotion classification using innovative features and combined SVM and HMM classifier,” 2017, doi: 10.1109/EMBC.2017.8036868.
• [50] R. Dhiman, Priyanka, and J. S. Saini, “Wavelet analysis of electrical signals from brain: The electroencephalogram,” in Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, 2013, vol. 115, doi: 10.1007/978-3-642-37949-9_24.
• [51] Z. Mohammadi, J. Frounchi, and M. Amiri, “Wavelet-based emotion recognition system using EEG signal,” Neural Comput. Appl., vol. 28, no. 8, 2017, doi: 10.1007/s00521-015-2149-8.
• [52] A. Q.-X. Ang, Y. Q. Yeong, and W. Wee, “Emotion Classification from EEG Signals Using Time-Frequency-DWT Features and ANN,” J. Comput. Commun., vol. 05, no. 03, 2017, doi: 10.4236/jcc.2017.53009.
• [53] R. E. J. Yohanes, W. Ser, and G. Bin Huang, “Discrete Wavelet Transform coefficients for emotion recognition from EEG signals,” 2012, doi: 10.1109/EMBC.2012.6346410.
• [54] H. Candra, M. Yuwono, R. Chai, H. T. Nguyen, and S. Su, “EEG emotion recognition using reduced channel wavelet entropy and average wavelet coefficient features with normal Mutual Information method,” 2017, doi: 10.1109/EMBC.2017.8036862.
• [55] N. D. Mai, B. G. Lee, and W. Y. Chung, “Affective computing on machine learning-based emotion recognition using a self-made eeg device,” Sensors, vol. 21, no. 15, 2021, doi: 10.3390/s21155135.
• [56] K. Luangrat, Y. Punsawad, and Y. Wongsawat, “On the development of EEG based emotion classification,” 2012, doi: 10.1109/BMEiCon.2012.6465506.
• [57] X. W. Wang, D. Nie, and B. L. Lu, “EEG-based emotion recognition using frequency domain features and support vector machines,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2011, vol. 7062 LNCS, no. PART 1, doi: 10.1007/978-3-642-24955-6_87.
• [58] V. Bajaj and R. B. Pachori, “Human emotion classification from eeg signals using multiwavelet transform,” 2014, doi: 10.1109/ICMB.2014.29.
• [59] A. E. Vijayan, D. Sen, and A. P. Sudheer, “EEG-based emotion recognition using statistical measures and auto-regressive modeling,” 2015, doi: 10.1109/CICT.2015.24.
• [60] S. Alhagry, A. Aly, and R. A., “Emotion Recognition based on EEG using LSTM Recurrent Neural Network,” Int. J. Adv. Comput. Sci. Appl., vol. 8, no. 10, 2017, doi: 10.14569/ijacsa.2017.081046.
• [61] N. Y. Oktavia, A. D. Wibawa, E. S. Pane, and M. H. Purnomo, “Human Emotion Classification Based on EEG Signals Using Naïve Bayes Method,” 2019, doi: 10.1109/ISEMANTIC.2019.8884224.
• [62] J. X. Chen, D. M. Jiang, and Y. N. Zhang, “A Hierarchical Bidirectional GRU Model with Attention for EEG-Based Emotion Classification,” IEEE Access, vol. 7, 2019, doi: 10.1109/ACCESS.2019.2936817.
• [63] Y. P. Lin and T. P. Jung, “Improving EEG-based emotion classification using conditional transfer learning,” front. Hum. Neurosci., vol. 11, 2017, doi: 10.3389/fnhum.2017.00334.
• [64] J. Atkinson and D. Campos, “Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers,” Expert Syst. Appl., vol. 47, 2016, doi: 10.1016/j.eswa.2015.10.049.
• [65] S. E. Moon, S. Jang, and J. S. Lee, “Convolutional Neural Network Approach for EEG-Based Emotion Recognition Using Brain Connectivity and its Spatial Information,” in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2018, vol. 2018-April, doi: 10.1109/ICASSP.2018.8461315.
• [66] H. Mei and X. Xu, “EEG-based emotion classification using convolutional neural network,” in 2017 International Conference on Security, Pattern Analysis, and Cybernetics, SPAC 2017, 2018, vol. 2018-January, doi: 10.1109/SPAC.2017.8304263.
• [67] R. M. Mehmood and H. J. Lee, “Emotion classification of EEG brain signal using SVM and KNN,” 2015, doi: 10.1109/ICMEW.2015.7169786.
• [68] B. Kaur, D. Singh, and P. P. Roy, “EEG Based Emotion Classification Mechanism in BCI,” Procedia Comput. Sci., vol. 132, no. Iccids, pp. 752–758, 2018, doi: 10.1016/j.procs.2018.05.087.
• [69] A. Bhardwaj, A. Gupta, P. Jain, A. Rani, and J. Yadav, “Classification of human emotions from EEG signals using SVM and LDA Classifiers,” 2015, doi: 10.1109/SPIN.2015.7095376.
• [70] T. Chen, S. Ju, F. Ren, M. Fan, and Y. Gu, “EEG emotion recognition model based on the LIBSVM classifier,” Meas. J. Int. Meas. Confed., vol. 164, 2020, doi: 10.1016/j.measurement.2020.108047.