IDEAS home Printed from https://ideas.repec.org/a/eee/phsmap/v603y2022ics0378437122004642.html
   My bibliography  Save this article

A Transformer based neural network for emotion recognition and visualizations of crucial EEG channels

Author

Listed:
  • Guo, Jia-Yi
  • Cai, Qing
  • An, Jian-Peng
  • Chen, Pei-Yin
  • Ma, Chao
  • Wan, Jun-He
  • Gao, Zhong-Ke

Abstract

With the rapid development of artificial intelligence and sensor technology, electroencephalogram-based (EEG) emotion recognition has attracted extensive attention. Various deep neural networks have been applied to it and achieved excellent results in classification accuracy. Except for classification accuracy, the interpretability of the feature extraction process is also considerable for model design for emotion recognition. In this study, we propose a novel neural network model (DCoT) with depthwise convolution and Transformer encoders for EEG-based emotion recognition by exploring the dependence of emotion recognition on each EEG channel and visualizing the captured features. Then we conduct subject-dependent and subject-independent experiments on a benchmark dataset, SEED, which contains EEG data of positive, neutral, and negative emotions. For subject-dependent experiments, the average accuracy of three classification tasks is 93.83%. For subject-independent experiments, the average accuracy of three classification tasks is 83.03%. Additionally, we assess the importance of each EEG channel in emotional activities by the DCoT model and visualize it as brain maps. Furthermore, satisfactory results are obtained by utilizing eight selected crucial EEG channels: FT7, T7, TP7, P3, FC6, FT8, T8, and F8, both in two classification tasks and three classification tasks. Using a small number of EEG channels for emotion recognition can reduce equipment costs and computing costs, which is suitable for practical applications.

Suggested Citation

  • Guo, Jia-Yi & Cai, Qing & An, Jian-Peng & Chen, Pei-Yin & Ma, Chao & Wan, Jun-He & Gao, Zhong-Ke, 2022. "A Transformer based neural network for emotion recognition and visualizations of crucial EEG channels," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 603(C).
    • Handle: RePEc:eee:phsmap:v:603:y:2022:i:c:s0378437122004642
    DOI: 10.1016/j.physa.2022.127700
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0378437122004642
    Download Restriction: Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000
    File URL: https://libkey.io/10.1016/j.physa.2022.127700?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ahmadi, Negar & Pei, Yulong & Pechenizkiy, Mykola, 2019. "Effect of linear mixing in EEG on synchronization and complex network measures studied using the Kuramoto model," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 520(C), pages 289-308.
    2. Rosso, O.A & Martin, M.T & Plastino, A, 2002. "Brain electrical activity analysis using wavelet-based informational tools," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 313(3), pages 587-608.
    3. Bozhokin, S.V. & Suslova, I.B., 2015. "Wavelet-based analysis of spectral rearrangements of EEG patterns and of non-stationary correlations," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 421(C), pages 151-160.
    4. Tuncer, Turker & Dogan, Sengul & Subasi, Abdulhamit, 2021. "A new fractal pattern feature generation function based emotion recognition method using EEG," Chaos, Solitons & Fractals, Elsevier, vol. 144(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cui, Qian & Li, Lulu & Cao, Jinde & Alsaadi, Fawaz E., 2022. "Synchronization of Kuramoto-oscillator networks under event-triggered delayed impulsive control," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 608(P1).
    2. Papapetrou, M. & Kugiumtzis, D., 2020. "Tsallis conditional mutual information in investigating long range correlation in symbol sequences," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 540(C).
    3. Zunino, L. & Pérez, D.G. & Garavaglia, M. & Rosso, O.A., 2007. "Wavelet entropy of stochastic processes," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 379(2), pages 503-512.
    4. Baravalle, Roman & Rosso, Osvaldo A. & Montani, Fernando, 2018. "Discriminating imagined and non-imagined tasks in the motor cortex area: Entropy-complexity plane with a wavelet decomposition," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 511(C), pages 27-39.
    5. Hongquan Qu & Mengyu Zhang & Liping Pang, 2022. "Mental Workload Classification Method Based on EEG Cross-Session Subspace Alignment," Mathematics, MDPI, vol. 10(11), pages 1-14, May.
    6. Peng, Xiaoyi & Zhao, Yi & Small, Michael, 2020. "Identification and prediction of bifurcation tipping points using complex networks based on quasi-isometric mapping," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 560(C).
    7. Lahmiri, Salim, 2018. "Generalized Hurst exponent estimates differentiate EEG signals of healthy and epileptic patients," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 490(C), pages 378-385.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:603:y:2022:i:c:s0378437122004642. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.