Domain Adaptation for Fear of Heights Classification in a VR Environment Based on EEG and ECG

Three levels of fear of heights were detected in subjects with different severities of acrophobia, based on the electroencephalographic (EEG) and electrocardiographic (ECG) signals. The study aims to demonstrate the feasibility of a data-fusion-based method for real-time assessment of the fear of heights intensity to integrate into adaptive Virtual Reality Exposure Therapy for acrophobia. The generalization performance of classification tasks on fear states is improved by exploiting both trait-based clustering and Domain Adaptation methods. Participants were gradually exposed to increasing height levels through a Virtual Reality (VR) scenario representing a canyon. The initial severity of fear of heights, the level of distress at each height, and the anxiety level before and after the exposure were assessed through the Acrophobia Questionnaire, the Subjective Unit of Distress, and the State and Trait Anxiety Inventory, respectively. The Simulator Sickness Questionnaire was administered to exclude possible motion sickness interference in the experiment. The EEG and ECG signals were acquired through a 32-channel headset and 1 Lead ECG derivation during the exposure to the eliciting VR scenario. Four classifiers (i.e. Support Vector Machines, Deep Neural Networks, Random Forests, and k-Nearest Neighbors) were adopted in the experimental environment. Preliminary tests were performed in a within-subject experiment, achieving the best classification accuracy of $$87.1 \% \pm 7.8 \%$$ 87.1 % ± 7.8 % with a Deep Neural Network. As the cross-subject approach is concerned, three strategies, namely Domain Adaptation (DA), data fusion (combining EEG with ECG), and participant clustering (based on the acrophobia severity), were evaluated. DA resulted in the most effective strategies by determining an improvement of more than 20 % in classification accuracy. Random Forest performed the best classification accuracy for the severe acrophobia cluster with a mean of $$63.6 \%$$ 63.6 % and a standard deviation of $$ 13.4 \%$$ 13.4 % over three classes by exploiting Stratified Normalization.