ARCHIVES

In high-stake situations like commercial aviation, the real time analysis of the mental state is a significant issue of biomedical engineering and human-computer interaction. In this paper, we propose an intensive, deep learning-based model of the detection of critical and key events of channelized attention, diverted attention, and startle/surprise using a high-dimensional and multi-modal bio signal data set. The data is a set of 20 Electroencephalography (EEG) channels in addition to Electrocardiography (ECG) channels, as well as Galvanic Skin Response (GSR) channels, and Respiration (R) channels. Trying to eliminate the reduced capabilities of manual feature engineering, latency of standard classifiers, we employ a hybrid architecture, which combines Convolutional Neural Networks (CNN) as spatial feature extractors with Long Short-Term Memory (LSTM) networks as temporal dependency models. We clarify the mathematical foundations to this topology, referred to as CNN-first, which can be shown to be effective at isolating non-linear, complex correlations across channels and at the same time provides long-range temporal evolution of physiological stress markers. This has been empirically validated to imply that this spatiotemporal modeling methodology is far more efficient than the baseline architectures in robustness and generalization.