Mind Spy: Unobtrusive Mental Workload State Detection Exploiting Heart Rate- and Posture Movement- Features using Machine Learning

Abstract

This paper explores mental workload classification using unobtrusive psychophysiological measures. Posture movements on a chair, a wrist-worn heart rate sensor, color- and infrared-spectrum remote photoplethysmography were recorded from sixteen expert train traffic operators in a railway human-in-the-loop simulator with low-, medium-, and high- mental workload conditions. Normalized heart rate- and posture movement- features were extracted and used as input for nearest-neighbor and ensemble machine learning classifiers. The classifiers were trained using a cross-validated, leave-one-out, and between-subject design. Results show that the classifiers can distinguish low-, and high- mental workload states of the expert operators above chance. Posture movements and heart rate variability measures from the infrared spectrum and yielded the highest performance and, combined with the properties of no physical contact to the subject in case of the gyroscope, and the invisibility of infrared light to the human eye, these measures make for the least obtrusive mental workload classification sensor-setup tested in this paper.