University of Houston

In this thesis I will describe a novel way to conduct stress studies via the combination of a physiological and an observational information channel. The method enables not only the quantification of aroused emotional states but also their disambiguation into positive or negative instances. The physiological channel targets sympathetic responses and is materialized as a perspiratory signal extracted from thermal imagery of the perinasal area. The observational channel is materialized via decoding of facial expressions. However, while such decoding is usually performed in the visible spectrum, I have developed an algorithm to carry this out in thermal imagery instead. Thus, thermal imaging is used for both physiological and observational analysis. The potential of this dual unobtrusive methodology was demonstrated with three stress studies. The first study was about surgeons' interaction with laparoscopic training boxes — representative of the dexterous genre. The second study was about operator overloading where the participants played a car driving game while being interrupted by a phone call and text messages. The third study was about guilt and intent detection where the design was to simulate a crime and have interrogation techniques to detect deception.