[Defense] A Mixed Emotions Framework and Its Applications in Affective Computing Studies
Monday, March 6, 2023
12:00 pm - 1:30 pm
A Mixed Emotions Framework and Its Applications in Affective Computing Studies
Deciphering human emotions is commanding increasing research attention, because of its potential impact in psychology and human-machine interactions. Estimating subjects’ emotional state through recognition of facial expressions has been a key focus of affective computing and computer vision. Current recognition methods operate on the assumption that only six basic emotions can manifest on the human face. My research aims to change this situation, introducing a new and more realistic framework, where the basic six emotions are complemented by an array of mixed emotions. In this direction, first I selected a Convolutional neural network (CNN) to recognize the basic six emotions on human faces. Second, I validated the CNN performance on RAVDESS – a state of the art benchmark featuring annotated emotional videos of speaking and non-speaking subjects. Third, I developed a meta method based on co-occurrence matrices, which generates mixed emotions as pairs of basic emotions, redistributing the CNN’s probabilistic vector through joint probability considerations. This new method expands the original set of six basic emotions to a set of 21 mixed emotions, approximating more accurately the human condition. Next, I compared the basic facial emotion framework with the mixed facial emotion framework in a knowledge work study, where participants (n=26) were engaged in an hour-long essay writing at their desktop. The analytic results brought to the fore the nuanced insights the mixed emotions method affords. Currently, I am working on comparing the mixed emotions framework with the basic emotions one in another knowledge work study, where participants (n=40) present a talk in front of an online audience. This second study introduces a human-to-human interaction element, complementing the solitary type of the first study, and thus providing a broad comparison basis to my investigation.
12:00 PM - 1:30 PM CT
Online via MS Teams
Dr. Ioannis Pavlidis, proposal advisor
Faculty, students, and the general public are invited.