Label Ambiguity and Affective Priming : A Framework for Enhancing Emotion Recognition Models

Affective computing is a multidisciplinary field where computer scientists, psychologists, and other experts collaborate to equip machines with the ability to recognize, interpret, process, and simulate human emotions. While various data modalities have been employed for emotion recognition, this thesis focuses concretely on physiological signals. Physiological responses are inherently less susceptible to deliberate manipulation compared to modalities such as facial expressions, speech, or text, thereby offering a more dependable basis for capturing authentic emotional states. Emotions are by nature ambiguous concepts, and there is still no consensus in the research community on the best way to define or categorize them. This lack of agreement poses a challenge for training machine learning (ML) models, as conventional methods typically assume that each sample is labeled with a single, unambiguous category. Furthermore, this ambiguity becomes more complex when emotional responses are influenced by prior experiences ,a phenomenon known as affective priming. For instance, the response to a happy stimulus may vary significantly depending on whether the subject was previously exposed to a stressful event. The challenges of label ambiguity and affective priming, despite their importance, have received limited attention in the literature. This thesis addresses these challenges as the two foundational pillars underlying its contributions. The aim is to develop an alternative methodology that explicitly accounts for label ambiguity and reduces the impact of affective priming during training ML models. To address these gaps, this thesis proposes a two-part solution: In real-world scenarios, a subject’s response to an emotional stimulus is often dominated by one primary emotion, but other emotional states may also be present to varying degrees. In such cases, the emotional reaction is better represented as a distribution of emotions rather than a single label. To address this, the thesis adopts Label Distribution Learning (LDL) as a framework for modeling label ambiguity during the learning process. Additionally, the sequentiality with which emotional stimuli are presented can be reflected in the collected data. Therefore, to quantify the influence of affective priming in datasets where emotions are induced sequentially, the thesis introduces the Affective Priming Score (APS), a data-driven method for estimating the degree of priming at the individual sample level. Finally, the proposed results for these challenges, LDL and APS, are integrated into a unified methodology that addresses both label ambiguity and affective priming. This approach leverages the APS value of each data point to guide the generation of label distributions, to optimize model training, and to calibrate model outputs during inference. Together, these contributions enable emotion recognition models that are more robust and reliable under real-world challenges, where ambiguity and sequential emotional influences are inherent.