The effects of vocal emotions and emotional context on the neural tracking of speech envelopes and listeners’ vigilance states

Introduction High-arousal emotional speech, such as angry and happy speech, is characterized by changes in signal amplitude that can substantially alter the temporal structure of the speech signal. In this EEG study, we investigated how these acoustic changes, and the structure of the preceding emotional context, influence neural tracking of temporal speech patterns, as well as alpha-band desynchronization associated with vigilance states in listeners. Methods EEGs were recorded from 30 adult native speakers of American English while they listened to angry, happy, or neutral spoken sentences presented either in a randomized order or blocked by emotion. To ensure sustained attention, participants answered occasional questions about sentence content. Results Angry speech elicited stronger alpha desynchronization than neutral and happy speech when stimuli were blocked by emotion but not when stimuli were fully randomized. In contrast, neural tracking of amplitude modulation patterns was more robust for neutral speech compared to happy or angry speech across both stimulus presentation contexts. When neural tracking was controlled for stimulus differences in amplitude variability, angry speech, which conveyed greater amplitude variability, was more faithfully tracked than both happy and neutral speech across stimulus presentation contexts. Discussion Our findings suggest that tonic modulations of alpha power are more sensitive to prolonged emotional context than to transient changes in speaker emotion. Furthermore, we found that emotional speech robustly modulates listeners’ vigilance, particularly following prolonged exposure to a single emotion, while exerting a limited influence on the neural encoding of amplitude modulation, which is primarily dominated by bottom-up amplitude variability in the acoustic signal.