Effect of Avatar Head Movements on Communication Behavior and Subjective Evaluations of Presence and Success in Triadic Conversations

Interactive communication in virtual reality can be used in experimental paradigms to increase the ecological validity of hearing device evaluations. This requires the virtual environment to elicit natural communication behavior in listeners. This study evaluates the effect of virtual animated characters' head movements on participants' communication behavior and experience. Triadic conversations were conducted between a study participant and two confederates. To facilitate the manipulation of head movements, the conversation was conducted in telepresence using a system that transmitted audio, head movement data and video with low delay. The confederates were represented by virtual animated characters (avatars) with different levels of animation: Static heads, automated head movement animations based on speech level onsets, and animated head movements based on the transmitted head movements of the confederates. A condition was also included in which the live videos of the confederates' heads were embedded in the visual scene. Sixteen young adults (19-32 years) with self-reported normal hearing participated in the study, i.e., 16 triads were measured. The results show significant effects of animation level on the participants' speech and head movement behavior as recorded by physical sensors, as well as on the subjective experience. The largest effects were found for the range of head orientation while speaking, the head orientation while listening, and the perceived realism of avatars. We therefore conclude that the representation of conversation partners affects communication behavior, which may be considered when natural speech and movement behavior is desired.