Designing Augmented Virtuality: Impact of Audio and Video Features on User Experience in a Virtual Opera Performance

Emerging technologies offer cultural institutions opportunities to expand their audiences and make their content more accessible. Augmented Virtuality (AV), which integrates real-world content into virtual environments, shows particular potential. It enables the transmission of live or pre-recorded stage performances, such as concerts and theater productions, from the stage to virtual audiences via increasingly affordable head-mounted displays (HMDs). However, as AV remains under-researched, little is known about which recording features enable an immersive user experience while maintaining cost-efficiency—an essential requirement for resource-constrained cultural institutions. In this context, we investigate the influence of features of audio and video recordings on key dimensions of user experience in the use case of a real opera performance integrated into a virtual opera house. We conducted a 2 × 2 within-subjects study with 30 participants to measure the effects of 2D versus 3D videos and stereophonic versus spatial audio rendering on several key dimensions of user experience. The results show that spatial audio has a positive impact on Place Illusion, whereas video dimensionality had no significant effect. Recommendations for the design of AV applications are derived from study results, aiming at balancing immersive user experience and cost-efficiency for virtual cultural participation.