Multi-sensory information and simulator immersion: impact on decision-making performance, presence, and cognitive load

Immersive technologies such as 360-degree video virtual reality (360VR) provide unique opportunities to simulate representative environments to investigate decision-making in a safe and cost-effective manner. Inclusion of multi-sensory visual, contextual, and auditory information is important to enhance simulator psychological fidelity. Yet, the influence of increased sensory information on decision-making, presence, and cognitive load in 360VR compared to less immersive two-dimensional video simulators is unknown. This study investigated multi-sensory information, presence, and cognitive load across immersion conditions (360VR and two-dimensional video) in an exemplar sport-specific decision-making task. Fifteen higher-skilled and 15 lesser-skilled Australian Rules Football players completed a decision-making task which presented visual information only, visual and contextual information, and visual, contextual and auditory information. Overall, there were little significant performance difference between simulators. There was a significant decrease in decision-making performance across both skill levels and simulators as contextual and auditory information was added. Decision-making performance decline was more pronounced for lesser-skilled than higher-skilled participants, when they had to utilise contextual information to make riskier decisions to win the game. Significantly more accurate decisions were made when congruent, compared to incongruent, auditory information was presented, particularly in two-dimensional video. Perceptions of presence and cognitive load were significantly higher in 360VR than two-dimensional video, regardless of skill level, whilst across both simulators, cognitive load increased as sensory information was added. These findings indicate provision of multi-sensory information is more important for decision-making than simulator immersion and presence. Therefore, a focus on increasing simulator immersion should be done so with caution.