Is AI an Algorithm by Any Other Name? Behavioral Reactions to AI ‐ and Model‐Based Demand Planning Algorithms

ABSTRACT With the ongoing deployment of AI algorithms, managers do not know whether existing demand planning processes account for possible differences in human behavior when using AI‐based systems in comparison to legacy model‐based systems. This study examines how human behavior may differ when performing demand forecasting tasks due to the disclosure of algorithm type (AI or model) along with associated algorithm performance (low and improving). Using signaling theory, we hypothesize that algorithm type and performance influence user forecast adjustment behavior. We find support for these predictions across two laboratory experiments and a large quasi‐natural field experiment with approximately 575,000 observations from a multinational retailer. We find no significant direct effect of algorithm type independent of performance in the lab. In contrast, in the field, users implement significantly greater adjustments for AI‐based algorithms compared to model‐based algorithms. Across both contexts, algorithm performance, whether low or improving, has a significant direct effect on user adjustments, with users adapting their behavior to the algorithm's performance. Finally, we find that in the lab and the field, users' responses to low performance are amplified when the forecasts originate from AI‐based algorithms. Our findings underscore the nuance and complexity in which users interact with AI‐based algorithms compared to model‐based algorithms and demonstrate the value of signaling theory for understanding human–AI collaboration.