Failures in the Loop: Human Leadership in AI-Based Decision-Making

The dark side of AI has been a persistent focus in discussions of popular science and academia (Appendix A), with some claiming that AI is "evil" 1 . Many commentators make compelling arguments for their concerns. Techno-elites have also contributed to the polarization of these discussions, with ultimatums that in this new era of industrialized AI, citizens will need to "join with the AI or risk being left behind" 2 . With such polarizing language, debates about AI adoption run the risk of being oversimplified. Discussion of technological trust frequently takes an all-or-nothing approach. All technologies – cognitive, social, material, or digital – introduce tradeoffs when they are adopted, and contain both 'light and dark' features 3 . But descriptions of these features can take on deceptively (or unintentionally) anthropomorphic tones, especially when stakeholders refer to the features as 'agents' 4 , 5 . When used as an analogical heuristic, this can inform the design of AI, provide knowledge for AI operations, and potentially even predict its outcomes 6 . However, if AI agency is accepted at face value, we run the risk of having unrealistic expectations for the capabilities of these systems.