New parameters of power: On LLM-based manipulation and control and the spectre of strategic AI

Large language models (LLMs) can reproduce a wide variety of rhetorical styles and generate text that expresses a broad spectrum of sentiments. This capacity, now available at low cost, makes them powerful tools for manipulation and control. We draw attention to this phenomenon as an instance of a broader restructuring of computational infrastructures that introduce novel mechanisms for power. In this paper we consider four types of power made possible by the rapid and largely unregulated adoption of LLMs. These include the power to: (a) pollute and uniformize information environments, (b) persuade users via conversational interfaces (e.g., via ‘AI personas’), (c) create novel computational models of human agents (e.g., ‘silicon subjects’) and (d) create novel computational models of human agent populations (e.g., ‘silicon societies’). We draw attention to Meta's ‘Cicero’ model as a proof of concept for how such techniques can be used to produce controllable and steerable strategic dialogue models. We draw these strands together to argue that coordinated use of such techniques make LLM-based systems powerful instruments for the exertion, modulation, and projection of power. We situate these novel expressions of power in relation to ambitions to establish a new generative foundation for computing.