Temporal dynamics of model-based control reveal arbitration between multiple task representations

Predominant frameworks categorize decisions dichotomously (e.g. “goal-directed” vs. “habitual”; “model-based” vs. “model-free”). However, extensive work has shown that many human behaviors exhibit features of both systems, such as those that require foresight (a goal-directed feature) but are not sensitive to environmental perturbations during action execution (a rigidity characteristic of habits). Here, we introduce and explain a new subdivision of goal-directed behaviors, linking features of execution to the format in which the decision-maker has represented environmental contingencies in memory. We exhibit this distinction by employing a novel variant of a standard, two-stage decision task, which allows us to behaviorally capture the within- and across-trial dynamics of planning. We jointly fit choices and response times with a new computational model that revealed how people select among multiple task representations during planning in environments of differing state-space complexity. In particular, we examined how the reliance on task representations changed both as a function of experience, within-subject, and task complexity, across-subjects (total n = 426). We show that both the complexity of the environment and experience with a given contingency structure inform the kinds of representations we use to make decisions: at the early stages of the task, people start with “conjunctive” representations (combining co-occurring first-stage states) in simpler environments, but a “separated” representation (splitting states according to their second-step outcomes) is preferred in more complex environments. With experience, this pattern is reversed. Finally, we show that this shift is governed by a change in approaches to optimizing reward rate: initially, people focus on minimizing uncertainty, and once this has reached asymptote, they transition to prioritizing efficiency. Taken together, we show that people not only arbitrate between different modes of control, but also between types of representations for efficient planning.