Model-based and model-free valuation signals in the human brain vary markedly in relation to individual differences in behavioral control

Summary Human action selection under reinforcement is believed to rely on two distinct strategies: model-free and model-based reinforcement learning. While behavior in sequential decision-making tasks often reflects a mixture of both, the neural basis of individual differences in their expression remains unclear. Here, we conduct a large-scale fMRI study with 179 participants performing a variant of the two-step task. Using both cluster-defined subgroups and computational parameter estimates, we find that in the ventromedial prefrontal cortex, model-based value signals are strongly linked to the degree of model-based behavioral reliance, whereas model-free signals are ubiquitous across individuals regardless of model-free behavioral influence. Individuals lacking model-based behaviors and model-based neural signals, exhibit impaired state prediction errors (a key signal for learning a model of the environment) in the dorsolateral prefrontal cortex and intraparietal sulcus, suggesting that reduced model-based control may depend in part on underlying difficulties in forming accurate model-based predictions.