A gradient of complementary learning systems emerges through meta-learning

Abstract Long-term learning and memory in the primate brain rely on a series of hierarchically organized subsystems extending from early sensory neocortical areas to the hippocampus. The components differ in their representational attributes and plasticity, with evidence for sparser, more decorrelated activity and faster plasticity in regions higher up in the hierarchy. How and why did the brain arrive at this organization? We explore the principles that allow such an organization to emerge by simulating a hierarchy of learning subsystems in artificial neural networks (ANNs) using a meta-learning approach. As ANNs optimized weights for a series of tasks, they concurrently meta-learned layer-wise plasticity and sparsity parameters. This approach enhanced the computational efficiency of ANNs, promoting hidden activation sparsity while benefitting task performance. Meta-learning also gave rise to a brain-like hierarchical organization, with higher layers displaying faster plasticity and a sparser, more pattern-separated neural code than lower layers. Early layers peaked early in their plasticity and stabilized, whereas higher layers continued to develop and maintained elevated plasticity over time, mirroring empirical developmental trajectories. Moreover, when trained on dual tasks imposing competing demands for item discrimination and categorization, ANNs with parallel pathways developed distinct representational and plasticity profiles, convergent with the distinct properties observed empirically across intra-hippocampal pathways. These results suggest that the macroscale organization and development of heterogeneous learning subsystems in the brain may emerge in part from optimizing biological variables that govern plasticity and sparsity.