Ventral Striatum Encodes the Value of Cortical Neural States and Dorsal Striatum the Value of Cortical Trajectories During Operant Learning

Abstract

During operant learning, reinforcement strengthens and weakens neural connections to increase the probability of obtaining reward. Critical to this process, the ventral striatum has been suggested to encode the value of states, and the dorsal striatum to encode the value of particular actions given a particular state. However, how states and actions correspond to specific patterns of neural activity in the brain is not well understood. To address this, we investigated the role of the ventral and dorsal striatum in encoding the value of states and trajectories of cortical activity that predict reward. We trained rats to volitionally control an auditory cursor by modulating neurons in motor cortex to obtain reward, while recording activity in the ventral and dorsal striatum. Using reinforcement learning models, we were able to obtain estimates of the value of current cortical states versus the value of cortical trajectories among states, and these value estimates were differentially associated with activity in distinct subregions of the striatum. Our results show that the ventral striatum assesses the value of neural states while the dorsal striatum assesses the value of the trajectories between those states. These findings highlight the role of specific circuits in neural reinforcement.

Keywords: Operant learning, dorsal striatum, ventral striatum, brain-machine interface, reinforcement learning, value functions, credit assignment.