Relative value learning: computational processes and neural foundations – RELATIVE

A wealth of evidence in behavioral economics and neuroscience suggests that preferences and decisions are dependent on the context. Yet, the specific way in which contextual information is dynamically integrated to influence valuation and decision-making remains debated, both at the computational and neurobiological level. This project combines principles from economics, psychology and neuroscience to determine the computational and neural bases of context-dependent decision-making in three mammalian species (rat, rhesus macaques and human). We hypothesize that contextual effects are pervasive in learning and represent an adaptive response resulting from common neurobehavioral adaptations to similar basic physiological and environmental constraints across species. We predict that, while adaptive in many circumstances, relative value learning can also represent a predictable source of systematic suboptimal decisions; i.e., choices that do not maximize the expected future rewards. Specifically, RELATIVE has two objectives:

Objective 1: Demonstrate a specific form of context-dependency, namely range adaptation, in both primate and rodent reinforcement learning.
This first objective will reveal how rats, macaques and humans adjust to modifications of the magnitude of the available rewards (what is the maximum reward value?). We hypothesize that context dependent choices across species can be accounted for by the same process, namely reinforcement learning with dynamic normalization, and that this process relies on the ventromedial prefrontal cortex (VMPFC).
Objective 2: Characterize the functional form of range adaptation as either a range normalization or a divisive normalization process.
This second objective will demonstrate how the 3 species adjust to modifications of the menu of the available rewards (how many options are presented?). We hypothesize that range adapation involves the same neuro-computational processes in all 3 species, namely range normalization, with corresponding activity in the VMPFC.

We will use original reinforcement learning tasks, inspired by previous human and animal research to manipulate two dimensions of the choice context: the range of reward (magnitude) and the number of options (menu). We will also test three models of how context-dependent (specifically range adapation) learning is achieved. The influence of outcome magnitudes will be captured using a "dynamic normalization” model, where individual rewards are rescaled based on the maximum available reward. The influence of the menu will be characterized by comparing two models - divisive normalization (DN) versus range normalization (RN), which predict extensive (DN) vs limited (RN) changes in reward learning when additional options are added to the task. Finally, we will examine the neurobiological underpinnings of value normalization by monitoring and manipulating the activity of the ventromedial prefrontal cortex (VMPFC), a region thought to be critical for value based-decision-making in all 3 species. Altogether, these three approaches will allow us to capture and compare, across 3 animal species, the computational and neurobiological processes underlying context dependent value learning and its influence on decision making. Our working hypotheses, based on careful consideration of the literature and preliminary results, is that range adaptation is preserved across the three species considered and is implemented as a range normalization process that relies on the ventromedial prefrontal cortex.