How many decisions did you have to make today? From the choice of breakfast, to the vigor of brushing our teeth, we can effortlessly make moment to moment decisions on innumerable aspects of daily life. Scientists have started to explore how the brain accomplishes such a feat, but have largely focused on one facet of decision making, which is choosing among discrete actions, or ‘action selection’. However, decision making not only includes choosing what to do, but also how, or the vigor behind our actions. We thus initiated the study of this more mysterious aspect of decision making, by first carefully watched how animal’s choose vigor, and then trying to find the underlying brain mechanisms.
We put rats into a behavioral box with three ports, a central odor port and two reward ports at either side. If a rat poked his nose into a central odor port, an odor would be released, whose identity told him which port will deliver water. Since our rats were thirsty, they frequently poked the odor port (each poke started a ‘trial’.. However, the rewards were not always the same. In any given block of trials (consecutive streak of trials), the reward ports may deliver any combination of three possible reward amounts: small, big, or medium. Thus, some blocks yielded higher average rewards than others.
How would the rats’ motivation be affected by these simple reward manipulations? One possibility is that rats may be most motivated immediately after receiving big rewards, discouraged after small rewards, and somewhere in between after medium rewards. This is like a student who becomes excited about academics right after getting an A on a test, less excited after a B, and discouraged after getting a C. Another possibility is that rats’ motivation might depend on the average, long-term value of the block. This is analogous to a student’s motivation that depends on the GPA of the semester (block) rather than on his/her latest test score.
Interestingly, the latter possibility proved to be true: in blocks that yielded high average reward (high GPA semester), the animal became significantly more motivated to perform. In such blocks, rats initiated new trials with alacrity regardless of previously receiving a big or small reward. Accordingly, in blocks with low average reward (low GPA semester), animals were significantly slower to perform. Thus, rats take into account the average value of their two options to choose their general motivation. This strategy is in fact a good one for this context: it allows rats to exploit their task when they expect high returns, and lets them relax on the job when the average return is too small.
To show that the associative striatum, a region thought to be important for decision making was important for this strategy, we lesioned this brain area in some of the rats. Upon lesioning, rats became unable to regulate their motivation based on the average value. Instead, their motivation to perform depended primarily on the reward size of the immediately preceding trial. This is like a student that becomes immediately discouraged after getting a B, forgetting that his/her GPA for the semester is nevertheless still quite good.
Moreover, we inserted electrodes into the intact associative striatum to record the electrical signals from individual neurons. Interestingly, a large proportion of neurons changed their firing level depending on the average value of the animal’s expected rewards.
By examining decision strategies for the selection of motivational level and identifying brain regions that are involved, we hope that we have opened the doors for better understanding the function and dysfunction of the brain in making decisions.
Read more in Nature Neuroscience (AOP) or download the PDF
Read more in Harvardgazette
Read more in The Harvard Crimson
