Dopamine neurons don't just encode valence -- Some encode salience

Two types of dopamine neuron distinctly convey positive and negative motivational signals

Nature advance online publication 17 May 2009.

Masayuki Matsumoto & Okihide Hikosaka

This is the classical kind of (putative) DA neurons, which show opposite responses to reward and punishment, thus encoding the valence of a stimulus.

This classical picture is well and good. But not so fast...

Mastumoto and Hikosaka here show that there is in fact another group of (putative) DA neurons, which have similar bursting responses to both reward and punishment. Thus, it appears that these DA neurons are encoding the salience of a stimulus, reminiscent of how basal forebrain neurons encode salience.

Hippocampal theta oscillations are travelling waves

Evgueniy V. Lubenov & Athanassios G. Siapas

Nature 459, 534-539 (28 May 2009)

A movie is worth a thousand words. This is cool!


Define anatomic boundary by large scale expression pattern analysis

Genomic Anatomy of the Hippocampus
Thompson et al, Volume 60, Issue 6, 26 December 2008, Pages 1010-1021 -- Neuron

What features really define the boundaries of neural systems? Anatomical landmarks and morphology have long been the guide, but these criteria come with obvious caveats that, at the transition zone where one region morphs into another, the determination of the exact boundary may be somewhat arbitrary. Is there an abrupt transition? or graded transition? and are there hidden subdivisions?

The authors here provides novel insight on this issue using genome scale gene expression data. Thompson et al applied powerful statistical pattern recognition tools to determine functional domains within the hippocampus based on the spatial expression pattern of ~3000 genes. This analysis not only recovers the major divisions of the hippocampus - DG, CA3, CA1 - but also uncovers subdivisions within. I really like this data-driven approach, which relies on the same tools that are used to uncover patterns in neurophysiological data.

Many molecules that define the map are adhesion molecules, which are important for forming neuronal circuits and finding projection targets. Supporting this idea, the subdivisions described in this paper (lower row) correspond to the spatial divisions in several previous reports studying the input-output relations of the hippocampus with other regions (upper row). Thus, this approach provides the molecular underpinning for these subdivisions and identifies molecular targets that can uniquely specify a functional domain. On a broader perspective, the same approach can be applied to all brain regions and shed new light on the functional organization of the brain.


Tracking of State Value in the Amygdala

Moment-to-Moment Tracking of State Value in the Amygdala
Belova et al. 28 (40): 10023 -- Journal of Neuroscience

The amygdala is important for the regulation of emotion and also in learning. Neurons in the amygdala changes their response properties during learning. But what exactly are the amygdala neurons doing? what are they encoding? what is their computational role?

Previous studies have establish that amygdala neurons encode the motivational value of the associated outcome. When animals learn that a cue CS is associated with an outcome US, a subset of amygdala neurons show stronger responses to CSs that are paired with rewarding USs (positive value coding, example neuron A at 1-2 sec). Conversely, another subset of neurons code negative value and respond more vigorously to CSs paired with punishment (like airpuff, example neuron B at 2-3 sec).

The current study extends these previously findings to suggest that these amygdala neurons in fact code the value of the state in the generic sense. In essence, what the authors found was that neurons encode value not only for the CS, but also for other behavioral epochs, including the fixation point and also to the US. In the above examples, neuron A prefered rewarding CS and also showed stronger response to the fixation point, while neuron B preferred aversive CS and reduced its firing rate to the fixation point. This pattern is generally true for the population (see D).

I think this paper is really powerful because the conclusion is based not only on one behavioral epoch but on all the epochs. This comes much closer to understanding the roles of these amygdala neurons, which is to keep track of the current state value. Since neurons are active all the time and in various behavioral contexts, a thourough understanding of their physiological/behavioral roles require considering all these scenarios, not just any particular one.

This paper is also important because the coding of state value has important implications in computational theories of learning.


Neural correlate of decision confidence and uncertainty

Adam Kepecs1, Naoshige Uchida1,2, Hatim A. Zariwala1,3 & Zachary F. Mainen1,4

Humans and other animals must often make decisions on the basis of imperfect evidence1, 2. Statisticians use measures such as P values to assign degrees of confidence to propositions, but little is known about how the brain computes confidence estimates about decisions. We explored this issue using behavioural analysis and neural recordings in rats in combination with computational modelling. Subjects were trained to perform an odour categorization task that allowed decision confidence to be manipulated by varying the distance of the test stimulus to the category boundary. To understand how confidence could be computed along with the choice itself, using standard models of decision-making3, 4, 5, 6, we defined a simple measure that quantified the quality of the evidence contributing to a particular decision. Here we show that the firing rates of many single neurons in the orbitofrontal cortex match closely to the predictions of confidence models and cannot be readily explained by alternative mechanisms, such as learning stimulus–outcome associations7, 8, 9, 10. Moreover, when tested using a delayed reward version of the task, we found that rats' willingness to wait for rewards increased with confidence, as predicted by the theoretical model. These results indicate that confidence estimates, previously suggested to require 'metacognition'11, 12 and conscious awareness13, 14, are available even in the rodent brain, can be computed with relatively simple operations, and can drive adaptive behaviour. We suggest that confidence estimation may be a fundamental and ubiquitous component of decision-making.

This is a really nice paper by Adam Kepecs et al from CSHL. The gist of the paper is in the following figure. The model (a-d) illustrates the perceptual decision process of determining which category the odorant stimulus (s) belongs to, by comparing the stimulus (si) to the memory of the boundary (bi), each drawn from its respective distribution. This leads to the determination of the perceptual category (si>bi or si less than bi) and the decision confidence (absolute difference between si and bi).  The behavior of the decision confidence construct (d) is matched perfectly with neuronal responses in the OFC (e, g). 

One important concern of the authors' interpretation is whether the OFC neuronal activity signals uncertainty or an error signal. This is nicely addressed in the method section:

The observed selectivity of neural activity for the upcoming outcome might arise if, after executing a choice, extra sensory or memory information enters decision-making circuits and causes the realization that an error occurred even before obtaining feedback. According to this interpretation the negative outcome selective population of OFC neurons would signal error44 instead of uncertainty. In contrast, the highest observed firing rates were associated with near chance level performance and not errors (Fig. 4g, f). To test this more rigorously, we asked whether an ideal observer could obtain better performance than the experimental subject if it could switch choices based on the firing rate after the choice and before feedback is provided. In all but one negative outcome selective neuron (1/133), the highest firing rates (top 5% of trials) were associated with chance level performance (within the 95% confidence interval). Therefore negative outcome selectivity does not imply that OFC neurons are actually able to predict error trials but rather that high firing rates predict near chance level performance consistent with an uncertainty signal.

The discovery of this neural correlate of decision confidence opens many new research questions. How is uncertainty/confidence calculated in the neural circuits? What are the influences of decision confidence on behavioral responses, learning and memory, and attention? No doubt these questions will be further explored by combining elegant experimental designs with a sound theoretical framework. Excellent work!

Synaptic plasticity differs for D1- and D2-striatal neurons

Neurons in the same region are not created equal. The exact cell type matters, and matters a lot.

Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity
Weixing Shen,1 Marc Flajolet,2 Paul Greengard,2 D. James Surmeier1*

At synapses between cortical pyramidal neurons and principal striatal medium spiny neurons (MSNs), postsynaptic D1 and D2 dopamine (DA) receptors are postulated to be necessary for the induction of long-term potentiation and depression, respectively—forms of plasticity thought to underlie associative learning. Because these receptors are restricted to two distinct MSN populations, this postulate demands that synaptic plasticity be unidirectional in each cell type. Using brain slices from DA receptor transgenic mice, we show that this is not the case. Rather, DA plays complementary roles in these two types of MSN to ensure that synaptic plasticity is bidirectional and Hebbian. In models of Parkinson's disease, this system is thrown out of balance, leading to unidirectional changes in plasticity that could underlie network pathology and symptoms."
What are the functional consequences of the differential regulatory rules for synaptic plasticity in D1- and D2-striatal neurons?
In the absence of behaviorally important stimuli, DA neurons spike autonomously to maintain striatal DA concentrations at levels sufficient to keep high-affinity D2 DA receptors active, but not low affinity D1 DA receptors —in principle enabling bidirectional, Hebbian plasticity in D2 MSNs, but not in D1 MSNs, where the low level of D1 receptor activity should permit only LTD. However, when behaviorally important stimuli drive phasic spiking of mesencephalic DA neurons, striatal DA levels rise transiently and activate D1 DA receptors ; this should enable the induction of Hebbian LTP in D1 MSNs.


Rubber hand illusion


"In the 'rubber hand' illusion, a person's hand and an adjacent rubber hand are both brushed gently. The real hand is kept out of sight. Before long, the subject's brain creates a new spatial link, imagining that the sensation in the real hand is arising where the rubber hand is."

Graduate student Matthew Mulvey of Leeds Metropolitan University has now shown that the effect will work if the researchers deliver transcutaneous electrical nerve stimulation (TENS) not to the hidden hand but to the wrist. After being primed with the illusion, subjects perceive the impulses--which hijack the nerve pathways between hand and brain--as a tingling located in the rubber hand.
This is such a cool idea to take advantage of this "rubber hand" illusion as a way of establishing sensory feedback on a neuro-prosthetic device.


Neuronal Ensemble Bursting in the Basal Forebrain Encodes Salience Irrespective of Valence

Neuronal Ensemble Bursting in the Basal Forebrain Encodes Salience Irrespective of Valence
Shih-Chieh Lin and Miguel A.L. Nicolelis

Our paper finally comes out in Neuron today, accompanied by a preview from Lau and Salzman:

Although noncholinergic neurons in the basal forebrain are known to contribute to cognition, their response properties in behaving animals is unclear. In this issue of Neuron, Lin and Nicolelis demonstrate that these neurons represent the motivational salience of sensory stimuli and may modulate cortical processing to direct top-down attention.
This is our abstract and the main figure
Both reward- and punishment-related stimuli are motivationally salient and attract the attention of animals. However, it remains unclear how motivational salience is processed in the brain. Here, we show that both reward- and punishment-predicting stimuli elicited robust bursting of many noncholinergic basal forebrain (BF) neurons in behaving rats. The same BF neurons also responded with similar bursting to primary reinforcement of both valences. Reinforcement responses were modulated by expectation, with surprising reinforcement eliciting stronger BF bursting. We further demonstrate that BF burst firing predicted successful detection of near-threshold stimuli. Together, our results point to the existence of a salience-encoding system independent of stimulus valence. We propose that the encoding of motivational salience by ensemble bursting of noncholinergic BF neurons may improve behavioral performance by affecting the activity of widespread cortical circuits and therefore represents a novel candidate mechanism for top-down attention.


Reward-encoding in dorsal raphe (serotonergic) neurons

Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus.
J Neurosci. 2008 May 14;28(20):5331-43
Authors: Nakamura K, Matsumoto M, Hikosaka O

Serotonergic neurons in the dorsal raphe (DR) constitute one of the major neuromodulatory systems. What is the role of DR neurons in encoding reward, in comparison with midbrain dopaminergic neurons? The authors noted several differences: (1) DA neurons encode reward prediction error -- responding to reward only when the reward size is larger or smaller than expected. DR neurons respond to both reward and reward-predicting cues, whether or not they are expected. (2) DA neurons respond to reward with a phasic bursting response, while DR neurons show slower tonic responses.
In addition, DR neurons are heterogeneous, some prefer large while others prefer small reward. Without an independent means of verifying the neurochemical identity, it remains unclear which subset of DR neurons are serotonergic neurons.