Decision-making invokes two fundamental axes of control: affect or valence, spanning reward and punishment, and effect or action, spanning invigoration and inhibition. We studied the acquisition of instrumental responding in healthy human volunteers in a task in which we orthogonalized action requirements and outcome valence. Subjects were much more successful in learning active choices in rewarded conditions, and passive choices in punished conditions. Using computational reinforcement-learning models, we teased apart contributions from putatively instrumental and Pavlovian components in the generation of the observed asymmetry during learning. Moreover, using model-based fMRI, we showed that BOLD signals in striatum and substantia nigra/ventral tegmental area (SN/VTA) correlated with instrumentally learnt action values, but with opposite signs for go and no-go choices. Finally, we showed that successful instrumental learning depends on engagement of bilateral inferior frontal gyrus. Our behavioral and computational data showed that instrumental learning is contingent on overcoming inherent and plastic Pavlovian biases, while our neuronal data showed this learning is linked to unique patterns of brain activity in regions implicated in action and inhibition respectively.
Working memory allows information from transient events to persist as active neural representations that can be used for goal-directed behaviors such as decision making and learning. Computational modeling based on neuronal firing patterns in animals suggests that one putative mechanism enabling working memory is periodic reactivation (henceforth termed "replay") of the maintained information coordinated by neural oscillations at theta (4-8 Hz) and gamma (30-80 Hz) frequency. To investigate this possibility, we trained multivariate pattern classifier decoding algorithms on oscillatory brain responses to images depicting natural scenes, recorded with high temporal resolution via magnetoencephalography. These classifiers were applied to brain activity recorded during the subsequent five second maintenance of the scenes. This decoding revealed replay during the entire maintenance interval. Replay was specific to whether an indoor or an outdoor scene was maintained and whether maintenance centered on configural associations of scene elements or just single scene elements. Replay was coordinated by the phase of theta and the amount of theta coordination was correlated with working memory performance. By confirming the predictions of a mechanistic model and linking these to behavioral performance in humans, these findings identify theta-coupled replay as a mechanism of working memory maintenance.
The acquisition of reward and the avoidance of punishment could logically be contingent on either emitting or withholding particular actions. However, the separate pathways in the striatum for go and no-go appear to violate this independence, instead coupling affect and effect. Respect for this interdependence has biased many studies of reward and punishment, so potential action–outcome valence interactions during anticipatory phases remain unexplored. In a functional magnetic resonance imaging study with healthy human volunteers, we manipulated subjects' requirement to emit or withhold an action independent from subsequent receipt of reward or avoidance of punishment. During anticipation, in the striatum and a lateral region within the substantia nigra/ventral tegmental area (SN/VTA), action representations dominated over valence representations. Moreover, we did not observe any representation associated with different state values through accumulation of outcomes, challenging a conventional and dominant association between these areas and state value representations. In contrast, a more medial sector of the SN/VTA responded preferentially to valence, with opposite signs depending on whether action was anticipated to be emitted or withheld. This dominant influence of action requires an enriched notion of opponency between reward and punishment.
Recollection of contextual information represents the core of human recognition memory. It has been associated with theta (4–8 Hz) power in electrophysiological recordings and, independently, with BOLD effects in a network including the hippocampus and frontal cortex. Although the notion of the hippocampus coordinating neocortical activity by synchronization in the theta range is common among theoretical models of recollection, direct evidence supporting this hypothesis is scarce. To address this apparent gap in our understanding of memory processes, we combined EEG and fMRI during a remember/know recognition task. We can show that recollection-specific theta-alpha (4–13 Hz) effects are correlated with increases in hippocampal connectivity with the PFC and, importantly, the striatum, areas that have been linked repeatedly to retrieval success. Together, our results provide compelling evidence that low-frequency oscillations in the theta and alpha range provide a mechanism to functionally bind the hippocampus, PFC, and striatum during successful recollection.SIGNIFICANCE STATEMENTLow-frequency oscillations are supposed to drive the binding of information across a large-scale network centered on the hippocampus, which supports mnemonic functions. The electrophysiological means to investigate this phenomenon in humans (EEG/MEG), however, are inherently limited by their spatial resolution and therefore do not allow a precise localization of the brain regions involved. By combining EEG with BOLD-derived estimates of hippocampal connectivity during recognition, we can identify the striatum and specific areas in the medial and lateral PFC as part of a circuit linked to low-frequency oscillations (4–13 Hz) that promotes hippocampus-dependent context retrieval. Therefore, the current study closes an apparent gap in our understanding of the network dynamics of memory retrieval.
Long-term memories are linked to cortical representations of perceived events, but it is unclear which types of representations can later be recollected. Using magnetoencephalography-based decoding, we examined which brain activity patterns elicited during encoding are later replayed during recollection in the human brain. The results show that the recollection of images depicting faces and scenes is associated with a replay of neural representations that are formed at very early (180 ms) stages of encoding. This replay occurs quite rapidly, ∼500 ms after the onset of a cue that prompts recollection and correlates with source memory accuracy. Therefore, long-term memories are rapidly replayed during recollection and involve representations that were formed at very early stages of encoding. These findings indicate that very early representational information can be preserved in the memory engram and can be faithfully and rapidly reinstated during recollection. These novel insights into the nature of the memory engram provide constraints for mechanistic models of long-term memory function.