▪ Abstract The newly defined field of cognitive neuroscience attempts to draw together the study of all brain mechanisms that underlie our mental life. Historically, the major sensory pathways have provided the most trustworthy insights into how the brain supports cognitive functions such as perception, attention, and short-term memory. The links between neural activity and perception, in particular, have been studied revealingly in recent decades. Here we review the striking progress in this area, giving particular emphasis to the kinds of neural events that underlie the perceptual judgments of conscious observers.

The role of the primate middle temporal area (MT) in depth perception was examined by considering the trial-to-trial correlations between neuronal activity and reported depth sensations. A set of moving random dots portrayed a cylinder rotating about its principal axis. In this structure-from-motion stimulus, the direction of rotation is ambiguous and the resulting percept undergoes spontaneous fluctuations. The stimulus can be rendered unambiguous by the addition of binocular disparities. We trained monkeys to report the direction of rotation in a set of these stimuli, one of which had zero disparity. Many disparity-selective neurons in area MT are selective for the direction of rotation defined by disparity. Across repeated presentations of the ambiguous (zero-disparity) stimulus, there was a correlation between neuronal firing and the reported direction of rotation, as found by Bradley et al. (1998). Quantification of this effect using choice probabilities (Britten et al., 1996) allowed us to demonstrate that the correlation cannot be explained by eye movements, behavioral biases, or attention to spatial location. MT neurons therefore appear to be involved in the perceptual decision process. The mean choice probability (0.67) was substantially larger than that reported for MT neurons in a direction discrimination task (Britten et al., 1996). This implies that MT neurons make a different contribution to the two tasks. For the depth task, either the pool of neurons used is smaller or the correlation between neurons in the pool is larger.

Stereoscopic depth perception is based on binocular disparities. Although neurons in primary visual cortex (V1) are selective for binocular disparity, their responses do not explicitly code perceived depth. The stereoscopic pathway must therefore include additional processing beyond V1. We used functional magnetic resonance imaging (fMRI) to examine stereo processing in V1 and other areas of visual cortex. We created stereoscopic stimuli that portrayed two planes of dots in depth, placed symmetrically about the plane of fixation, or else asymmetrically with both planes either nearer or farther than fixation. The interplane disparity was varied parametrically to determine the stereoacuity threshold (the smallest detectable disparity) and the upper depth limit (largest detectable disparity). fMRI was then used to quantify cortical activity across the entire range of detectable interplane disparities. Measured cortical activity covaried with psychophysical measures of stereoscopic depth perception. Activity increased as the interplane disparity increased above the stereoacuity threshold and dropped as interplane disparity approached the upper depth limit. From the fMRI data and an assumption that V1 encodes absolute retinal disparity, we predicted that the mean response of V1 neurons should be a bimodal function of disparity. A post hoc analysis of electrophysiological recordings of single neurons in macaques revealed that, although the average firing rate was a bimodal function of disparity (as predicted), the precise shape of the function cannot fully explain the fMRI data. Although there was widespread activity within the extrastriate cortex (consistent with electrophysiological recordings of single neurons), area V3A showed remarkable sensitivity to stereoscopic stimuli, suggesting that neurons in V3A may play a special role in the stereo pathway.

Most neurophysiological accounts of disparity selectivity in neurons of the primary visual cortex (V1) imply that they are selective for absolute retinal disparities. By contrast, a number of psychophysical observations indicate that relative disparities play a more important role in depth perception. During recordings from disparity selective neurons in area V1 of awake behaving monkeys, we used a disparity feedback loop (Rashbass and Westheimer, 1961) to add controlled amounts of absolute disparity to a display containing both absolute and relative disparities. This manipulation changed the absolute disparity of all the visible features in the display but left unchanged the relative disparities signalled by these features. The addition of absolute disparities produced clear changes in the neural responses to unchanged external stimuli, which were well predicted by the measured change in absolute disparity: in 45/53 cases, the neuron maintained a consistent firing pattern with respect to absolute disparity so that the manipulation created no significant change in the absolute disparity preferred by the neuron. No neuron in V1 maintained a consistent relationship with relative disparity. We conclude that the relative disparity signals used in primate depth perception are constructed outside area V1.