Significance: Visual recognition is often thought to depend on neural representations that primarily reflect the physical properties of the environment. However, in this study we demonstrate that the intent of the observer fundamentally perturbs cortical representations of visual objects. Using functional MRI we measured the patterns of response to identical objects under six different tasks. In any given task, these patterns could be used to distinguish which object was being viewed. However, this ability was disrupted when the task changed, indicating that object representations reflect not only the physical properties of the stimulus, but also the internal state of the observer.

Despite the importance of an observer’s goals in determining how a visual object is categorized, surprisingly little is known about how humans process the task context in which objects occur and how it may interact with the processing of objects. Using magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI) and multivariate techniques, we studied the spatial and temporal dynamics of task and object processing. Our results reveal a sequence of separate but overlapping task-related processes spread across frontoparietal and occipitotemporal cortex. Task exhibited late effects on object processing by selectively enhancing task-relevant object features, with limited impact on the overall pattern of object representations. Combining MEG and fMRI data, we reveal a parallel rise in task-related signals throughout the cerebral cortex, with an increasing dominance of task over object representations from early to higher visual areas. Collectively, our results reveal the complex dynamics underlying task and object representations throughout human cortex.

Real-world visual scenes are complex cluttered, and heterogeneous stimuli engaging scene- and object-selective cortical regions including parahippocampal place area (PPA), retrosplenial complex (RSC), and lateral occipital complex (LOC). To understand the unique contribution of each region to distributed scene representations, we generated predictions based on a neuroanatomical framework adapted from monkey and tested them using minimal scenes in which we independently manipulated both spatial layout (open, closed, and gradient) and object content (furniture, e.g., bed, dresser). Commensurate with its strong connectivity with posterior parietal cortex, RSC evidenced strong spatial layout information but no object information, and its response was not even modulated by object presence. In contrast, LOC, which lies within the ventral visual pathway, contained strong object information but no background information. Finally, PPA, which is connected with both the dorsal and the ventral visual pathway, showed information about both objects and spatial backgrounds and was sensitive to the presence or absence of either. These results suggest that 1) LOC, PPA, and RSC have distinct representations, emphasizing different aspects of scenes, 2) the specific representations in each region are predictable from their patterns of connectivity, and 3) PPA combines both spatial layout and object information as predicted by connectivity.

Perceptual expertise is traditionally associated with enhanced brain activity in response to objects of expertise in category-selective visual cortex, primarily face-selective regions. We reevaluated this view by investigating whether the brain activity associated with expertise in object recognition is limited to category-selective cortex and specifically whether the extent of expertise-related activity manifests automatically or whether it can be top-down modulated. We conducted 2 functional magnetic resonance imaging studies comparing changes in hemodynamic activity associated with car expertise in a conventional 1-back task (Experiment 1) and when the task relevance of cars was explicitly manipulated (Experiment 2). Whole-brain analysis unveiled extensive expertise-related activity throughout the visual cortex, starting as early as V1 and extending into nonvisual areas. However, when the cars were task irrelevant, the expertise-related activity drastically diminished, indeed, becoming similar to the activity elicited by cars in novices. We suggest that expertise entails voluntary top-down engagement of multiple neural networks in addition to stimulus-driven activation associated with perceptual mechanisms.

Our remarkable ability to process complex visual scenes is supported by a network of scene-selective cortical regions. Despite growing knowledge about the scene representation in these regions, much less is known about the temporal dynamics with which these representations emerge. We conducted two experiments aimed at identifying and characterizing the earliest markers of scene-specific processing. In the first experiment, human participants viewed images of scenes, faces, and everyday objects while event-related potentials (ERPs) were recorded. We found that the first ERP component to evince a significantly stronger response to scenes than the other categories was the P2, peaking ∼220 ms after stimulus onset. To establish that the P2 component reflects scene-specific processing, in the second experiment, we recorded ERPs while the participants viewed diverse real-world scenes spanning the following three global scene properties: spatial expanse (open/closed), relative distance (near/far), and naturalness (man-made/natural). We found that P2 amplitude was sensitive to these scene properties at both the categorical level, distinguishing between open and closed natural scenes, as well as at the single-image level, reflecting both computationally derived scene statistics and behavioral ratings of naturalness and spatial expanse. Together, these results establish the P2 as an ERP marker for scene processing, and demonstrate that scene-specific global information is available in the neural response as early as 220 ms.