10 August 2010 in PLOS Biology
Integrating information across sensory domains to construct a unified representation of multi-sensory signals is a fundamental characteristic of perception in ecological contexts. One provocative hypothesis deriving from neurophysiology suggests that there exists early and direct cross-modal phase modulation. We provide evidence, based on magnetoencephalography (MEG) recordings from participants viewing audiovisual movies, that low-frequency neuronal information lies at the basis of the synergistic coordination of information across auditory and visual streams. In particular, the phase of the 2–7 Hz delta and theta band responses carries robust (in single trials) and usable information (for parsing the temporal structure) about stimulus dynamics in both sensory modalities concurrently. These experiments are the first to show in humans that a particular cortical mechanism, delta-theta phase modulation across early sensory areas, plays an important “active” role in continuously tracking naturalistic audio-visual streams, carrying dynamic multi-sensory information, and reflecting cross-sensory interaction in real time. When faced with ecologically relevant stimuli in natural scenes, our brains need to coordinate information from multiple sensory systems in order to create accurate internal representations of the outside world. Unfortunately, we currently have little information about the neuronal mechanisms for this cross-modal processing during online sensory perception under natural conditions. Neurophysiological and human imaging studies are increasingly exploring the response properties elicited by natural scenes. In this study, we recorded magnetoencephalography (MEG) data from participants viewing audiovisual movie clips. We developed a phase coherence analysis technique that captures—in single trials of watching a movie—how the phase of cortical responses is tightly coupled to key aspects of stimulus dynamics. Remarkably, auditory cortex not only tracks auditory stimulus dynamics but also reflects dynamic aspects of the visual signal. Similarly, visual cortex mainly follows the visual properties of a stimulus, but also shows sensitivity to the auditory aspects of a scene. The critical finding is that cross-modal phase modulation appears to lie at the basis of this integrative processing. Continuous cross-modal phase modulation may permit the internal construction of behaviorally relevant stimuli. Our work therefore contributes to the understanding of how multi-sensory information is analyzed and represented in the human brain.
2 April 2014 in Journal of Neuroscience
Neuronal oscillations are ubiquitous in the brain and contribute to perception and attention. However, most associated evidence derives frompost hoccorrelations between brain dynamics and behavior. Although a few recent studies demonstrate rhythms in behavior, it remains largely unknown whether behavioral performances manifest spectrotemporal dynamics in a neurophysiologically relevant manner (e.g., the temporal modulation of ongoing oscillations, the cross-frequency coupling). To investigate the issue, we examined fine spectrotemporal dynamics of behavioral time courses in a large sample of human participants (n= 49), by taking a high time-resolved psychophysical measurement in a precuing attentional task. We observed compelling dynamic oscillatory patterns directly in behavior. First, typical attentional effects are demonstrated in low-pass (0–2 Hz) filtered time courses of behavioral responses. Second, an uninformative peripheral cue elicits recurring α-band (8–20 Hz) pulses in behavioral performances, and the elicited α pulses for cued and uncued conditions are in a temporally alternating relationship. Finally, ongoing α-band power is phase locked to ongoing θ-bands (3–5 Hz) in behavioral time courses. Our findings constitute manifestation of oscillations at physiologically relevant rhythms and power-phase locking, as widely observed in neurophysiological recordings, in behavior. The findings suggest that behavioral performance actually consists of rich dynamic information and may reflect underlying neuronal oscillatory substrates. Our data also speak to a neural mechanism for item attention based on successive cycles (θ) of a sequential attentional sampling (α) process.
1 January 2012 in Frontiers in Psychology
Natural sounds, including vocal communication sounds, contain critical information at multiple time scales. Two essential temporal modulation rates in speech have been argued to be in the low gamma band (∼20–80 ms duration information) and the theta band (∼150–300 ms), corresponding to segmental and diphonic versus syllabic modulation rates, respectively. It has been hypothesized that auditory cortex implements temporal integration using time constants closely related to these values. The neural correlates of a proposed dual temporal window mechanism in human auditory cortex remain poorly understood. We recorded MEG responses from participants listening to non-speech auditory stimuli with different temporal structures, created by concatenating frequency-modulated segments of varied segment durations. We show that such non-speech stimuli with temporal structure matching speech-relevant scales (∼25 and ∼200 ms) elicit reliable phase tracking in the corresponding associated oscillatory frequencies (low gamma and theta bands). In contrast, stimuli with non-matching temporal structure do not. Furthermore, the topography of theta band phase tracking shows rightward lateralization while gamma band phase tracking occurs bilaterally. The results support the hypothesis that there exists multi-time resolution processing in cortex on discontinuous scales and provide evidence for an asymmetric organization of temporal analysis (asymmetrical sampling in time, AST). The data argue for a mesoscopic-level neural mechanism underlying multi-time resolution processing: the sliding and resetting of intrinsic temporal windows on privileged time scales.
11 February 2015 in Journal of Neuroscience
The brain constantly creates perceptual predictions about forthcoming stimuli to guide perception efficiently. Abundant studies have demonstrated that perceptual predictions modulate sensory activities depending on whether the actual inputs are consistent with one particular prediction. In real-life contexts, however, multiple and even conflicting predictions might concurrently exist to be tested, requiring a multiprediction coordination process. It remains largely unknown how multiple hypotheses are conveyed and harmonized to guide moment-by-moment perception. Based on recent findings revealing that multiple locations are sampled alternatively in various phase of attentional rhythms, we hypothesize that this oscillation-based temporal organization mechanism may also underlie the multiprediction coordination process. To address the issue, we used well established priming paradigms in combination with a time-resolved behavioral approach to investigate the fine temporal dynamics of the multiprediction harmonization course in human subjects. We first replicate classical priming effects in slowly developing trends of priming time courses. Second, after removing the typical priming patterns, we reveal a new theta-band (∼4 Hz) oscillatory component in the priming behavioral data regardless of whether the prime was masked. Third, we show that these theta-band priming oscillations triggered by congruent and incongruent primes are in an out-of-phase relationship. These findings suggest that perceptual predictions return to low-sensory areas not continuously but recurrently in a theta-band rhythm (every 200–300 ms) and that multiple predictions are dynamically coordinated in time by being conveyed in different phases of the theta-band oscillations to achieve dissociated but temporally organized neural representations.