Information from multiple sensory modalities interacts. Using functional magnetic resonance imaging (fMRI), we aimed to identify the neural structures correlated with how cooccurring sound modulates the visual motor response execution. The reaction time (RT) to audiovisual stimuli was significantly faster than the RT to visual stimuli. Signal detection analyses showed no significant difference in the perceptual sensitivity (d') between audiovisual and visual stimuli, while the response criteria (β or c) of the audiovisual stimuli was decreased compared to the visual stimuli. The functional connectivity between the left inferior parietal lobule (IPL) and bilateral superior temporal gyrus (STG) was enhanced in Go processing compared with No-go processing of audiovisual stimuli. Furthermore, the left precentral gyrus (PreCG) showed enhanced functional connectivity with the bilateral STG and other areas of the ventral stream in Go processing compared with No-go processing of audiovisual stimuli. These results revealed that the neuronal network correlated with modulations of the motor response execution after the presentation of both visual stimuli along with cooccurring sound in a multisensory Go/Nogo task, including the left IPL, left PreCG, bilateral STG and some areas of the ventral stream. The role of the interaction between the IPL and STG in transforming audiovisual information into motor behavior is discussed. The current study provides a new perspective for exploring potential brain mechanisms underlying how humans execute appropriate behaviors on the basis of multisensory information.

Sound-induced flash illusion (SIFI) is an auditory-dominated multisensory integration phenomenon in which flashes presented in conjunction with an unequal number of auditory sounds are illusorily perceived as equal in number to the auditory sounds. Previous studies on the factors that impact SIFI have mainly focused on top-down and bottom-up factors. This study aimed to explore the effects of top-down cognitive expectations on the SIFI by manipulating the proportion of trial types. The results showed that the accuracy of judgment was improved and reaction times were shortened when the instructions were consistent with the actual proportion of trial type. When the instructions were not consistent with the actual proportion of trial types, the instructions could still regulate the accuracy and reaction times in judging the fission illusion (i.e., a brief flash accompanied by two auditory stimuli tends to be perceived as two flashes) regardless of the actual proportion of trial types. The results indicated that top-down cognitive expectations could significantly reduce the fission illusion and accelerate the judgment, but the effect was not significant in the fusion illusion (i.e., two brief flashes accompanied by single auditory stimuli tend to be perceived as a single flash) due to the instability of the illusion.

Although previous studies have shown that semantic multisensory integration can be differentially modulated by attention focus, it remains unclear whether attentionally mediated multisensory perceptual facilitation could impact further cognitive performance. Using a delayed matching-to-sample paradigm, the present study investigated the effect of semantically congruent bimodal presentation on subsequent unisensory working memory (WM) performance by manipulating attention focus. The results showed that unisensory WM retrieval was faster in the semantically congruent condition than in the incongruent multisensory encoding condition. However, such a result was only found in the divided-modality attention condition. This result indicates that a robust multisensory representation was constructed during semantically congruent multisensory encoding with divided-modality attention; this representation then accelerated unisensory WM performance, especially auditory WM retrieval. Additionally, an overall faster unisensory WM retrieval was observed under the modality-specific selective attention condition compared with the divided-modality condition, indicating that the division of attention to address two modalities demanded more central executive resources to encode and integrate crossmodal information and to maintain a constructed multisensory representation, leaving few resources for WM retrieval. Additionally, the present finding may support the amodal view that WM has an amodal central storage component that is used to maintain modal-based attention-optimized multisensory representations.

Attention deficit hyperactivity disorder (ADHD) is a common neurodevelopmental disorder in school-age children. Although it has been well documented that children with ADHD are associated with impairment of executive functions including working memory (WM) and inhibitory control, there is not yet a consensus as to the relationship between ADHD and memory-driven attentional capture (i.e., representations in WM bias attention toward the WM-matched distractors). The present study herein examined whether children with ADHD have sufficient cognitive control to modulate memory-driven attentional capture. 73 school-age children (36 with ADHD and 37 matched typically developing (TD) children) were instructed to perform a visual search task while actively maintaining an item in WM. In such a paradigm, the modality and the validity of the memory sample were manipulated. The results showed that under the visual WM encoding condition, no memory-driven attentional capture was observed in TD children, but significant capture was found in children with ADHD. In addition, under the audiovisual WM encoding condition, memory-matched distractors did not capture the attention of both groups. The results indicate a deficit of cognitive control over memory-driven attentional capture in children with ADHD, which can be improved by multisensory WM encoding. These findings enrich the relationship between ADHD and cognitive control and provide new insight into the influence of cross-modal processing on attentional guidance.