People can seamlessly integrate a vast array of information from what they see and hear in the noisy and uncertain world. However, the neural underpinnings of audiovisual integration continue to be a topic of debate. Using strict inclusion criteria, we performed an activation likelihood estimation meta-analysis on 121 neuroimaging experiments with a total of 2,092 participants. We found that audiovisual integration is linked with the coexistence of multiple integration sites, including early cortical, subcortical, and higher association areas. Although activity was consistently found within the superior temporal cortex, different portions of this cortical region were identified depending on the analytical contrast used, complexity of the stimuli, and modality within which attention was directed. The context-dependent neural activity related to audiovisual integration suggests a flexible rather than fixed neural pathway for audiovisual integration. Together, our findings highlight a flexible multiple pathways model for audiovisual integration, with superior temporal cortex as the central node in these neural assemblies.
While artificial Intelligence (AI) has made significant advancements, the seeming absence of its emotional ability has hindered effective communication with humans. This study explores how ChatGPT (ChatGPT-3.5 Mar 23, 2023 Version) represents affective responses to emotional narratives and compare these responses to human responses. Thirty-four participants read affect-eliciting short stories and rated their emotional responses and 10 recorded ChatGPT sessions generated responses to the stories. Classification analyses revealed the successful identification of affective categories of stories, valence, and arousal within and across sessions for ChatGPT. Classification analyses revealed the successful identification of affective categories of stories, valence, and arousal within and across sessions for ChatGPT. Classification accuracies predicting affective categories of stories, valence, and arousal of humans based on the affective ratings of ChatGPT and vice versa were not significant, indicating differences in the way the affective states were represented., indicating differences in the way the affective states were represented. These findings suggested that ChatGPT can distinguish emotional states and generate affective responses consistently, but there are differences in how the affective states are represented between ChatGPT and humans. Understanding these mechanisms is crucial for improving emotional interactions with AI.
This study introduces a novel structure-based classification (SBC) framework that leverages pairwise distance representations of rating data to enhance classification performance while mitigating individual differences in scale usage. Unlike conventional feature-based approaches that rely on absolute rating scores, SBC transforms rating data into structured representations by computing pairwise distances between rating dimensions. This transformation captures the relational structure of ratings, ensuring consistency between training and test datasets and enhancing model robustness. To evaluate the effectiveness of this approach, we conducted a simulation study in which participants rated stimuli across multiple affective dimensions, with systematic individual differences in scale usage. The results demonstrated that SBC successfully classified affective stimuli despite these variations, performing comparably to traditional classification methods. The findings suggest that relational structures among rating dimensions contain meaningful information for affective classification, akin to functional connectivity approaches in cognitive neuroscience. By focusing on rating interdependencies as well as absolute values, SBC provides a robust and generalizable method for analyzing subjective responses, with implications for psychological research.