(1) Background: In the era of generative AI (GenAI), assessing AI literacy is essential for understanding how effectively non-expert users can interact with AI. However, existing assessment tools primarily focus on users’ understanding of AI principles or rely on self-reported scales, neglecting critical thinking and actual interaction capabilities. To address this gap, this study aims to design and validate evaluation indicators targeting the behavioral process of human–GenAI interactions and analyze the impact of critical thinking. (2) Methods: Grounded in information literacy and critical thinking frameworks, this study operationalized human–AI interaction capabilities into behavioral indicators and rubrics through observation, surveys, and pilot studies. Data were collected from 121 undergraduates completing two real-world tasks with GenAI, and their interaction processes were documented and evaluated. (3) Results: The indicators showed acceptable inter-rater and internal consistency reliability. Exploratory and Confirmatory Factor Analysis confirmed a three-dimensional structure. Further analysis showed that interaction capabilities varied across gender, academic background, AIGC use frequency, critical thinking disposition levels, and question chain logic. (4) Conclusions: The developed evaluation indicators are reliable and valid. Further analysis reveals that a high critical thinking disposition can offset the disadvantage of lower usage frequency. This highlights the significance of critical thinking in enhancing human–GenAI interaction capabilities.
15 March 2023 in Educational Measurement: Issues and Practice
Previous research has indicated that frontal midline theta (FMθ) reflects a domain-general cognitive control mechanism of the prefrontal cortex. Brain imaging studies have shown that the inhibition of implicit stereotypes was dependent on this domain-general cognitive control mechanism. Based on this knowledge, the present study investigated the neural oscillatory correlates of implicit gender stereotype control in an extrinsic affective Simon task (EAST) using electrophysiological methods. Participants in this task conducted verification to white gender names and colored gender traits, and their behavioral response and electroencephalogram (EEG) were recorded during their performances. As predicted, stereotype-inconsistent trials resulted in reduced response accuracies and slower response times than stereotype-consistent trials. For the event-related potential (ERP) results, the enhanced performance of stereotype-inconsistent trials was accompanied by an enhanced N400 amplitude but an attenuated late positive potential amplitude. In contrast, early attentional components such as P2 and N2 as well as their amplitudes were impacted by the experimental manipulations and individual differences in gender factors. In addition, based on time–frequency (TF) analysis, we found that the enhanced performance of stereotype-inconsistent trials was also accompanied by an event-related synchronization on the frontal theta oscillation. This frontal theta appeared at a late processing stage and persisted across a time window from N400 to late positive potential. Additionally, this enhanced frontal theta effect was not modulated by the experimental manipulations and individual differences in gender factors. Based on these findings, the feature of frontal theta oscillation in the implicit gender stereotype control process was discussed.
Introduction: Humans have the ability to efficiently extract summary statistics (i.e., mean) from a group of similar objects, referred to as ensemble coding. Recent studies have demonstrated that ensemble perception of simple objects is modulated by the visual working memory (VWM) task through matching features in VWM. However, few studies have examined the extending scope of such a matching feature effect and the influence of the organization mode (i.e., the way of combining memory matching features with ensemble properties) on this effect. Two experiments were done to explore these questions.Methods: We used a dual-task paradigm for both experiments, which included a VWM task and a mean estimation task. Participants were required to adjust a test face to the mean identity face and report whether the irregular objects in a memory probe were identical or different to the studied objects. In Experiment 1, using identity faces as ensemble stimuli, we compared participants’ performances in trials where a subset color matched that of the studied objects to those of trials without color-matching subsets. In Experiment 2, we combined memory matching colors with ensemble properties in common region cues and compared the effect with that of Experiment 1.Results: Results of Experiments 1 and 2 showed an effect of the VWM task on high-level ensemble perception that was similar to previous studies using a low-level averaging task. However, the combined analysis of Experiments 1 and 2 revealed that memory matching features had less influence on mean estimations when matching features and ensemble properties combined in the common region than when combined as parts of a complete unit.Conclusion: These findings suggest that the impact of memory matching features is not limited by the level of stimulus feature, but can be impacted by the organization between matching features and ensemble target properties.