Learning with hands‐on experiments can be supported by providing essential information virtually during lab work. Augmented reality (AR) appears especially suitable for presenting information during experimentation, as it can be used to integrate both physical and virtual lab work. Virtual information can be displayed in close spatial proximity to the correspondent components in the experimentation environment, thereby ensuring a basic design principle for multimedia instruction: the spatial contiguity principle. The latter is assumed to reduce learners' extraneous cognitive load and foster generative processing, which supports conceptual knowledge acquisition. For the present study, a tablet‐based AR application has been developed to support learning from hands‐on experiments in physics education. Real‐time measurement data were displayed directly above the components of electric circuits, which were constructed by the learners during lab work. In a two group pretest–posttest design, we compared university students' ( N = 50) perceived cognitive load and conceptual knowledge gain for both the AR‐supported and a matching non‐AR learning environment. Whereas participants in both conditions gave comparable ratings for cognitive load, learning gains in conceptual knowledge were only detectable for the AR‐supported lab work.

Recently, the option to use large language models as a middleware connecting various AI tools and other large language models led to the development of so-called large multimodal foundation models, which have the power to process spoken text, music, images and videos. In this overview, we explain a new set of opportunities and challenges that arise from the integration of large multimodal foundation models in education.

Multimedia learning theories suggest presenting associated pieces of information in spatial and temporal contiguity. New technologies like Augmented Reality allow for realizing these principles in science laboratory courses by presenting virtual real-time information during hands-on experimentation. Spatial integration can be achieved by pinning virtual representations of measurement data to corresponding real components. In the present study, an Augmented Reality-based presentation format was realized via a head-mounted display and contrasted to a separate display, which provided a well-arranged data matrix in spatial distance to the real components and was therefore expected to result in a spatial split-attention effect. Two groups of engineering students (N = 107; Augmented Reality vs. separate display) performed six experiments exploring fundamental laws of electric circuits. Cognitive load and conceptual knowledge acquisition were assessed as main outcome variables. In contrast to our hypotheses and previous findings, the Augmented Reality group did not report lower extraneous load and the separate display group showed higher learning gains. The pre- and posttest assessing conceptual knowledge were monitored by eye tracking. Results indicate that the condition affected the visual relevancy of circuit diagrams to final problem completion. The unexpected reverse effects could be traced back to emphasizing coherence formation processes regarding multiple measurements.

Driving simulators are becoming increasingly common in driver training and assessment. Since virtual reality is generally regarded as an appropriate environment for measuring risk behavior, simulators are also used to assess hazard perception, which is considered to be one of the most important skills for safe driving. Simulators, which offer challenges that are indeed comparable to driving in real traffic, but at a very low risk of physical injury, have the potential to complement theoretical and practical driver trainings and tests. Although configurations and fidelity differ considerably between driving simulators, studies comparing the impact of their distinct features on driving performance and test validity remain rare. In this context, prior research demonstrated that a wider field of view (three monitors compared to a single monitor) led to earlier speed adjustments in response to potential hazards—especially for experienced drivers. The wider field of view was assumed to cause the drivers to be more present in the virtual world, which in turn provoked more natural scanning of the road and therefore, earlier hazard detection in experienced drivers. Research on spatial presence in other contexts support this assumption. The present experiment investigated whether this effect could be enhanced by an even more immersive presentation technique for driving simulation: a head-mounted display (HMD). Moreover, we studied the interplay between display mode, sense of presence and simulation sickness. Eighty experienced and less experienced drivers completed six simulation-based hazard perception scenarios, which were displayed either via a triple-monitor set-up or an HMD. Results indicate that the experienced drivers showed very similar driving and risk behavior as the inexperienced drivers in both experimental conditions. However, there were significant differences between the two display conditions. The use of an HMD resulted in a clearer and more abrupt speed reduction, more virtual presence, and a higher degree of simulation sickness. However, the interrelation between these three variables could not be conclusively clarified in the present study and thus represents a research aim that could be addressed in future studies.

Augmenting reality via head-mounted displays (HMD-AR) is an emerging technology in education. The interactivity provided by HMD-AR devices is particularly promising for learning, but presents a challenge to human activity recognition, especially with children. Recent technological advances regarding speech and gesture recognition concerning Microsoft’s HoloLens 2 may address this prevailing issue. In a within-subjects study with 47 elementary school children (2nd to 6th grade), we examined the usability of the HoloLens 2 using a standardized tutorial on multimodal interaction in AR. The overall system usability was rated “good”. However, several behavioral metrics indicated that specific interaction modes differed in their efficiency. The results are of major importance for the development of learning applications in HMD-AR as they partially deviate from previous findings. In particular, the well-functioning recognition of children’s voice commands that we observed represents a novelty. Furthermore, we found different interaction preferences in HMD-AR among the children. We also found the use of HMD-AR to have a positive effect on children’s activity-related achievement emotions. Overall, our findings can serve as a basis for determining general requirements, possibilities, and limitations of the implementation of educational HMD-AR environments in elementary school classrooms.

Representational competence is essential for the acquisition of conceptual understanding in physics. It enables the interpretation of diagrams, graphs, and mathematical equations, and relating these to one another as well as to observations and experimental outcomes. In this study, we present the initial validation of a newly developed cross-contextual assessment of students’ competence in representing vector-field plots and field lines, the most common visualization of the concept of vector fields. The Representational Competence of Fields Inventory (RCFI) consists of ten single choice items and two items that each contain three true or false questions. The tool can be easily implemented within an online assessment. It assesses the understanding of the conventions of interpreting field lines and vector-field plots, as well as the translation between these. The intended use of the tool is both to scale students’ representational competences in respect to representations of vector fields and to reveal related misconceptions (areas of difficulty). The tool was administered at three German-speaking universities in Switzerland and Germany to a total of 515 first- and third-semester students from science, technology, engineering, and mathematics subjects. In these first steps of the validation of the RCFI, we evaluated its psychometric quality via classical test theory in combination with Rasch scaling and examined its construct validity by conducting student interviews. The RCFI exhibits a good internal consistency of $\omega =0.86$, and the results of the Rasch analysis revealed that the items discriminate well among students from lower to medium-high competence levels. The RCFI revealed several misunderstandings and shortcomings, such as the confusion of the conventions for representing field lines and vector-field plots. Moreover, it showed that many students believed that field lines must not exhibit a curvature, that the lengths of field lines matter, and that field lines may have sharp corners. In its current version, the RCFI allows assessing students’ competence to interpret field representations, a necessary prerequisite for learning the widespread concept of vector fields. We report on planned future adaptations of the tool, such as optimizing some of the current distractors.