The COVID-19 pandemic has significantly affected the education system worldwide, which was forced to respond with a sudden shift to distance learning. While successful distance teaching requires careful thinking, planning, and the development of technological and human resources, there was no time for preparation in the current situation. Various physics courses, including lectures, tutorials, and laboratory courses, had to be transferred to online formats, resulting in a variety of simultaneous, asynchronous, and mixed activities. To investigate how physics students perceived the sudden shift to online learning, we developed a questionnaire and gathered data from $N=578$ physics students from five universities in Germany, Austria, and Croatia. In this article, we report how the problem-solving sessions (recitations) and laboratories were adapted, how students judge the different formats of the courses, and how useful and effective they perceived them. The results are correlated with the students’ self-efficacy ratings and other behavioral measures (such as self-regulated learning skills). This study is descriptive in nature, and a survey study design was implemented to examine the relationships among the variables. We found that good communication abilities ($r=0.48$, $p<0.001$) and self-organization skills ($r=0.63$, $p<0.001$) are positively correlated with perceived learning achievement. Furthermore, the previous duration of studies had a significant impact on several self-reported achievement measures, resulting in consistently lower scores of students in their first academic year compared with students who were further along academically. We draw conclusions and suggest implications for future online classes on the instructor and faculty level. Suggestions include (i) focusing on first-year courses with on-campus teaching when facing limited lecture hall capacities, (ii) offering special courses for promoting self-regulated learning skills, (iii) emphasizing the positive aspects of distance learning, and (iv) installing networking services for supporting student communication.

This study aimed at evaluating how students perceive the linguistic quality and scientific accuracy of ChatGPT responses to physics comprehension questions. A total of 102 first- and second-year physics students were confronted with three questions of progressing difficulty from introductory mechanics (rolling motion, waves, and fluid dynamics). Each question was presented with four different responses. All responses were attributed to ChatGPT, but in reality, one sample solution was created by the researchers. All ChatGPT responses obtained in this study were wrong, imprecise, incomplete, or misleading. We found little differences in the perceived linguistic quality between ChatGPT responses and the sample solution. However, the students rated the overall scientific accuracy of the responses significantly differently, with the sample solution being rated best for the questions of low and medium difficulty. The discrepancy between the sample solution and the ChatGPT responses increased with the level of self-assessed knowledge of the question content. For the question of highest difficulty (fluid dynamics) that was unknown to most students, a ChatGPT response was rated just as good as the sample solution. Thus, this study provides data on the students’ perception of ChatGPT responses and the factors influencing their perception. The results highlight the need for careful evaluation of ChatGPT responses both by instructors and students, particularly regarding scientific accuracy. Therefore, future research could explore the potential of similar “spot the bot” activities in physics education to foster students’ critical thinking skills.

A two-tier instrument for representational competence in the field of kinematics (KiRC) is presented, designed for a standard (1st year) calculus-based introductory mechanics course. It comprises 11 multiple choice (MC) and 7 multiple true-false (MTF) questions involving multiple representational formats, such as graphs, pictures, and formal (mathematical) expressions (1st tier). Furthermore, students express their answer confidence for selected items, providing additional information (2nd tier). Measurement characteristics of KiRC were assessed in a validation sample (pre- and post-test, $N=83$ and $N=46$, respectively), including usefulness for measuring learning gain. Validity is checked by interviews and by benchmarking KiRC against related measures. Values for item difficulty, discrimination, and consistency are in the desired ranges; in particular, a good reliability was obtained ($\mathrm{KR}20=0.86$). Confidence intervals were computed and a replication study yielded values within the latter. For practical and research purposes, KiRC as a diagnostic tool goes beyond related extant instruments both for the representational formats (e.g., mathematical expressions) and for the scope of content covered (e.g., choice of coordinate systems). Together with the satisfactory psychometric properties it appears a versatile and reliable tool for assessing students’ representational competency in kinematics (and of its potential change). Confidence judgments add further information to the diagnostic potential of the test, in particular for representational misconceptions. Moreover, we present an analytic result for the question—arising from guessing correction or educational considerations—of how the total effect size (Cohen’s $d$) varies upon combination of two test components with known individual effect sizes, and then discuss the results in the case of KiRC (MC and MTF combination). The introduced method of test combination analysis can be applied to any test comprising two components for the purpose of finding effect size ranges.

Eye tracking is becoming increasingly popular in physics education research (PER). As technology has advanced considerably in recent years and has become more user friendly, it is anticipated that eye tracking will play an increasingly significant role in assessing student learning at the process level in future studies. The main objective of this systematic review is to summarize the current status quo regarding eye tracking in PER and reviewing (a) the dissemination, (b) the methodological implementation, and (c) the insights provided by eye tracking in PER. We identified 33 journal articles, published between March 2005 and April 2021, that used eye tracking for original empirical research in the area of physics education. The results reveal that although eye tracking has been used in many different areas of physics, a clear focus on mechanics is evident, particularly for measuring visual attention in assessment scenarios like problem solving. While a high methodological rigor in the selection and analysis of the visual stimuli was apparent, only a few studies have provided a complete documentation of the technological implementation (e.g., movement restrictions, accuracy, and calibration information) and a theoretical embedding for interpreting eye-tracking data. To synthesize the results of the different studies, we created an inductive category system in accordance with the considered independent variables of the studies. Accordingly, visual attention was most frequently compared between levels of performance (correct vs incorrect or high vs low achievers), thereby leading to performance-discriminating factors of eye movement across studies. Furthermore, learners’ eye movements were compared across different stimuli, different time points, or between student groups to inform multimedia design and shed light on students’ learning progression. In summary, eye tracking is particularly useful for studying processes in different domains that are relevant to PER. Specific gaps in the literature, methodological limitations, and implications of existing findings were also identified to recommend future research and practices.

Relating mathematical concepts to graphical representations is a challenging task for students. In this paper, we introduce two visual strategies to qualitatively interpret the divergence of graphical vector field representations. One strategy is based on the graphical interpretation of partial derivatives, while the other is based on the flux concept. We test the effectiveness of both strategies in an instruction-based eye-tracking study with $N=41$ physics majors. We found that students’ performance improved when both strategies were introduced (74% correct) instead of only one strategy (64% correct), and students performed best when they were free to choose between the two strategies (88% correct). This finding supports the idea of introducing multiple representations of a physical concept to foster student understanding. Relevant eye-tracking measures demonstrate that both strategies imply different visual processing of the vector field plots, therefore reflecting conceptual differences between the strategies. Advanced analysis methods further reveal significant differences in eye movements between the best and worst performing students. For instance, the best students performed predominantly horizontal and vertical saccades, indicating correct interpretation of partial derivatives. They also focused on smaller regions when they balanced positive and negative flux. This mixed-method research leads to new insights into student visual processing of vector field representations, highlights the advantages and limitations of eye-tracking methodologies in this context, and discusses implications for teaching and for future research. The introduction of saccadic direction analysis expands traditional methods, and shows the potential to discover new insights into student understanding and learning difficulties.

The coordination of multiple external representations is important for learning, but yet a difficult task for students, requiring instructional support. The subject in this study covers a typical relation in physics between abstract mathematical equations (definitions of divergence and curl) and a visual representation (vector field plot). To support the connection across both representations, two instructions with written explanations, equations, and visual representations (differing only in the presence of visual cues) were designed and their impact on students’ performance was tested. We captured students’ eye movements while they processed the written instruction and solved subsequent coordination tasks. The results show that students instructed with visual cues (VC students) performed better, responded with higher confidence, experienced less mental effort, and rated the instructional quality better than students instructed without cues. Advanced eye-tracking data analysis methods reveal that cognitive integration processes appear in both groups at the same point in time but they are significantly more pronounced for VC students, reflecting a greater attempt to construct a coherent mental representation during the learning process. Furthermore, visual cues increase the fixation count and total fixation duration on relevant information. During problem solving, the saccadic eye movement pattern of VC students is similar to experts in this domain. The outcomes imply that visual cues can be beneficial in coordination tasks, even for students with high domain knowledge. The study strongly confirms an important multimedia design principle in instruction, that is, that highlighting conceptually relevant information shifts attention to relevant information and thus promotes learning and problem solving. Even more, visual cues can positively influence students’ perception of course materials.