ChatGPT, a general-purpose conversation chatbot released on November 30, 2022, by OpenAI, is expected to impact every aspect of society. However, the potential impacts of this NLP tool on education remain unknown. Such impact can be enormous as the capacity of ChatGPT may drive changes to educational learning goals, learning activities, and assessment and evaluation practices. This study was conducted by piloting ChatGPT to write an academic paper, titled Artificial Intelligence for Education (see Appendix A). The piloting result suggests that ChatGPT is able to help researchers write a paper that is coherent, (partially) accurate, informative, and systematic. The writing is extremely efficient (2-3 hours) and involves very limited professional knowledge from the author. Drawing upon the user experience, I reflect on the potential impacts of ChatGPT, as well as similar AI tools, on education. The paper concludes by suggesting adjusting learning goals—students should be able to use AI tools to conduct subject-domain tasks and education should focus on improving students’ creativity and critical thinking rather than general skills. To accomplish the learning goals, researchers should design AI-involved learning tasks to engage students in solving real-world problems. ChatGPT also raises concerns that students may outsource assessment tasks. This paper concludes that new formats of assessments are needed to focus on creativity and critical thinking that AI cannot substitute.

Machine learning (ML) is an emergent computerised technology that relies on algorithms built by ‘learning’ from training data rather than ‘instruction’, which holds great potential to revolutionise science assessment. This study systematically reviewed 49 articles regarding ML-based science assessment through a triangle framework with technical, validity, and pedagogical features on three vertices. We found that a majority of the studies focused on the validity vertex, as compared to the other two vertices. The existing studies primarily involve text recognition, classification, and scoring with an emphasis on constructing scientific explanations, with a vast range of human-machine agreement measures. To achieve the agreement measures, most of the studies employed a cross-validation method, rather than self- or split-validation. ML allows complex assessments to be used by teachers without the burden of human scoring, saving both time and cost. Most studies used supervised ML, which relies on extraction of attributes from student work that was first coded by humans to achieve automaticity, rather than semi- or unsupervised ML. We found that 24 studies were explicitly embedded in science learning activities, such as scientific inquiry and argumentation, to provide feedback or learning guidance. This study identifies existing research gaps and suggests that all three vertices of the ML triangle should be addressed in future assessment studies, with an emphasis on the pedagogy and technology features.

While ongoing efforts have continuously emphasized the integration of ChatGPT with science teaching and learning, there are limited empirical studies exploring its actual utility in the classroom. This study aims to fill this gap by analyzing the lesson plans developed by 29 pre-service elementary teachers and assessing how they integrated ChatGPT into science learning activities. We first examined how ChatGPT was integrated with the subject domains, teaching methods/strategies and then evaluated the lesson plans using a GenAI-TPACK-based rubric. We further examined pre-service teachers' perceptions and concerns about integrating ChatGPT into science learning. Results show a diverse number of ChatGPT applications in different science domains—e.g., Biology (9/29), Chemistry (7/29), and Earth Science (7/29). Fourteen types of teaching methods/strategies were identified in the lesson plans. On average, the pre-service teachers' lesson plans scored high on the modified TPACK-based rubric (M = 3.29; SD = .91; on a 1-4 scale), indicating a reasonable envisage of integrating ChatGPT into science learning, particularly in ‘instructional strategies & ChatGPT’ (M = 3.48; SD = .99). However, they scored relatively lower on exploiting ChatGPT's functions toward its full potential (M = 3.00; SD = .93), compared to other aspects. We also identified several inappropriate use cases of ChatGPT in lesson planning (e.g., as a source of hallucinated internet material and technically unsupported visual guidance). Pre-service teachers anticipated ChatGPT to afford high-quality questioning, self-directed learning, individualized learning support, and formative assessment. Meanwhile, they also expressed concerns about its accuracy and the risks that students may be overly dependent on ChatGPT. They further suggested solutions to systemizing classroom dynamics between teachers and students. The study underscores the need for more research on the roles of generative AI in actual classroom settings and provides insights for future AI-integrated science learning.