Recently, artificial intelligence (AI) technologies have been widely used in the field of education, and artificial intelligence in education (AIEd) has gained increasing attention. However, no quantitative meta-analysis has been conducted on the overall effectiveness of AI on learning achievement and learning perception. To close this research gap, this study conducted a comprehensive meta-analysis of the effects of AI on learning achievement and learning perception. The present meta-analysis synthesized 24 articles with a total of 2908 participants from 2001 to 2020. The findings reveal that AI had a high effect size on learning achievement and a small effect size on learning perception. The effect sizes of 13 moderator variables were analyzed, including sample levels, sample size, learning domains, learning methods, research design, research settings, intervention duration, types of organization for treatment, role of AI, areas of AI application, AI software, AI hardware, and AI technologies. It was found that sample size, sample level, learning domains, types of organization, roles of AI, and hardware significantly moderated the effectiveness of AI. The results and the implications for educators and practitioners are discussed in depth.

This study explores the effectiveness of chatbots empowered by generative artificial intelligence (GAI) in assisting university students’ creative problem-solving (CPS). We used quasi-experiments to compare the performance of dialogue dynamics, learner perceptions, and practical competencies in CPS during students’ interactions with: (1) a GAI chatbot, and (2) their peers. In total, 80 postgraduate students participated. The assigned CPS task was the creation of an innovative research proposal. We found that there were significant differences in the dialogic exchanges observed between the two types of interaction. Student-GAI chatbot interactions featured more knowledge-based dialogue and elaborate discussions, with less subjective expression compared to student-peer interactions. Notably, students contributed significantly less dialogue when interacting with a GAI chatbot than they did during peer interactions. The dialogic exchanges arising from student-GAI chatbot interactions tended to follow distinct patterns, while those from student-peer interactions were less predictable. The students perceived interacting with a GAI chatbot as more useful and easier than interacting with peers. Furthermore, they exhibited higher intention levels when utilising a GAI chatbot to tackle the CPS task compared to engaging in discussions with their peers. Ultimately, practical performance was significantly enhanced through interactions with a GAI chatbot. This study implies that the prudent use of GAI-based techniques can facilitate university students’ learning achievement.