This study evaluates “I Learn with Prompt Engineering”, a self-paced, self-regulated elective course designed to equip university students with skills in prompt engineering to effectively utilize large language models (LLMs), foster self-directed learning, and enhance academic English proficiency through generative AI applications. By integrating prompt engineering concepts with generative AI tools, the course supports autonomous learning and addresses critical skill gaps in language proficiency and market-ready capabilities. The study also examines EnSmart, an AI-driven tool powered by GPT-4 and integrated into Canvas LMS, which automates academic test content generation and grading and delivers real-time, human-like feedback. Performance evaluation, structured questionnaires, and surveys were used to evaluate the course’s impact on prompting skills, academic English proficiency, and overall learning experiences. Results demonstrated significant improvements in prompt engineering skills, with accessible patterns like “Persona” proving highly effective, while advanced patterns such as “Flipped Interaction” posed challenges. Gains in academic English were most notable among students with lower initial proficiency, though engagement and practice time varied. Students valued EnSmart’s intuitive integration and grading accuracy but identified limitations in question diversity and adaptability. The high final success rate demonstrated that proper course design (taking into consideration Panadero’s four dimensions of self-regulated learning) can facilitate successful autonomous learning. The findings highlight generative AI’s potential to enhance autonomous learning and task automation, emphasizing the necessity of human oversight for ethical and effective implementation in education.

Chatbots have gained widespread popularity for their task automation capabilities and consistent availability in various domains, including education. However, their ability to adapt to the continuously evolving and dynamic nature of knowledge is limited. This research investigates the implementation of an internet wizard to enhance the knowledge base of an open-domain question-answering chatbot. The proposed approach leverages search engines, particularly Google, and its features, including feature snippets, knowledge graph, and organic search, in conjunction with data science and natural language models. This mechanism empowers the chatbot to dynamically access the extensive and up-to-date knowledge available on the web, enabling the provision of real time and pertinent answers to user queries sourced from web documents. A pilot study in a higher education context evaluated the chatbot’s mechanism and features, confirming its proficiency in generating responses across a broad range of educational and non-educational topics. Positive feedback and high user satisfaction validate these findings. Notably, the chatbot’s dynamic feature of retrieving related or follow-up questions from search engines significantly enhances student engagement and facilitates exploration of supplementary information beyond the curriculum.

Integration of Large Language Models (LLMs) with Learning Management Systems (LMSs) can enhance task automation and accessibility in education. However, hallucination where LLMs generate inaccurate or misleading information remains a challenge. This study introduces the Dynamic Course Content Integration (DCCI) mechanism, which dynamically retrieves course content from Canvas LMS and structures it within an LLM’s context window via prompt engineering, enabling the LLM-powered assistant, Ask ME, to deliver context-aware, curriculum-aligned responses while mitigating hallucinations. A mixed-methods pilot study grounded in Self-Determination Theory (autonomy, competence) and the Technology Acceptance Model (perceived usefulness, ease of use) evaluated DCCI’s effectiveness with 120 first-year programming students at Eötvös Loránd University. The course focused on foundational programming patterns in C#, including writing program specifications. We analyzed 14,746 logged interactions and a post-course survey completed by 101 students. User satisfaction was measured via a 5-point Likert scale (turn-level ratings), while the survey assessed usability, engagement, and ethical concerns. Results indicated high satisfaction (mean 4.65/5) and strong recognition of Ask ME’s ability to provide timely, contextually relevant answers to administrative and course-related queries. 78.06% agreed that Ask ME’s Canvas integration reduced platform switching, improving usability, engagement, comprehension, and topic exploration. Many students reported reduced hesitation to ask questions and increased motivation for self-directed learning, though concerns about over-reliance on AI and reduced student–teacher interaction emerged. This study demonstrates that DCCI enhances LLM reliability, student satisfaction, and engagement in AI-driven educational automation, while highlighting the importance of balancing AI efficiency with pedagogical integrity and human–AI collaboration.

In recent years, chatbot technologies have evolved into modern information and communication technology applications that perform many virtual tasks, including learning. One of the challenges in improving the chatbot is the insufficient knowledge base of chatbots, including education-oriented conversational agents, the challenges in connecting the chatbot with course content on Massive Open Online Course platforms. In this study, a chatbot was developed to answer questions using publicly available technologies, specifically Application Program Interfaces (APIs) that promise convenient user accessibility via APIs, such as the Facebook Messenger platform along with wit.ai API, Canvas MOOC API, and Wikipedia API. API technologies were used to connect the chatbot to selected course content on the MOOC platform as well as to large knowledge bases such as Wikipedia to expand the knowledge base of the Conversational Agent. The course selected for the chatbot integration was on general informatics topics. Most course participants interacted with the chatbot via the Facebook Messenger platform using their handheld devices. Thus, integrating the chatbot into a widely used platform such as Facebook Messenger is a convenient and effective way for reaching learners. The API technology enabled an efficient connection between the chatbot and third-party apps, including the Messenger app, wit.ai, Canvas MOOC, and Wikipedia. This was due to the variety, richness, manipulation capabilities, and format of data that an API can transfer. In addition, the Wikipedia API seemed to be a vast source of information for expanding the chatbot's knowledge base. Not all of the queries posed to the chatbot were part of the course content. Some participants questioned the personality of the chatbot and were curious about the persona of the conversational agent. This suggests that a chatbot that has been endowed with some personality traits is stimulating and more likely to be accepted by learners.

The integration of Large Language Models (LLMs) with Learning Management Systems (LMSs) has the potential to enhance task automation and accessibility in education. However, hallucination where LLMs generate inaccurate or misleading information remains a significant challenge. This study introduces the Dynamic Course Content Integration (DCCI) mechanism, which dynamically retrieves and integrates course content and curriculum from Canvas LMS into the LLM-powered assistant, Ask ME. By employing prompt engineering to structure retrieved content within the LLM’s context window, DCCI ensures accuracy, relevance, and contextual alignment, mitigating hallucination. To evaluate DCCI’s effectiveness, Ask ME’s usability, and broader student perceptions of AI in education, a mixed-methods approach was employed, incorporating user satisfaction ratings and a structured survey. Results from a pilot study indicate high user satisfaction (4.614/5), with students recognizing Ask ME’s ability to provide timely and contextually relevant responses for both administrative and course-related inquiries. Additionally, a majority of students agreed that Ask ME’s integration with course content in Canvas LMS reduced platform-switching, improving usability, engagement, and comprehension. AI’s role in reducing classroom hesitation and fostering self-directed learning and intellectual curiosity was also highlighted. Despite these benefits and positive perception of AI tools, concerns emerged regarding over-reliance on AI, accuracy limitations, and ethical issues such as plagiarism and reduced student-teacher interaction. These findings emphasize the need for strategic AI implementation, ethical safeguards, and a pedagogical framework that prioritizes human-AI collaboration over substitution. This study contributes to AI-enhanced education by demonstrating how context-aware retrieval mechanisms like DCCI improve LLM reliability for educational automation and student engagement while ensuring responsible AI integration.

This study evaluates I Learn with Prompt Engineering, a self-regulated course designed to equip university students with skills in prompt engineering to effectively utilize large language models (LLMs), foster self-directed learning, and enhance academic English proficiency through generative AI applications. By integrating prompt engineering concepts with generative AI tools, the course supports autonomous learning and addresses critical skill gaps in language proficiency and market-ready capabilities. The study also examines EnSmart, an AI-driven tool powered by GPT-4 and integrated into Canvas LMS, which automates academic test content generation, grading, and delivers real-time, human-like feedback. Structured questionnaires and surveys were used to evaluate the course’s impact on prompting skills, academic English proficiency, and overall learning experiences. Results demonstrated significant improvements in prompt engineering skills, with accessible patterns like “Persona” proving highly effective, while advanced patterns such as “Flipped Interaction” posed challenges. Gains in academic English were most notable among students with lower initial proficiency, though engagement and practice time varied. Students valued EnSmart’s intuitive integration and grading accuracy but identified limitations in question diversity and adaptability. The findings highlight generative AI's potential to enhance autonomous learning and task automation, emphasizing the necessity of human oversight for ethical and effective implementation in education.