Web accessibility evaluation is a costly process that usually requires manual intervention. Currently, large language model (LLM) based systems have gained popularity and shown promising capabilities to perform tasks that seemed impossible or required programming knowledge specific to a given area or were supposed to be impossible to be performed automatically. Our research explores whether an LLM-based system would be able to evaluate web accessibility success criteria that require manual evaluation. Three specific success criteria of the Web Content Accessibility Guidelines (WCAG) that currently require manual checks were tested: 1.1.1 Non-text Content, 2.4.4 Link Purpose (In Context), and 3.1.2 Language of Parts. LLM-based scripts were developed to evaluate the test cases. Results were compared against current web accessibility evaluators. While automated accessibility evaluators were unable to reliably test the three WCAG criteria, often missing or only warning about issues, the LLM-based scripts successfully identified accessibility issues the tools missed, achieving overall 87.18% detection across the test cases. Conclusion The results demonstrate LLMs can augment automated accessibility testing to catch issues that pure software testing misses today. Further research should expand evaluation across more test cases and types of content.
Over 2.5 billion people have at least one messaging app installed. In 2015, messaging apps have surpassed social networks. In this context, chat-bots or bots, are a new kind of applications that leverage these messaging ecosystem advantages. They need no installation effort, they are ubiquitous and they reside in students' most valuable gadgets, their smartphones. Armed with AI techniques, machine learning algorithms and natural language processing features, bots could be an ideal teacher assistant, able to answer and propose personalized questions to the students, anywhere, anytime. But before that near future arrives, there is a need for designing a new breed of mobile learning applications that take advantage of the ubiquity of these bots for everyday use. In this article we present the design and implementation of a Telegram bot, @dawebot, for training students in any subject using multiple choice question quizzes. We also present an evaluation of @dawebot in a 15 week long subject with 23 students of Computer Science, documenting their opinion about this bot in particular and the usage of bots in the elearning area in general. Students think that using bots for practicing tests is a good idea (89%), that using testing bots could help them to engage more in the subject (72%) and and will definitely recommend to use such testing bots also in other subjects of their university grade (94%).
31 October 2024 in Computer Standards & Interfaces