User experience (UX) is a quality aspect that considers the emotions evoked by the system, extending the usability concept beyond effectiveness, efficiency, and satisfaction. Practitioners and researchers are aware of the importance of evaluating UX. Thus, UX evaluation is a growing field with diverse approaches. Despite various approaches, most of them produce a general indication of the experience as a result and do not seek to capture the problem that gave rise to the bad UX. This information makes it difficult to obtain relevant results to improve the application, making it challenging to identify what caused a negative user experience. To address this gap, we developed a UX evaluation technique called UX-Tips. This paper presents UX-Tips and reports two empirical studies performed in an academic and an industrial setting to evaluate it. Our results show that UX-Tips had good performance in terms of efficiency and effectiveness, making it possible to identify the causes that led to a negative user experience, and it was easy to use. In this sense, we present a new technique suitable for use in both academic and industrial settings, allowing UX evaluation and finding the problems that may lead to a negative experience.
The success of a software application is related to users’ willingness to keep using it. In this sense, evaluating User eXperience (UX) became an important part of the software development process. Researchers have been carrying out studies by employing various methods to evaluate the UX of software products. Some studies reported varied and even contradictory results when applying different UX evaluation methods, making it difficult for practitioners to identify which results to rely upon. However, these works did not evaluate the developers’ perspectives and their impacts on the decision process. Moreover, such studies focused on one-shot evaluations, which cannot assess whether the methods provide the same big picture of the experience (i.e., deteriorating, improving, or stable). This paper presents a longitudinal study in which 68 students evaluated the UX of an online judge system by employing AttrakDiff, UEQ, and Sentence Completion methods at three moments along a semester. This study reveals contrasting results between the methods, which affected developers’ decisions and interpretations. With this work, we intend to draw the HCI community’s attention to the contrast between different UX evaluation methods and the impact of their outcomes in the software development process.
1 January 2019 in Revista Brasileira de Informática na Educação
As technology evolves, learning institutions have sought to invest in new learning technologies. Many universities have been adopting platforms designed to support the learning process, called Learning Management Systems (LMSs), to complement face-to-face learning, as well as to support distance learning. This widespread adoption of LMSs raises the need for evaluating them regarding aspects related to quality in use, such as User eXperience (UX). UX is very important in e-learning context, given that it may influence learner’s attitude towards the use of the LMS. Despite its importance, few studies evaluated the UX of LMSs. Moreover, researchers have been using generic techniques to evaluate them. In this context, there is no evidence of studies conducted to identify whether these techniques can fully capture the experience conveyed by these platforms. This paper presents an analysis of two UX evaluation techniques (User Experience Questionnaire, and Integrated Experience Acceptance Model) applied to an LMS called Edmodo. We carried out a study with 34 students, divided into two groups, to evaluate the UX of Edmodo and obtain their perceptions about the technique they used. We also gathered the difficulties faced by the students during the execution of the tasks on the platform, while correlating those difficulties with learning issues. In addition, we performed a qualitative analysis in order to better understand the results. Regarding Edmodo, the results showed that the students perceived its UX as positive. Regarding the techniques, both groups considered them easy to use and useful. However, around 41% of the students in both groups reported that they were not able to fully evaluate their UX using the techniques. The qualitative results allowed the identification of positive and negative aspects that researchers may consider during the improvement or development of new UX evaluation techniques.
27 October 2017 in Simpósio Brasileiro de Informática na Educação
As technology evolves and new interaction patterns emerge, User eXperience (UX) is becoming more important. Although several UX evaluation techniques have been proposed, there is no enough evidence regarding their adequacy in the e-learning context. This paper presents an analysis of two UX evaluation techniques applied to the Edmodo Learning Management System. We performed a study with 34 students to evaluate the UX of Edmodo and the perceptions of the techniques. Results showed that Edmodo provided a positive UX. However, around 41% of the students reported that they were not able to fully evaluate their UX using the techniques, indicating a need for improvement.