In recent years, rapid advancements have taken place for automatic speech recognition (ASR) systems and devices. Though ASR technologies have increased, the accessibility of these novel interaction systems is underreported and may present difficulties for people with speech impediments. In this article, we attempt to identify gaps in current research on the interaction between people with dysarthria and ASR systems and devices. We cover the period from 2011, when Siri (the first and the leading commercial voice assistant) was launched, to 2020. The review employs an interaction framework in which each element (user, input, system, and output) contributes to the interaction process. To select the articles for review, we conducted a search of scientific databases and academic journals. A total of 36 studies met the inclusion criteria, which included use of the word error rate (WER) as a measurement for evaluating ASR systems. This review determines that challenges in interacting with ASR systems persist even in light of the most recent commercial technologies. Further, understanding of the entire interaction process remains limited; thus, to improve this interaction, the recent progress of ASR systems must be elucidated.

People with speech impairments usually use assistive technology devices to assist them with communication and daily tasks. These devices can be controlled using different modalities, such as touch, eye gaze, gestures, and others. This article proposes a standardized methodology for designing non-verbal voice cue interactive systems that enable people with dysarthria to vocally interact with virtual home assistants (VHAs). We adopted a qualitative data-gathering approach to gain insights into users’ experiences and requirements and to determine crucial design elements for designing interactive voice assistants for people with dysarthria. Nineteen participants with varying levels of dysarthria took part in the study to create the framework. A system was then built using the proposed framework, and an additional test was performed with a further seven participants to validate the created system, thus inferring the validity of the framework. Our work empirically demonstrates how an informed, structured design of a fast, direct (verbal rather than forcing users to change modalities or using an intermediate device) method of communication improves the usability of VHAs for people with dysarthria while simultaneously allowing for a more authentic experience. The current data also highlighted that using non-verbal voice cues would be a convenient option. By providing a reproducible framework for developing non-verbal interactive systems for VHAs, we can increase the accessibility of said devices, their usability, and their user experience.

This study explores the effectiveness and user experience of different interaction methods used by individuals with dysarthria when engaging with Smart Virtual Assistants (SVAs). It focuses on three primary modalities: direct speech commands through Alexa, non-verbal voice cues via the Daria system, and eye gaze control. The objective is to assess the usability, workload, and user preferences associated with each method, catering to the varying communication capabilities of individuals with dysarthria. While Alexa and Daria facilitate voice-based interactions, eye gaze control offers an alternative for those unable to use voice commands, including users with severe dysarthria. This comparative approach aims to determine how the usability of each interaction method varies, conducted with eight participants with dysarthria. The results indicated that non-verbal voice interactions, particularly with the Daria system, were favored because of their lower workload and ease of use. The eye gaze technology, while viable, presented challenges in terms of the higher workload and usability. These findings highlight the necessity of diversifying interaction methods with SVAs to accommodate the unique needs of individuals with dysarthria.

Inclusive user experience design that caters to a broad range of users can enhance the ease of interaction with systems. Despite progress, certain systems like smart voice assistants often remain inaccessible to some user types, particularly individuals with dysarthria whose speech may not be recognized by these devices. In this article, we present a novel interaction technique that relies on non-verbal voice cues instead of words and sentences. To evaluate the usability, user experience, and workload of this approach, we developed a system (DARIA) and conducted two studies, exploring two different design options. The first study tested the system without a customization option, while the second introduced a customization option. We used three questionnaires and conducted post-study interviews for thorough evaluation. The findings indicated user acceptance of this interaction technique in general was high. When compared with a market leader product (Alexa), DARIA showed higher performance in terms of accessibility. Users found the non-verbal system to be more usable and likable. Interestingly, the customizable design was found less usable than the pre-defined.