As virtual reality (VR) is typically designed in terms of visual experience, it poses major challenges for blind people to understand and interact with the environment. To address this, we propose a design space to explore how to augment objects and their behaviours in VR with a nonvisual audio representation. It intends to support designers in creating accessible experiences by explicitly considering alternative representations to visual feedback. To demonstrate its potential, we recruited 16 blind users and explored the design space under two scenarios in the context of boxing: understanding the location of objects (the opponent's defensive stance) and their movement (opponent's punches). We found that the design space enables the exploration of multiple engaging approaches for the auditory representation of virtual objects. Our findings depicted shared preferences but no one-size-fits-all solution, suggesting the need to understand the consequences of each design choice and their impact on the individual user experience.
A number of studies have been conducted to improve the accessibility of images using touchscreen devices for screen reader users. In this study, we conducted a systematic review of 33 papers to get a holistic understanding of existing approaches and to suggest a research road map given identified gaps. As a result, we identified types of images, visual information, input device and feedback modalities that were studied for improving image accessibility using touchscreen devices. Findings also revealed that there is little study how the generation of image-related information can be automated. Moreover, we confirmed that the involvement of screen reader users is mostly limited to evaluations, while input from target users during the design process is particularly important for the development of assistive technologies. Then we introduce two of our recent studies on the accessibility of artwork and comics, AccessArt and AccessComics, respectively. Based on the identified key challenges, we suggest a research agenda for improving image accessibility for screen reader users.
While clinical notes are essential to the field of healthcare, they pose several challenges for clinicians since it is difficult to write down medical information, review prior notes, and extract the desired information at the same time while examining a patient. Thus, we designed a system that can automatically generate clinical notes from dialogues between patients and clinicians and provide specific information upon clinicians’ query using a Large Language Model (LLM) both in real-time. To explore how this system can be used to support clinicians in practice, we conducted an interview with six clinicians followed by a design probe study with the current version of our system for feedback. Findings suggest that our system has the potential to enable clinicians to write and access clinical notes and examine the patients simultaneously with reduced cognitive loads and increased efficiency and accuracy.
29 March 2023 in ACM Transactions on Accessible Computing
For people with visual impairments, many studies have been conducted to improve the accessibility of various types of images on the web. However, the majority of the work focused on photos or graphs. In this study, we propose AccessComics, an accessible digital comic book reader for people with visual impairments. To understand the accessibility of existing platforms, we first conducted a formative online survey with 68 participants who are blind or have low vision asking about their prior experiences with audiobooks and eBooks. Then, to learn the implications of designing an accessible comic book reader for people with visual impairments, we conducted an interview study with eight participants and collected feedback about our system. Considering our findings that a brief description of the scene and sound effects are desired when listening to comic books, we conducted a follow-up study with 16 participants (8 blind, 8 sighted) to explore how to effectively provide scene descriptions and sound effects, generated based on the onomatopoeia and mimetic words that appear in comics. Then we assessed the impact of the overall reading experience and if it differs depending on the user group. The results show that the presence of scene descriptions was perceived to be useful for concentration and understanding the situation, while the sound effects were perceived to make the book-reading experience more immersive and realistic. Based on the findings, we suggest design implications specifying features that future accessible comic book readers should support.
Encountered-type haptic rendering provides realistic, free-to-touch, and move-and-collide haptic sensation to a user. However, inducing haptic-texture sensation without complicated tactile actuators is challenging for encountered-type haptic rendering. In this paper, we propose a novel texture synthesizing method for an encountered-type haptic display using spatial and temporal encoding of roughness, which provides both active and passive touch sensation requiring no complicated tactile actuation. Focused on macro-scale roughness perception, we geometrically model the textured surface with a grid of hemiellipsoidal bumps, which can provide a variety of perceived roughness as the user explores the surface with one's bare hand. Our texture synthesis method is based on two important hypotheses. First, we assume that perceptual roughness can be spatially encoded along the radial direction of a textured surface with hemiellipsoidal bumps. Second, perceptual roughness temporally varies with the relative velocity of a scanning human hand with respect to the surface. To validate these hypotheses on our spatiotemporal encoding method, we implemented an encountered-type haptic texture rendering system using an off-the-shelf collaborative robot that can also track the user's hand using IR sensors.