We present a software tool developed to help the visually impaired to perceive characteristics of the environment related to color and luminosity. The tool converts color information extracted from images into sound, where the images can be still images, or video frames from live or stored video. The color information is extracted from the hue, saturation and value attributes, which are mapped into audio attributes directly related to the perception of pitch, timbre and loudness. The tool can be used to gather information about the range of colors present in the images, presence or absence of light sources as well the location and shape of the objects in the images. It is useful for blind people, but it has also applications in other areas such as chemistry and arts

Background: Serious games (SGs) are used as complementary approaches to stimulate patients with dementia. However, many of the SGs use out-of-the-shelf technologies that may not always be suitable for such populations, as they can lead to negative behaviors, such as anxiety, fatigue, and even cybersickness. Objective: This study aims to evaluate how patients with dementia interact and accept 5 out-of-the-shelf technologies while completing 10 virtual reality tasks. Methods: A total of 12 participants diagnosed with dementia (mean age 75.08 [SD 8.07] years, mean Mini-Mental State Examination score 17.33 [SD 5.79], and mean schooling 5.55 [SD 3.30]) at a health care center in Portugal were invited to participate in this study. A within-subject experimental design was used to allow all participants to interact with all technologies, such as HTC VIVE, head-mounted display (HMD), tablet, mouse, augmented reality (AR), leap motion (LM), and a combination of HMD with LM. Participants’ performance was quantified through behavioral and verbal responses, which were captured through video recordings and written notes. Results: The findings of this study revealed that the user experience using technology was dependent on the patient profile; the patients had a better user experience when they use technologies with direct interaction configuration as opposed to indirect interaction configuration in terms of assistance required (P=.01) and comprehension (P=.01); the participants did not trigger any emotional responses when using any of the technologies; the participants’ performance was task-dependent; the most cost-effective technology was the mouse, whereas the least cost-effective was AR; and all the technologies, except for one (HMD with LM), were not exposed to external hazards. Conclusions: Most participants were able to perform tasks using out-of-the-shelf technologies. However, there is no perfect technology, as they are not explicitly designed to address the needs and skills of people with dementia. Here, we propose a set of guidelines that aim to help health professionals and engineers maximize user experience when using such technologies for the population with dementia.

Traditional speech therapy approaches for speech sound disorders have a lot of advantages to gain from computer-based therapy systems. In this paper, we propose a robust phoneme recognition solution for an interactive environment for speech therapy. With speech recognition techniques the motivation elements of computer-based therapy systems can be automated in order to get an interactive environment that motivates the therapy attendee towards better performances. The contribution of this paper is a robust phoneme recognition to control the feedback provided to the patient during a speech therapy session. We compare the results of hierarchical and flat classification, with naive Bayes, support vector machines and kernel density estimation on linear predictive coding coefficients and Mel-frequency cepstral coefficients.

This paper presents a statistical data-driven method for learning intrinsic structures of impact sounds. The method applies principal and independent component analysis to learn low-dimensional representations that model the distribution of both the time-varying spectral and amplitude structure. As a result, the method is able to decompose sounds into a small number of underlying features that characterize acoustic properties such as ringing, resonance, sustain, decay, and onsets. The method is highly flexible and makes no a priori assumptions about the physics, acoustics, or dynamics of the objects. In addition, by modeling the underlying distribution, the method can capture the natural variability of ensembles of related impact sounds.

Speech is the main form of human communication. Thus it is important to detect and treat speech sound disorders as early as possible during childhood. When children need to attend speech therapy it is critical to keep them motivated on doing the therapy exercises. Software systems for speech therapy can be a useful tool to keep the child interested in keep practicing the therapy exercises. Several software systems have been developed to assist speech and language therapists during the therapy sessions. However most software focus on articulation disorders while voice disorders have been mostly neglected. Here we propose a voice-controlled serious computer game for the sustained vowel exercise, which is an exercise commonly used in speech therapy to treat voice disorders. The main novelty of this application is the combination of real time speech processing, with the gamification of the speech therapy exercises and the parameterization of the difficulty level.