This paper presents a computer vision based sensory substitution device for the visually impaired. Its main objective is to provide the users with a 3D representation of the environment around them, conveyed by means of the hearing and tactile senses. One of the biggest challenges for this system is to ensure pervasiveness, i.e., to be usable in any indoor or outdoor environments and in any illumination conditions. This work reveals both the hardware (3D acquisition system) and software (3D processing pipeline) used for developing this sensory substitution device and provides insight on its exploitation in various scenarios. Preliminary experiments with blind users revealed good usability results and provided valuable feedback for system improvement.

The development of computer vision based systems dedicated to help visually impaired people to perceive the environment, to orientate and navigate has been the main research subject of many works in the recent years. A significant ensemble of resources has been employed to support the development of sensory substitution devices (SSDs) and electronic travel aids for the rehabilitation of the visually impaired. The Sound of Vision (SoV) project used a comprehensive approach to develop such an SSD, tackling all the challenging aspects that so far restrained the large scale adoption of such systems by the intended audience: Wearability, real-time operation, pervasiveness, usability, cost. This article is set to present the artificial vision based component of the SoV SSD that performs the scene reconstruction and segmentation in outdoor environments. In contrast with the indoor use case, where the system acquires depth input from a structured light camera, in outdoors SoV relies on stereo vision to detect the elements of interest and provide an audio and/or haptic representation of the environment to the user. Our stereo-based method is designed to work with wearable acquisition devices and still provide a real-time, reliable description of the scene in the context of unreliable depth input from the stereo correspondence and of the complex 6 DOF motion of the head-worn camera. We quantitatively evaluate our approach on a custom benchmarking dataset acquired with SoV cameras and provide the highlights of the usability evaluation with visually impaired users.

The visual servoing architectures with an eye-in-hand configuration, composed of a 6-d.o.f. manipulator robot and a video camera, are categorised as high-coupled and nonlinear systems. For this reason, in order to have a stable and convergent behaviour, advanced techniques like predictive control are needed. These types of techniques can solve the potential problems that can appear in classical (image-based visual servoing) while different type of constraints for visual servoing system can be fulfilled easier. Three different visual predictive control architectures based on image moments are proposed. The model for the image moments predictor used in the first architecture is an extension of a local model point features predictor. The second predictive approach uses an image moment-based predictor to describe the future behaviour of the system over a certain camera velocity scenario directly in the image moments space. In order to have a faster convergence and in the same time a desired behaviour of the servoing system, a reference trajectory is employed. This represents the main contribution of the last predictive control technique proposed in this study. A Matlab simulator was developed and all the visual servoing architectures described in the study were implemented. The simulation results showed that all the proposed visual predictive control approaches are stable and convergent when dealing with visual servoing tasks, but the best performances were assigned to the reference trajectory-based image moments architecture.

The rapid advancement of autonomous vehicle technology has brought into focus the critical need for enhanced road safety systems, particularly in the areas of road damage detection and surface classification. This paper explores these two essential components, highlighting their importance in autonomous driving. In the domain of road damage detection, this study explores a range of deep learning methods, particularly focusing on one-stage and two-stage detectors. These methodologies, including notable ones like YOLO and SSD for one-stage detection and Faster R-CNN for two-stage detection, are critically analyzed for their efficacy in identifying various road damages under diverse conditions. The review provides insights into their comparative advantages, balancing between real-time processing and accuracy in damage localization. For road surface classification, the paper investigates the classification techniques based on both environmental conditions and material road composition. It highlights the role of different convolutional neural network architectures and innovations at the neural level in enhancing classification accuracy under varying road and weather conditions. The main finding of this work is that it offers a comprehensive overview of the current state of the art, showcasing significant strides in utilizing deep learning for road analysis in autonomous vehicle systems. The study concludes by underscoring the importance of continued research in these areas to further refine and improve the safety and efficiency of autonomous driving.