A novel visual and infrared sensor data-based system to assist visually impaired users in detecting obstacles in their path while independently navigating indoors is presented. The system has been developed for the recently introduced Google Project Tango Tablet Development Kit equipped with a powerful graphics processor and several sensors which allow it to track its motion and orientation in 3D space in real-time. It exploits the inbuilt functionalities of the Unity engine in the Tango SDK to create a 3D reconstruction of the surrounding environment, then associates a Unity collider component with the user and utilizes it to determine his interaction with the reconstructed mesh in order to detect obstacles. The user is provided with audio feedback consisting of obstacle warnings. An extensive empirical evaluation of the obstacle detection component has yielded favorable results, thus, confirming the potential of this system for future development work.

The development of a novel depth-data based real-time obstacle detection and avoidance application for visually impaired (VI) individuals to assist them in navigating independently in indoors environments is presented in this paper. The application utilizes a mainstream, computationally efficient mobile device as the development platform in order to create a solution which not only is aesthetically appealing, cost-effective, lightweight and portable but also provides real-time performance and freedom from network connectivity constraints. To alleviate usability problems, a user-centered design approach has been adopted wherein semi-structured interviews with VI individuals in the local context were conducted to understand their micro-navigation practices, challenges and needs. The invaluable insights gained from these interviews have not only informed the design of our system but would also benefit other researchers developing similar applications. The resulting system design along with a detailed description of its obstacle detection and unique multimodal feedback generation modules has been provided. We plan to iteratively develop and test the initial prototype of the system with the end users to resolve any usability issues and better adapt it to their needs.

A novel, low-cost, Arabic language tangible user interface (TUI)-based solution for teaching and reinforcing the concepts of planar shapes and their relationships to visually impaired (VI) children is presented. The solution employs a computer vision-based system to track tagged 3D-printed objects which children can manipulate to receive audio instructions and feedback in the context of various activities for learning progressively difficult shape concepts in a systematic manner. A graphical user interface (GUI) is also provided for the teachers to create and manage student accounts, track their students’ individual progress and download and print the templates and models of the tangible tokens along with their barcodes. The aim is to provide an engaging, accessible solution that would mitigate the demands on the time, efforts and financial resources of the teachers while allowing children to autonomously learn and review shape concepts at their own pace. By utilizing off-the-shelf components for setting up the TUI and providing templates and CAD models for generating the tangible tokens, we have endeavored to make the system affordable and easily reproducible. The system has been developed in consultation with teachers for VI children to ensure its compliance with current teaching and classroom practices. Initial evaluations of the system with VI children and teachers have validated its potential and yielded valuable design recommendations which would not only inform the design of future iterations of our system but also benefit other researchers seeking to build similar systems.