Navigation assistance for visually impaired (NAVI) refers to systems that are able to assist or guide people with vision loss, ranging from partially sighted to totally blind, by means of sound commands. In this paper, a new system for NAVI is presented based on visual and range information. Instead of using several sensors, we choose one device, a consumer RGB-D camera, and take advantage of both range and visual information. In particular, the main contribution is the combination of depth information with image intensities, resulting in the robust expansion of the range-based floor segmentation. On one hand, depth information, which is reliable but limited to a short range, is enhanced with the long-range visual information. On the other hand, the difficult and prone-to-error image processing is eased and improved with depth information. The proposed system detects and classifies the main structural elements of the scene providing the user with obstacle-free paths in order to navigate safely across unknown scenarios. The proposed system has been tested on a wide variety of scenarios and data sets, giving successful results and showing that the system is robust and works in challenging indoor environments.

In this paper, we present a visual servo controller that effects optimal paths for a nonholonomic differential drive robot with field-of-view constraints imposed by the vision system. The control scheme relies on the computation of homographies between current and goal images, but unlike previous homography-based methods, it does not use the homography to compute estimates of pose parameters. Instead, the control laws are directly expressed in terms of individual entries in the homography matrix. In particular, we develop individual control laws for the three path classes that define the language of optimal paths: rotations, straight-line segments, and logarithmic spirals. These control laws, as well as the switching conditions that define how to sequence path segments, are defined in terms of the entries of homography matrices. The selection of the corresponding control law requires the homography decomposition before starting the navigation. We provide a controllability and stability analysis for our system and give experimental results.

Driving mobile robots to precise locations is of recognized interest, and using vision sensors in this context supplies many advantages. We propose a novel control law based on sliding-mode theory in order to drive mobile robots to a target location, which is specified by a previously acquired reference image. The control scheme exploits the piecewise epipolar geometry of three views on the basis of image-based visual servoing, in such a way that no 3-D scene information is required. The contribution of the paper is a new control law that achieves convergence to the target with no auxiliary images and without changing to any approach other than epipolar-based control. Additionally, the use of sliding-mode control deals with singularities, thus allowing the robot to move directly toward the target as well as avoiding the need of a precise camera calibration. The effectiveness of our approach is tested with simulations and real-world experiments.

Addressing the problem of enclosing and tracking a target requires multiple agents with adequate motion strategies. We consider a team of unicycle robots with a standard camera on board. The robots must maintain the desired enclosing formation while dealing with their nonholonomic motion constraints. The reference formation trajectories must also guarantee permanent visibility of the target by overcoming the limited field of view of the cameras. In this article, we present a novel approach to characterize the conditions on the robots’ trajectories taking into account the motion and visual constraints. We also propose online and offline motion planning strategies to address the constraints involved in the task of enclosing and tracking the target. These strategies are based on maintaining the formation shape with variable size or, alternatively, on maintaining the size of the formation with flexible shape.

A novel method to detect stairs with a RGB-D camera is proposed.We get the fully measured model with pose to validate and help in navigation.The system is designed to be wearable and aimed to assist the visually impaired.Visual odometry is computed to enhance the navigation system in video sequences.On-line stair measurements are used to correct the drift of the visual odometry. Stairs are one of the most common structures present in human-made scenarios, but also one of the most dangerous for those with vision problems. In this work we propose a complete method to detect, locate and parametrise stairs with a wearable RGB-D camera. Our algorithm uses the depth data to determine if the horizontal planes in the scene are valid steps of a staircase judging their dimensions and relative positions. As a result we obtain a scaled model of the staircase with the spatial location and orientation with respect to the subject. The visual odometry is also estimated to continuously recover the current position and orientation of the user while moving. This enhances the system giving the ability to come back to previously detected features and providing location awareness of the user during the climb. Simultaneously, the detection of the staircase during the traversal is used to correct the drift of the visual odometry. A comparison of results of the stair detection with other state-of-the-art algorithms was performed using public dataset. Additional experiments have also been carried out, recording our own natural scenes with a chest-mounted RGB-D camera in indoor scenarios. The algorithm is robust enough to work in real-time and even under partial occlusions of the stair.

In this paper, we propose a novel procedure for 3D layout recovery of indoor scenes from single 360 degrees panoramic images. With such images, all scene is seen at once, allowing to recover closed geometries. Our method combines strategically the accuracy provided by geometric reasoning (lines and vanishing points) with the higher level of data abstraction and pattern recognition achieved by deep learning techniques (edge and normal maps). Thus, we extract structural corners from which we generate layout hypotheses of the room assuming Manhattan world. The best layout model is selected, achieving good performance on both simple rooms (box-type) and complex shaped rooms (with more than four walls). Experiments of the proposed approach are conducted within two public datasets, SUN360 and Stanford (2D-3D-S) demonstrating the advantages of estimating layouts by combining geometry and deep learning and the effectiveness of our proposal with respect to the state of the art.