Convolutional neural networks (CNNs) have been used successfully in solving many challenging visual perception tasks facing mobile robots and self-driving cars. To facilitate deploying of such models on embedded hardware onboard mobile robots which has limited resources, multi-task learning approaches have become common. In typically used multi-task learning, a shared encoder network extracts features from inputs whereas multiple task-specific decoders transform these features into their target output. However, properly combining different tasks' losses into the final network loss such that each task is making progress learning is a major challenge in these approaches. In this paper, we present an innovative approach to extend a typical single-task network with the capability of performing two tasks without multiple decoders. That is, a single-stream two-task network. The two output tasks are semantic segmentation and monocular depth prediction which are essential tasks in visual perception for autonomous driving. The method is centered on solving semantic segmentation with a regression loss function rather than a classification one. With our approach, we seize multi-task learning benefits of reduced overhead and enhanced generalization while alleviating the need to balance different loss functions. Experimental evaluations with baseline single tasks and a multi-task network are presented.

Vision-based motion estimation is an effective means for mobile robot localization and is often used in conjunction with other sensors for navigation and path planning. This paper presents a low-overhead real-time ego-motion estimation (visual odometry) system based on either a stereo or RGB-D sensor. The algorithm’s accuracy outperforms typical frame-to-frame approaches by maintaining a limited local map, while requiring significantly less memory and computational power in contrast to using global maps common in full visual SLAM methods. The algorithm is evaluated on common publicly available datasets that span different use-cases and performance is compared to other comparable open-source systems in terms of accuracy, frame rate and memory requirements. This paper accompanies the release of the source code as a modular software package for the robotics community compatible with the Robot Operating System (ROS).

Detection and tracking of different moving objects in the surrounding environment is an essential capability for any robotic perception system. This enables forecasting of object trajectories and safe navigation of a mobile robot, such as a self-driving car, toward its goal. In this paper, we demonstrate a multi-object tracker from image data in a tracking-by-detection paradigm. The tracking algorithm imposes no prior requirements on the tracked objects. This paper demonstrates how with a simple and intuitive cost function, competitive and real-time performance is achieved. Furthermore, the multi-object tracker is combined with a stereo visual odometry system to obtain a more complete knowledge of the environment. Beside 2D object detections and tracking in the image domain, this provides robot motion estimates and 3D object trajectories in the world frame. Evaluation and discussion of the system is presented along with a study on the effect of removing features lying on tracked objects on visual odometry performance.

Cameras are valuable sensors for robotics perception tasks. Among these perception tasks are motion estimation, localization, and object detection. Cameras are attractive sensors because they are passive and relatively cheap and can provide rich information. However, being passive sensors, they rely on external illumination from the environment which means that their performance degrades in low-light conditions. In this paper, we present and investigate four methods to enhance images under challenging night conditions. The findings are relevant to a wide range of feature-based vision systems, such as tracking for augmented reality, image registration, localization, and mapping, as well as deep learning-based object detectors. As autonomous mobile robots are expected to operate under low-illumination conditions at night, evaluation is based on state-of-the-art systems for motion estimation, localization, and object detection.