The ability to sense and understanding the driving environment is a key technology for ADAS and autonomous driving. Human drivers have to pay more visual attention to important or target elements and ignore unnecessary ones present in their field of sight. A model that computes this visual attention of targets in a specific driving environment is essential and useful in supporting autonomous driving, object-specific tracking & detection, driving training, car collision warning, traffic sign detection, etc. In this paper, we propose a new framework of visual attention that can predict important objects in the driving scene using a conditional generative adversarial network. A large scale Visual Attention Driving Database (VADD) of saliency heat-maps is built from existing driving datasets using a saliency mechanism. The proposed framework model takes its strength from these saliency heat-maps as conditioning label variables. The results show that the proposed approach makes us able to predict heat-maps of most important objects in a driving environment.

Traffic signs play an important role for Advanced Driver Assistance Systems (ADAS) as well as for autonomous driving vehicles. Most of the works done focus on recognizing symbol based signs leaving apart important information provided by other type of signs like complementary panels or text based signs. In this paper, we include detection and classification of both symbol and text based signs focusing on the most common ones found in European urban environments. The system consists of three stages, traffic sign detection, refinement and classification. The detection and refinement is performed using Mask R-CNN while the classification is achieved with a proposed Convolutional Neural Network (CNN) architecture. We introduced the extended version of the German Traffic Sign Detection Benchmark (GTSDB), labeled in a pixel manner (masks) with 164 classes grouped into 8 categories. It is used for the detection and classification steps. Experimental results on German environments show that our proposed system is capable of detecting all categories of traffic signs while at the same time recognizing them with high accuracy achieving comparable performance with the state of the art.

A critical concern of autonomous vehicles is safety. Different approaches have tried to enhance driving safety to reduce the number of fatal crashes and severe injuries. As an example, Intelligent Speed Adaptation (ISA) systems warn the driver when the vehicle exceeds the recommended speed limit. However, these systems only take into account fixed speed limits without considering factors like road geometry. In this paper, we consider road curvature with speed limits to automatically adjust vehicle’s speed with the ideal one through our proposed Dynamic Speed Adaptation (DSA) method. Furthermore, ‘curve analysis extraction’ and ‘speed limits database creation’ are also part of our contribution. An algorithm that analyzes GPS information off-line identifies high curvature segments and estimates the speed for each curve. The speed limit database contains information about the different speed limit zones for each traveled path. Our DSA senses speed limits and curves of the road using GPS information and ensures smooth speed transitions between current and ideal speeds. Through experimental simulations with different control algorithms on real and simulated datasets, we prove that our method is able to significantly reduce lateral errors on sharp curves, to respect speed limits and consequently increase safety and comfort for the passenger.

In this paper, we extend the discriminant correlation filter (DCF) based deep learning tracker to multi-object tracking. For each object, we use an individual tracker to estimate the position. Two different pre-trained networks are used as feature ex

A precise GNSS (Global Navigation Satellite System) localization is vital for autonomous road vehicles, especially in cluttered or urban environments where satellites are occluded, preventing accurate positioning. We propose to fuse GPS (Global Positioning System) data with fisheye stereovision to face this problem independently to additional data, possibly outdated, unavailable, and needing correlation with reality. Our stereoscope is sky-facing with 360° × 180° fisheye cameras to observe surrounding obstacles. We propose a 3D modelling and plane extraction through following steps: stereoscope self-calibration for decalibration robustness, stereo matching considering neighbours epipolar curves to compute 3D, and robust plane fitting based on generated cartography and Hough transform. We use these 3D data with GPS raw data to estimate NLOS (Non Line Of Sight) reflected signals pseudorange delay. We exploit extracted planes to build a visibility mask for NLOS detection. A simplified 3D canyon model allows to compute reflections pseudorange delays. In the end, GPS positioning is computed considering corrected pseudoranges. With experimentations on real fixed scenes, we show generated 3D models reaching metric accuracy and improvement of horizontal GPS positioning accuracy by more than 50%. The proposed procedure is effective, and the proposed NLOS detection outperforms CN0-based methods (Carrier-to-receiver Noise density).

Occupancy grid map is a popular tool for representing the surrounding environments of mobile robots/intelligent vehicles. Its applications can be dated back to the 1980s, when researchers utilized sonar or LiDAR to illustrate environments by occupancy grids. However, in the literature, research on vision-based occupancy grid mapping is scant. Furthermore, when moving in a real dynamic world, traditional occupancy grid mapping is required not only with the ability to detect occupied areas, but also with the capability to understand dynamic environments. The paper addresses this issue by presenting a stereo-vision-based framework to create a dynamic occupancy grid map, which is applied in an intelligent vehicle driving in an urban scenario. Besides representing the surroundings as occupancy grids, dynamic occupancy grid mapping could provide the motion information of the grids. The proposed framework consists of two components. The first is motion estimation for the moving vehicle itself and independent moving objects. The second is dynamic occupancy grid mapping, which is based on the estimated motion information and the dense disparity map. The main benefit of the proposed framework is the ability of mapping occupied areas and moving objects at the same time. This is very practical in real applications. The proposed method is evaluated using real data acquired by our intelligent vehicle platform “SeTCar” in urban environments.