The development of connected and automated vehicles (CAVs) presents a great opportunity to extend the current range of vehicle vision, by gathering information outside of its sensors. Two main sources could be aggregated for this extended perception; vehicles making use of vehicle-to-vehicle communication (V2V), and infrastructure using vehicle-to-infrastructure communication (V2I). In this paper, we focus on the infrastructure side and make the case for low-latency obstacle mapping using V2I communication. A map management framework is proposed, which allows vehicles to broadcast and subscribe to traffic information-related messages using the Message Queuing Telemetry Transport (MQTT) protocol. This framework makes use of our novel candidate/employed map (C/EM) model for the real-time updating of obstacles broadcast by individual vehicles. This solution has been implemented and tested using a scenario that contains real and simulated CAVs tasked with doing lane change and braking maneuvers. As a result, the simulated vehicle can optimize its trajectory planning based on information which could not be observed by its sensor suite but is instead received from the presented map-management module, while remaining capable of performing the maneuvers in an automated manner.

3-dimensional, accurate, and up-to-date maps are essential for vehicles with autonomous capabilities, whose functionality is made possible by machine learning-based algorithms. Since these solutions require a tremendous amount of data for parameter optimization, simulation-to-reality (Sim2Real) methods have been proven immensely useful for training data generation. For creating realistic models to be used for synthetic data generation, crowdsourcing techniques present a resource-efficient alternative. In this paper, we show that using the Carla simulation environment, a crowdsourcing model can be created that mimics a multi-agent data gathering and processing pipeline. We developed a solution that yields dense point clouds based on monocular images and location information gathered by individual data acquisition vehicles. Our method provides scene reconstructions using the robust Structure-from-Motion (SfM) solution of Colmap. Moreover, we introduce a solution for synthesizing dense ground truth point clouds originating from the Carla simulator using a simulated data acquisition pipeline. We compare the results of the Colmap reconstruction with the reference point cloud after aligning them using the iterative closest point algorithm. Our results show that a precise point cloud reconstruction was feasible with this crowdsourcing-based approach, with 54\% of the reconstructed points having an error under 0.05 m, and a weighted root mean square error of 0.0449 m for the entire point cloud.

In this paper, a novel solution is introduced for visual Simultaneous Localization and Mapping (vSLAM) that is built up of Deep Learning components. The proposed architecture is a highly modular framework in which each component offers state of the art results in their respective fields of vision-based Deep Learning solutions. The paper shows that with the synergic integration of these individual building blocks, a functioning and efficient all-through deep neural (ATDN) vSLAM system can be created. The Embedding Distance Loss function is introduced and using it the ATDN architecture is trained. The resulting system managed to achieve 4.4% translation and 0.0176 deg/m rotational error on a subset of the KITTI dataset. The proposed architecture can be used for efficient and low-latency autonomous driving (AD) aiding database creation as well as a basis for autonomous vehicle (AV) control.

The problem of autonomous vehicle navigation requires the use of high-definition and well-maintained maps. Such a map can be constructed using the method developed for the so-called CrowdMapping architecture. This paper proposes a method for constructing masks for such map creation purposes via segmenting dynamic and static regions of an image sequence. The segmentation is performed by comparing a calculated and a predicted optical flow field. The proposed segmentation algorithm contains a single image depth estimation part for predicting the expected optical flow field. The comparison method of the two flow fields is also presented in this paper. The proposed method has been evaluated both qualitatively and quantitatively using the KITTI vision dataset, and achieved a filtering error of 7…12% compared to manually prepared ground truth images.

Neural network-based solutions have revolutionized the field of computer vision by achieving outstanding performance in a number of applications. Yet, while these deep learning models outclass previous methods, they still have significant shortcomings relating to generalization and robustness to input disturbances, such as occlusion. Most existing methods that tackle this latter problem use passive neural network architectures that are unable to act on and, thus, influence the observed scene. In this paper, we argue that an active observer agent may be able to achieve superior performance by changing the parameters of the scene, thus, avoiding occlusion by moving to a different position in the scene. To demonstrate this, a reinforcement learning environment is introduced that implements OpenAI Gym’s interface, and allows the creation of synthetic scenes with realistic occlusion. The environment is implemented using differentiable rendering, allowing us to perform direct gradient-based optimization of the camera position. Moreover, two additional methods are also presented, one utilizing self-supervised learning to predict occlusion segments, and optimal camera positions, while the other learns to avoid occlusion using Reinforcement Learning. We present comparative experiments of the proposed methods to demonstrate their efficiency. It was shown, via Bayesian t-tests, that the neural network-based methods credibly outperformed the gradient-based avoidance strategy by avoiding occlusion with an average of 5.0 fewer steps in multi-object scenes.