In this paper, we are presenting a short overview of the sensors and sensor fusion in autonomous vehicles. We focused on the sensor fusion from the key sensors in autonomous vehicles: camera, radar, and lidar. The current state-of-the-art in this area will be presented, such as 3D object detection method for leveraging both image and 3D point cloud information, moving object detection and tracking system, and occupancy grid mapping used for navigation and localization in dynamic environments. It is shown that including more sensors into sensor fusion system benefits with better performance and the robustness of the solution. Moreover, usage of camera data in localization and mapping, that is traditionally solved by radar and lidar data, improves the perceived model of the environment. Sensor fusion has a crucial role in autonomous systems overall, therefore this is one of the fastest developing areas in the autonomous vehicles domain.

In this paper, one solution for an end-to-end deep neural network for autonomous driving is presented. The main objective of our work was to achieve autonomous driving with a light deep neural network suitable for deployment on embedded automotive platforms. There are several end-to-end deep neural networks used for autonomous driving, where the input to the machine learning algorithm are camera images and the output is the steering angle prediction, but those convolutional neural networks are significantly more complex than the network architecture we are proposing. The network architecture, computational complexity, and performance evaluation during autonomous driving using our network are compared with two other convolutional neural networks that we re-implemented with the aim to have an objective evaluation of the proposed network. The trained model of the proposed network is four times smaller than the PilotNet model and about 250 times smaller than AlexNet model. While complexity and size of the novel network are reduced in comparison to other models, which leads to lower latency and higher frame rate during inference, our network maintained the performance, achieving successful autonomous driving with similar efficiency compared to autonomous driving using two other models. Moreover, the proposed deep neural network downsized the needs for real-time inference hardware in terms of computational power, cost, and size.

In this paper, one solution for driver behavioral cloning using deep learning is presented. The aim of this paper is to achieve autonomous driving in simulated conditions. In the simulator, images obtained from a camera mounted in the vehicle mimics the driver's vision, and then the reaction, control of the vehicle. Based on the images from the cameras gained in the manual mode, the neural network is trained using deep learning. Using this trained deep neural network leads to autonomous driving. The driver behavioral cloning algorithm, developed and described in this paper, is based on the NVIDIA convolutional neural network model.

A calibration platform for geometric calibration of multi-sensor image fusion system is presented in this paper. The accurate geometric calibration of the extrinsic geometric parameters of cameras that uses planar calibration pattern is applied. For calibration procedure specific software is made. Patterns used in geometric calibration are prepared with aim to obtain maximum contrast in both visible and infrared spectral range - using chessboards which fields are made of different emissivity materials. Experiments were held in both indoor and outdoor scenarios. Important results of geometric calibration for multi-sensor image fusion system are extrinsic parameters in form of homography matrices used for homography transformation of the object plane to the image plane. For each camera a corresponding homography matrix is calculated. These matrices can be used for image registration of images from thermal and low light camera. We implemented such image registration algorithm to confirm accuracy of geometric calibration procedure in multi-sensor image fusion system. Results are given for selected patterns - chessboard with fields made of different emissivity materials. For the final image registration algorithm in surveillance system for object tracking we have chosen multi-resolution image registration algorithm which naturally combines with a pyramidal fusion scheme. The image pyramids which are generated at each time step of image registration algorithm may be reused at the fusion stage so that overall number of calculations that must be performed is greatly reduced.

In this paper a generic, configurable solution for an Electronic Control Unit (ECU) simulator is described. The ECU simulator is a component of an Automated Testing System that aims at testing and verifying new devices for vehicle diagnostics. A vehicle has many electronic control units that are communicating over various automotive protocols. The vehicle diagnostic device must be able to communicate with all the ECUs. One of the key components in the automated test system is the ECU simulator that is able to simulate any of the ECUs, communicating with various automotive protocols, simulating any possible state. In this paper, the solution for the challenging task of designing and implementing the ECU simulator is presented.

The implementation of a sensor board of a high resolution camera is presented in this paper. Sensor board performs an acquisition of image sensor data using high speed serial interface HiSPi, then converts HiSPi interface to parallel interface using FPGA component and sends the data to a digital image processor. The advantage of this solution is that the high speed serial pixel interface is used, which enables faster data transmission. On the other hand, the standard image processor needs logic to translate to a parallel interface. The importance of presented implementation is FPGA conversion of the HiSPi interface to a parallel format, which is a simple, cost-effective programmable bridge between high-speed sensors and image signal processors.