Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving.

For environmental perception, autonomous vehicles and intelligent roadside infrastructure systems contain multiple sensors, i.e. radar (Radio Detection and Ranging), lidar (Light Detection and Ranging), and camera sensors with the aim to detect, classify and track multiple road users. Data from multiple sensors are fused together to enhance the perception quality of the sensor system because each sensor has strengths and weaknesses, e.g. resolution, distance measurement, and dependency on weather conditions. For data fusion, it is necessary to transform the data from the different sensor coordinates to a common coordinate frame. This process is referred to as multi-sensor calibration and is a challenging task, which is mostly performed manually. This paper introduces a new method for auto-calibrating three-dimensional (3D) radar, 3D lidar, and red-green-blue (RGB) mono-camera sensors using a static multi-target-based system. The proposed method can be used with sensors operating at different frame rates without time synchronization. Furthermore, the described static multi-target system is cost-effective, easy to build, and applicable for short to long-distance calibration. The experimental results for multiple sets of measurements show good results with projection errors measured as maximum root mean square error (RMSE) of (u,v) = (2.4, 1.8) pixels for lidar-to-camera calibration, RMSE of (u,v) = (2.2, 3.0) pixels for 3D radar-to-camera calibration and RMSE of (x,y,z) = (2.6, 2.7, 14.0) centimeters for 3D radar-to-lidar calibration.

Single-board computers, such as Raspberry Pi, have enabled a wide variety of monitoring applications. Microcontroller development boards offer a similar functionality but at a lower cost. Still, they are not widely used, because they are supposed to be slow. This paper aims to evaluate the abilities of modern microcontrollers and compares them with those of single-board computers. For this purpose, experiments concerning power consumption and performance evaluation are conducted. The results suggest that promising low-cost microcontrollers exist, which could be applied to reduce cost and energy consumption per node or increase the number of nodes used simultaneously in one monitoring system.