Safety validation of automated driving functions is a major challenge that is partly tackled by means of simulation-based testing. The virtual validation approach always entails the modeling of automotive perception sensors and their environment. In the real world, these sensors are exposed to adverse influences by environmental conditions such as rain, fog, snow, etc. Therefore, such influences need to be reflected in the simulation models. In this publication, a novel data set is introduced and analyzed. This data set contains lidar data with synchronized reference measurements of weather conditions from a stationary long-term experiment. Recorded weather conditions comprise fog, rain, snow, and direct sunlight. The data are analyzed by pairing lidar values, such as the number of detections in the atmosphere, with weather parameters such as rain rate in mm/h. This results in expectation values, which can directly be utilized for stochastic modeling or model calibration and validation. The results show vast differences in the number of atmospheric detections, range distribution, and attenuation between the different sensors of the data set.

Automated vehicles use light detection and ranging (LiDAR) sensors for environmental scanning. However, the relative motion between the scanning LiDAR sensor and objects leads to a distortion of the point cloud. This phenomenon is known as the motion distortion effect, significantly degrading the sensor’s object detection capabilities and generating false negative or false positive errors. In this work, we have introduced ray tracing-based deterministic and analytical approaches to model the motion distortion effect on the scanning LiDAR sensor’s performance for simulation-based testing. In addition, we have performed dynamic test drives at a proving ground to compare real LiDAR data with the motion distortion effect simulation data. The real-world scenarios, the environmental conditions, the digital twin of the scenery, and the object of interest (OOI) are replicated in the virtual environment of commercial software to obtain the synthetic LiDAR data. The real and the virtual test drives are compared frame by frame to validate the motion distortion effect modeling. The mean absolute percentage error (MAPE), the occupied cell ratio (OCR), and the Barons cross-correlation coefficient (BCC) are used to quantify the correlation between the virtual and the real LiDAR point cloud data. The results show that the deterministic approach matches the real measurements better than the analytical approach for the scenarios in which the yaw rate of the ego vehicle changes rapidly.

In virtual validation of automated driving, trustworthy simulation models of perception sensors are required. Radar sensors are particularly hard to model, as their measurements are notoriously difficult to interpret. This is due to their complex measurement principle, involving multi-path propagation of mm-waves, varying backscattering characteristics of objects, and further factors such as limited measurement ranges and resolutions that introduce uncertainty to the measurements. This work presents a method for studying the backscatter characteristics of vehicles under real-world driving conditions. A slalom-like driving scenario, which is representative for road-driving where the vehicle is visible under different aspect angles, has been designed. It aims at a high level of reproducibility of the trajectories driven by the vehicles, hence reducing additional sources of uncertainty that were otherwise present in the measurements. In a large-scale measurement campaign, 13 vehicles have been studied. The vehicles-under-test are observed by multiple radars, mounted at different heights, and carry reference sensors for obtaining their positions. In this paper, we present the measurement campaign and show major findings from our measurement results. Our focus lies on drawing conclusions for trustworthy sensor simulation. Both sensor measurement data, and MATLAB code for data analysis are made publicly available alongside this paper.

In the course of the development of automated driving, there has been increasing interest in obtaining ground truth information from sensor recordings and transferring road traffic scenarios to simulations. The quality of the “ground truth” annotation is dictated by its accuracy. This paper presents a method for calibrating the accuracy of ground truth in practical applications in the automotive context. With an exemplary measurement device, we show that the proclaimed accuracy of the device is not always reached. However, test repetitions show deviations, resulting in non-uniform reliability and limited trustworthiness of the reference measurement. A similar result can be observed when reproducing the trajectory in the simulation environment: the exact reproduction of the driven trajectory does not always succeed in the simulation environment shown as an example because deviations occur. This is particularly relevant for making sensor-specific features such as material reflectivities for lidar and radar quantifiable in dynamic cases.

Virtual validation of radar sensor models is becoming increasingly important for the safety validation of Light Detection and Rangings (lidars). Therefore, methods for quantitative comparison of radar measurements in the context of model validation need to be developed. This paper presents a novel methodology for accessing and quantifying validation measurements of radar sensor models. This method uses Light Detection and Rangings (lidars) and the so-called Double Validation Metric (DVM) to effectively quantify deviations between distributions. By applying this metric, the study measures the reproducibility and repeatability of radar sensor measurements. Different interfaces and different levels of detail are investigated. By comparing the radar signals from real-world experiments where different objects are present, valuable insights are gained into the performance of the sensor. In particular, the research extends to assessing the impact of varying rain intensities on the measurement results, providing a comprehensive understanding of the sensor’s behavior under these conditions. This holistic approach significantly advances the evaluation of radar sensor capabilities and enables the quantification of the maximum required quality of radar simulation models.

Virtual validation of radar sensor models is becoming increasingly important for the safety validation of adf. Therefore, methods for quantitative comparison of radar measurements in the context of model validation need to be developed. This paper presents a novel methodology for accessing and quantifying validation measurements of radar sensor models. This method uses edf and the so-called dvm to effectively quantify deviations between distributions. By applying this metric, the study measures the consistency, reproducibility and repeatability of radar sensor measurements. Different interfaces and different levels of detail are investigated. By comparing the radar signals from real world experiments where different objects are present, valuable insights are gained into the performance of the sensor. In particular, the research extends to assessing the impact of varying rain intensities on the measurement results, providing a comprehensive understanding of the sensor’s behaviour under these conditions. This holistic approach significantly advances the evaluation of radar sensor capabilities and enables the quantification of the maximum required quality of radar simulation models.