Perception sensors such as camera, radar, and lidar have gained considerable popularity in the automotive industry in recent years. In order to reach the next step towards automated driving it is necessary to implement fault diagnosis systems together with suitable mitigation solutions in automotive perception sensors. This is a crucial prerequisite, since the quality of an automated driving function strongly depends on the reliability of the perception data, especially under adverse conditions. This publication presents a systematic review on faults and suitable detection and recovery methods for automotive perception sensors and suggests a corresponding classification schema. A systematic literature analysis has been performed with focus on lidar in order to review the state-of-the-art and identify promising research opportunities. Faults related to adverse weather conditions have been studied the most, but often without providing suitable recovery methods. Issues related to sensor attachment and mechanical damage of the sensor cover were studied very little and provide opportunities for future research. Algorithms, which use the data stream of a single sensor, proofed to be a viable solution for both fault detection and recovery.

Sensor models are essential for virtual testing of Advanced Driver Assistance Systems/Autonomous Driving (ADAS/AD) functions. This article gives an overview of the state-of-the-art of ADAS/AD sensor models. The considered sensors are radar, lidar, and camera. To get a common understanding and a common language in sensor model research, a new classification method into low-, medium-, and high-fidelity sensor models is introduced. Low-fidelity sensor models are based on geometrical aspects like the Field Of View (FOV) of the sensor and object positions in the virtual environment. Object lists are used as input and output data formats. Medium-fidelity sensor models consider the detection probability and physical aspects in addition to geometrical aspects of the sensor. They have object lists as input and object lists or raw data as output. High-fidelity sensor models are based on rendering techniques. They have the virtual three-dimensional (3D) environment provided by the environment simulation as an input and sensor raw data as an output. The classification is useful for virtual testing of ADAS/AD functions since the classes can be correlated to the phases of the Systems Development Process (SDP) of ADAS/AD.

Lidar is an important component of the perception suite for automated systems. The effects of vibration on lidar point clouds are mostly unknown, despite the lidar’s wide adaption and usual application under conditions where vibration occurs frequently. In this study, we performed controlled vibration tests from 6 to 2000 Hz at 9 and 12 m/s ² in vertical direction on the automotive lidar OS1-64 by Ouster. An information loss emerged which is mostly independent from frequency and acceleration. The loss of points is randomly distributed and does not correlate with range, intensity, or ring number (the horizontal line of the rotating lidar unit). The resonance frequency of 1426 Hz proved to be unproblematic as no pronounced negative effects on the point cloud could be identified. For vibration detection, the internal Inertial Measurement Unit (IMU) of the OS1-64 is accurate and sufficient for vibrations up to 50 Hz. Above 50 Hz, external IMUs would be required for vibration detection. Counting the number of points on a target close to the edges was investigated as an exemplary way to detect vibration purely based on the point cloud, i.e., independent of the lidar’s IMU.

Lidar sensors play an essential role in the perception system of automated vehicles. Fault Detection, Isolation, Identification, and Recovery (FDIIR) systems are essential for increasing the reliability of lidar sensors. Knowing the influence of different faults on lidar data is the first crucial step towards fault detection for lidar sensors in automated vehicles. We investigate the influences of sensor cover contaminations on the output data, i.e., on the lidar point cloud and full waveform. Different contamination types were applied (dew, dirt, artificial dirt, foam, water, and oil) and the influence on the output data of the single beam lidar RIEGL LD05-A20 and the automotive mechanically spinning lidar Ouster OS1-64 was evaluated. The LD05-A20 measurements show that dew, artificial dirt, and foam lead to unwanted reflections at the sensor cover. Dew, artificial dirt over the entire transmitter, and foam measurements lead to severe faults, i.e., complete sensor blindness. The OS1-64 measurements also show that dew can lead to almost complete sensor blindness. The results look promising for further studies on fault detection and isolation, since the different contamination types lead to different symptom combinations.

Sensor models provide the required environmental perception information for the development and testing of automated driving systems in virtual vehicle environments. In this article, a configurable sensor model architecture is introduced. Based on methods of model-based systems engineering (MBSE) and functional decomposition, this approach supports a flexible and continuous way to use sensor models in automotive development. Modeled sensor effects, representing single-sensor properties, are combined to an overall sensor behavior. This improves reusability and enables adaptation to specific requirements of the development. Finally, a first practical application of the configurable sensor model architecture is demonstrated, using two exemplary sensor effects: the geometric field of view (FoV) and the object-dependent FoV.