The characterization of the detection capability assumes significance when the reliable monitoring of the region of interest by a non-contact sensor is a safety-relevant function. This paper introduces new validation scores that evaluate the detection capability of non-contact sensors intended to be applied to outdoor machines. The scores quantify, in terms of safety, the suitability of the sensor for the intended implementation in an environmental perception system of (highly) automated machines. This was achieved by developing an extension to the new Real Environment Detection Area (REDA) method and linking the methodology with the sensor standard IEC/TS 62998-1. The extension includes point-by-point and statistic-based error evaluation which leads to the Usability-Score, Availability-Score, and Reliability-Score. By applying the principle in the agricultural sector using ISO 18497 and linking this with data from a real outdoor test stand, it was possible to show that the validation scores offer a generic approach to quantify the detection capability and express this in a machine manufacturer-oriented manner. The findings of this study have significant implications for the advancement of safety-related sensor systems integrated into machines operating in complex environments. In order to achieve full implementation, it is necessary to define in the standards which score is required for each Performance Level (PL).
Perception of the environment by sensor systems in variable environmental conditions is very complex due to the interference influences. In the field of autonomous machines or autonomous vehicles, environmental conditions play a decisive role in safe person detection. A uniform test and validation method can support the manufacturers of sensor systems during development and simultaneously provide proof of functionality. The authors have developed a concept of a novel test method, “REDA”, for this purpose. In this article, the concept is applied and measurement data are presented. The results show the versatile potential of this test method, through the manifold interpretation options of the measurement data. Using this method, the strengths and weaknesses of sensor systems have been identified with an unprecedented level of detail, flexibility, and variance to test and compare the detection capability of sensor systems. The comparison was possible regardless of the measuring principle of the sensor system used. Sensor systems have been tested and compared with each other with regard to the influence of environmental conditions themselves. The first results presented highlight the potential of the new test method. For future applications, the test method offers possibilities to test and compare manifold sensing principles, sensor system parameters, or evaluation algorithms, including, e.g., artificial intelligence.