With the goal of reducing time to market and project costs, the current trend of real time business and mission critical systems is evolving from the development of custom made applications to the use of commercial off the shelf (COTS) products. Obviously, the same confidence and quality of the custom made software components is expected from the commercial applications. In most cases, such products (COTS) are not designed with stringent timing and/or safety requirements as priorities. Thus, to decrease the gap between the use of custom made components and COTS components, this paper presents a methodology for evaluating COTS products in the scope of dependable, real time systems, through the application of fault injection techniques at key points of the software engineering process. By combining the use of robustness testing (fault injection at interface level) with software fault injection (using educated fault injection operators), a COTS component can be assessed in the context of the system it will belong to, with special emphasis given to timing and safety constraints that are usually imposed by the target real time dependable environment. In the course of this work, three case studies have been performed to assess the methodology using realistic scenarios that used common COTS products. Results for one case study are presented
Techniques to classify defects have been used for decades, providing relevant information on how to improve systems. Such techniques heavily rely on human experience and have been generalized to cover different types of systems at different maturity levels. However, their application to safety-critical systems development and operation phases neither is very common, or at least not spread publicly, nor disseminated in the industrial and academic worlds. This practical experience report presents the results and conclusions from applying a mature Orthogonal Defect Classification (ODC) to a large set of safety-critical issues. The work is based on the analysis of more than 240 real issues (defects) identified during all the lifecycle phases of 4 safety-critical systems in the aerospace and space domains. The outcomes reveal the challenges in properly classifying this specific type of issues with the broader ODC approach. The difficulties are identified and systematized and specific proposals for improvement are proposed.
1 February 2017 in Reliability Engineering & System Safety
In each application domain for safety-critical systems, international organizations have issued regulations concerned with the development, implementation, validation and maintenance of safety-critical systems. In particular, each of them indicate a definition of what safety means, proper qualitative and quantitative properties for evaluating the quality of the system under development, and a set of methodologies to be used for assessing the fulfilment of the mentioned properties. These standards are today and essential tool for ensuring the required safety levels in many domains that require extremely high dependability. This paper summarizes the analysis on a set of well-known safety standards in different domains of critical systems with the intend of highlighting similarities and differences among them, pointing out common areas of interest and reporting on which features the newest (and upcoming) standards are focusing.
Fault injection is a well-known technology that enables assessing dependability attributes of computer systems. Many works on fault injection have been developed in the past, and fault injection has been used in different application domains. This fast abstract briefly revises previous applications of fault injection, especially for embedded systems, and puts forward ideas on its future use, both in terms of application areas and business markets.
Safety-critical systems usually need to be qualified and certified, they follow specific and strict development standards that recommend the use of techniques and processes, specific personnel training and domain expertise. These systems are very sensitive to failures and thus there is a need to guarantee the higher quality and dependability levels. The goal of this paper is to present the PhD work plan that shall lead to a disruptive approach to identify the quality gaps, root-causes and improve safety-critical systems engineering. The main idea is to start from the classification of real issues, map them to engineering properties and root causes, and identify how to avoid and reduce the impact of those causes. The foreseen improvements shall be reflected in development and V&V techniques, resources training or preparation, and international standards adaptations in order to reflect measurable improvement in the safety and quality of the systems.