Artificial intelligence systems are increasingly being used in industrial applications, security and military contexts, disaster response complexes, policing and justice practices, finance, and healthcare systems. However, disruptions to these systems can have negative impacts on health, mortality, human rights, and asset values. The protection of such systems from various types of destructive influences is thus a relevant area of research. The vast majority of previously published works are aimed at reducing vulnerability to certain types of disturbances or implementing certain resilience properties. At the same time, the authors either do not consider the concept of resilience as such, or their understanding varies greatly. The aim of this study is to present a systematic approach to analyzing the resilience of artificial intelligence systems, along with an analysis of relevant scientific publications. Our methodology involves the formation of a set of resilience factors, organizing and defining taxonomic and ontological relationships for resilience factors of artificial intelligence systems, and analyzing relevant resilience solutions and challenges. This study analyzes the sources of threats and methods to ensure each resilience properties for artificial intelligence systems. As a result, the potential to create a resilient artificial intelligence system by configuring the architecture and learning scenarios is confirmed. The results can serve as a roadmap for establishing technical requirements for forthcoming artificial intelligence systems, as well as a framework for assessing the resilience of already developed artificial intelligence systems.
Background: The healthcare sector demands a higher degree of responsibility, trustworthiness, and accountability when implementing Artificial Intelligence (AI) systems. Machine learning operations (MLOps) for AI-based medical diagnostic systems are primarily focused on aspects such as data quality and confidentiality, bias reduction, model deployment, performance monitoring, and continuous improvement. However, so far, MLOps techniques do not take into account the need to provide resilience to disturbances such as adversarial attacks, including fault injections, and drift, including out-of-distribution. This article is concerned with the MLOps methodology that incorporates the steps necessary to increase the resilience of an AI-based medical diagnostic system against various kinds of disruptive influences.Methods: Post-hoc resilience optimization, post-hoc predictive uncertainty calibration, uncertainty monitoring, and graceful degradation are incorporated as additional stages in MLOps. To optimize the resilience of the AI based medical diagnostic system, additional components in the form of adapters and meta-adapters are utilized. These components are fine-tuned during meta-training based on the results of adaptation to synthetic disturbances. Furthermore, an additional model is introduced for post-hoc calibration of predictive uncertainty. This model is trained using both in-distribution and out-of-distribution data to refine predictive confidence during the inference mode.Results: The structure of resilience-aware MLOps for medical diagnostic systems has been proposed. Experimentally confirmed increase of robustness and speed of adaptation for medical image recognition system during several intervals of the system’s life cycle due to the use of resilience optimization and uncertainty calibration stages. The experiments were performed on the DermaMNIST dataset, BloodMNIST and PathMNIST. ResNet-18 as a representative of convolutional networks and MedViT-T as a representative of visual transformers are considered. It is worth noting that transformers exhibited lower resilience than convolutional networks, although this observation may be attributed to potential imperfections in the architecture of adapters and meta-adapters.Сonclusion: The main novelty of the suggested resilience-aware MLOps methodology and structure lie in the separating possibilities and activities on creating a basic model for normal operating conditions and ensuring its resilience and trustworthiness. This is significant for the medical applications as the developer of the basic model should devote more time to comprehending medical field and the diagnostic task at hand, rather than specializing in system resilience. Resilience optimization increases robustness to disturbances and speed of adaptation. Calibrated confidences ensure the recognition of a portion of unabsorbed disturbances to mitigate their impact, thereby enhancing trustworthiness.
Artificial intelligence technologies are becoming increasingly prevalent in resource-constrained, safety-critical embedded systems. Numerous methods exist to enhance the resilience of AI systems against disruptive influences. However, when resources are limited, ensuring cost-effective resilience becomes crucial. A promising approach for reducing the resource consumption of AI systems during test-time involves applying the concepts and methods of dynamic neural networks. Nevertheless, the resilience of dynamic neural networks against various disturbances remains underexplored. This paper proposes a model architecture and training method that integrate dynamic neural networks with a focus on resilience. Compared to conventional training methods, the proposed approach yields a 24% increase in the resilience of convolutional networks and a 19.7% increase in the resilience of visual transformers under fault injections. Additionally, it results in a 16.9% increase in the resilience of convolutional network ResNet-110 and a 21.6% increase in the resilience of visual transformer DeiT-S under adversarial attacks, while saving more than 30% of computational resources. Meta-training the neural network model improves resilience to task changes by an average of 22%, while achieving the same level of resource savings.
Trainable visual navigation systems based on deep learning demonstrate potential for robustness of onboard camera parameters and challenging environment. However, a deep model requires substantial computational resources and large labelled training sets for successful training. Implementation of the autonomous navigation and training-based fast adaptation to the new environment for a compact drone is a complicated task. The article describes an original model and training algorithms adapted to the limited volume of labelled training set and constrained computational resource. This model consists of a convolutional neural network for visual feature extraction, extreme-learning machine for estimating the position displacement and boosted information-extreme classifier for obstacle prediction. To perform unsupervised training of the convolution filters with a growing sparse-coding neural gas algorithm, supervised learning algorithms to construct the decision rules with simulated annealing search algorithm used for finetuning are proposed. The use of complex criterion for parameter optimization of the feature extractor model is considered. The resulting approach performs better trajectory reconstruction than the well-known ORB-SLAM. In particular, for sequence 7 from the KITTI dataset, the translation error is reduced by nearly 65.6% under the frame rate 10 frame per second. Besides, testing on the independent TUM sequence shot outdoors produces a translation error not exceeding 6% and a rotation error not exceeding 3.68 degrees per 100 m. Testing was carried out on the Raspberry Pi 3+ single-board computer.
4 October 2022 in RADIOELECTRONIC AND COMPUTER SYSTEMS
Modern methods of image recognition are sensitive to various types of disturbances, which actualize the development of resilient intelligent algorithms for safety-critical applications. The current article develops a model and method of training a classifier that exhibits characteristics of resilience to adversarial attacks, fault injection, and concept drift. The proposed model has a hierarchical structure of prototypes and hyperspherical boundaries of classes formed in the space of high-level features. Class boundaries are optimized during training and provide perturbation absorption and graceful degradation. The proposed learning method involves the use of a combined loss function, which allows the use of both labeled and unlabeled data, implements the compression of the feature representation to a discrete form and ensures the compactness of the distribution of classes and the maximization of the buffer zone between classes. The main component of the loss function is the value of the normalized modification of Shannon's information measure, averaged over the alphabet of the classes, expressed as a function of accuracy characteristics. Simultaneously, accuracy characteristics are calculated on the basis of smoothed versions of the distribution of statistical hypothesis testing results. It is experimentally confirmed that the proposed approach provides a certain level of disturbance absorption, graceful degradation and recovery. During testing of the proposed algorithm on the Cifar10 data set, it was established that the integral metric of resilience to tensor damage by inversion of one randomly selected bit is about 0.95 if the share of damaged tensors does not exceed 30%. Also, during testing of the proposed algorithm, it was established that an adversarial attack with a disturbance that does not exceed the L∞-norm threshold equal to 3 provides resilience that exceeds the value of 0.95 according to the integral metric. Additionally, the integral metric of resilience during adaptation to the appearance of two new classes is 0.959. The integral metric of resilience to the real drift of concepts between the two classes is 0.973. The ability to adapt to the appearance of new classes or the concept drift has been confirmed 8 times faster than learning from scratch.