The recent surge in deploying machine learning (ML) models at the edge has revolutionized various industries by enabling real-time decision-making on resource-constrained devices, such as TinyML models on microcontrollers. However, this trend brings forth a critical concern - the vulnerability of these models to adversarial examples. ML at the edge offers tremendous potential but demands heightened vigilance in the realm of cybersecurity. Our research has shown that any adversarial robustness attained in standard TensorFlow models through adversarial training can be completely nullified during post-training full integer quantization to address resource constraints of edge devices. This finding raises crucial questions about the adversarial robustness of TinyML models on microcontrollers limited to integer-only operations. As edge computing continues to proliferate, addressing these vulnerabilities and developing lightweight defenses tailored to resource-constrained environments becomes imperative for ensuring the security and trustworthiness of edge-deployed ML models.

The current body of work on adversarial robustness seems to imply that theoretical robustness against adversarial examples leads to more secure systems. In this paper we demonstrate that this premise is erroneous by assessing the strengths and limitations of prominent robustness methods in light of facial authentication, a realistic use case where adversarial perturbations pose a real threat, which allows for a natural reflection on the security gained by the obtained robustness. The main contribution of this paper is an evaluation and critical reflection upon why prominent robustness methods fail to deliver a secure system despite living up to their promises of adding robustness. Our analysis shows that state-of-the-art robustness methods such as Adversarial Training and Guided Complement Entropy struggle to accommodate for two key requirements of facial authentication: (1)~the threat model of facial authentication assumes physical adversarial examples that can be added to the scene as opposed to "classical" adversarial examples that are applied to the digital input. Moreover, an attacker that can directly perturb digital input, does not require adversarial perturbations to impersonate their victim; (2)~robustness properties are only validated for standard classification problems, and often ignore the impact of more practical training paradigms that re-purpose models. Our extensive evaluation of robustness in light of facial authentication allowed us to pinpoint the limitations of these methods. To ensure that the concepts of adversarial robustness and security are more tightly coupled, we recommend to evaluate new defences with applications where adversarial perturbations pose a security threat.

Adversarial training, a widely used technique for fortifying the robustness of machine learning models, has seen its effectiveness further bolstered by modifying loss functions or incorporating additional terms into the training objective. While these adaptations are validated through empirical studies, they lack a solid theoretical basis to explain the models’ secure and robust behavior. In this paper, we investigate the integration of adversarial triplets within the adversarial training framework, a method previously shown to enhance robustness. However, the reasons behind this increased robustness are poorly understood, and the impact of different adversarial triplet configurations remains unclear. To address this gap, we utilize the robust and non-robust features framework to analyze how various adversarial triplet compositions influence robustness, providing deeper insights into the robustness guarantees of this approach. Specifically, we introduce a novel framework that explains how different compositions of adversarial triplets lead to distinct training dynamics, thereby affecting the model’s adversarial robustness. We validate our theoretical findings through empirical analysis, demonstrating that our framework accurately characterizes the effects of adversarial triplets on the training process. Our results offer a comprehensive explanation of how adversarial triplets influence the security and robustness of models, providing a theoretical foundation for methods that employ adversarial triplets to improve robustness. This research not only enhances our theoretical understanding but also has practical implications for developing more robust machine learning models.

The success of machine learning has significantly impacted biometric authentication systems, such as facial recognition, but it has also exposed them to substantial risks through adversarial examples. This work investigates physically realizable adversarial examples against RGB-D facial recognition systems, which combine traditional RGB images with depth data. While showing increased performance in a benign setting, we demonstrate that RGB-D systems remain vulnerable to adversarial examples in white and black-box settings. Furthermore, we show that attacks targeting a single modality, such as only making perturbations on the RGB color space without changing the depth data, remain highly successful, even against RGB-D models. Our research compares the robustness of RGB and RGB-D systems to identify the benefits and limitations of incorporating depth data into face recognition systems. We quantify how different fusion architectures of RGB-D systems influence their resistance to attack and evaluate how the size and positioning of adversarial patches impact attack success rates. Our findings highlight that while multi-modal systems offer potential advantages, they still face challenges in mitigating adversarial vulnerabilities.