Robust variants represent a promising method to enhance model robustness against adversarial attacks through exploring diverse neural network architectures. However, the significant computational demand of training multiple variants often restricts adversarial defense techniques to a narrow range of model architectures, thus failing to fully exploit the robustness benefits of architectural variations. In this paper, we first reveal that function-preserving knowledge transfer can significantly speed up adversarial training of different architecture variants. Then, we propose ROBIN, a framework for accelerating adversarially robust multi-variant training. By utilizing the architectural similarities among variants, ROBIN facilitates efficient weight transformation across models via two tensor-level atomic operations, hastening the convergence of multiple variants. Our experiments indicate that ROBIN can accelerate the adversarial training process of various architecture variants by 2.56 × to 4.27 ×, enabling efficient exploration of robust network architectures.
In the face of huge threats from adversarial attacks, developing an efficient defense mechanism is crucial for deep learning systems. Adversarial ensemble defense method is one of the most effective techniques for defending against adversarial attacks, which constructs ensembles of multiple DNNs to improve model's robustness. However, deploying ensemble defense methods on existing DNN inference systems is inefficient and impractical due to their dynamics and randomness. To this end, we propose an inference system for adversarial ensemble defense called Garrison, which can deliver robust and low-latency predictions using Multi-Instance GPUs. Garrison employs a multi-granularity GPU partitioning strategy, optimizing hardware utilization by capitalizing on the intrinsic heterogeneity of GPUs. It also integrates a reinforcement learning-based scheduling mechanism, enabling random ensemble of diverse defense models to enhance robustness while maintaining bounded latency. Our evaluations show that Garrison can improve adversarial robustness by up to 24.5%, while accelerating ensemble inference by up to 6.6X compared to the state-of-the-art inference framework.