Deep neural networks (DNNs) have shown huge superiority over humans in image recognition, speech processing, autonomous vehicles, and medical diagnosis. However, recent studies indicate that DNNs are vulnerable to adversarial examples (AEs), which are designed by attackers to fool deep learning models. Different from real examples, AEs can mislead the model to predict incorrect outputs while hardly be distinguished by human eyes, therefore threaten security-critical deep-learning applications. In recent years, the generation and defense of AEs have become a research hotspot in the field of artificial intelligence (AI) security. This article reviews the latest research progress of AEs. First, we introduce the concept, cause, characteristics, and evaluation metrics of AEs, then give a survey on the state-of-the-art AE generation methods with the discussion of advantages and disadvantages. After that, we review the existing defenses and discuss their limitations. Finally, future research opportunities and challenges on AEs are prospected.

Physical unclonable function (PUF) is a promising lightweight hardware security primitive for resource-constrained systems. It can generate a large number of challenge-response pairs (CRPs) for device authentication based on process variations. However, attackers can collect the CRPs to build a machine learning (ML) model with high prediction accuracy for the PUF. Recently, a lot of ML-resistant PUF structures have been proposed, e.g., a multiplexer-based PUF (MPUF) was introduced to resist ML attacks and its two variants (rMPUF and cMPUF) were further proposed to resist reliability-based and cryptanalysis modeling attacks, respectively. In this article, we propose a general framework for ML attacks on strong PUFs, then based on the framework, we present two novel modeling attacks, named logical approximation and global approximation, that use artificial neural network (ANN) to characterize the nonlinear structure of MPUF, rMPUF, cMPUF, and XOR Arbiter PUF. The logical approximation method uses linear functions to approximate logical operations and builds a precise soft model based on the combination of logical gates in the PUF. The global approximation method uses the function sinc with filtering characteristics to fit the mapping relationship between the challenge and response. The experimental results show that the proposed two approximation attacks can successfully model the (n, k)-MPUF (k = 3,4), (n, k)-rMPUF (k = 2,3), cMPUF (k = 4,5), and l-XOR Arbiter PUF (l = 3, 4, 5) (n = 32, 64) with the average accuracies of 96.85%, 95.33%, 94.52%, and 96.26%, respectively.

In many Industry Internet of Things (IIoT) applications, resources like CPU, memory, and battery power are limited and cannot afford the classic cryptographic security solutions. Silicon Physical Unclonable Function (PUF) is a lightweight security primitive that exploits manufacturing variations during the chip fabrication process for key generation and/or device authentication. However, traditional weak PUFs such as Ring Oscillator (RO) PUF generate chip-unique key for each device, which restricts their application in security protocols where the same key is required to be shared in resource-constrained devices. In order to address this issue, we propose a PUF-based key sharing method for the first time. The basic idea is to implement one-to-one input-output mapping with Lookup Table (LUT)-based interstage crossing structures in each level of inverters of RO PUF. Individual customization on configuration bits of interstage crossing structure and different RO selections with challenges bring high flexibility. Therefore, with the flexible configuration of interstage crossing structures and challenges, CRO PUF can generate the same shared key for resource-constrained devices, which enables a new application for lightweight key sharing protocols.

Physical unclonable function (PUF) is a promising lightweight hardware security primitive for resource-limited Internet-of-Things (IoT) devices. Strong PUFs are suitable for lightweight device authentication because it can generate quantities of challenge-response pairs. Unfortunately, while the machine learning (ML) techniques have benefited various areas, such as Internet, industrial automation, robotics and gaming, they pose a severe threat to PUFs by easily modelling their behavior. This article first shows that even a recently reported dual-mode PUF can be cloned by ML (prediction accuracy of up to 95%). To solve this issue, we propose a configurable tristate (CT) PUF which can flexibly perform as an arbiter PUF, a ring oscillator (RO) PUF, or a bistable ring (BR) PUF with a bitwise XOR-based mechanism to obfuscate the relationship between the challenge and the response, hence resisting the ML attacks. An authentication protocol for the use in IoT security is presented. The CT PUF is implemented on Xilinx ZedBoard FPGAs with placement and routing details described. The experimental results show that the modelling accuracy of logistic regression (LR), support vector machine (SVM), covariance matrix adaptation evolutionary strategies (CMA-ES), and artificial neural network (ANN) is close to 60% (50% as the ideal number in theory) while meeting the PUF requirements for uniformity, reliability, and uniqueness. The hardware overhead and power consumption are slight. The entire project has been open sourced.

Training a Deep Learning (DL) model requires proprietary data and computing-intensive resources. To recoup their training costs, a model provider can monetize DL models through Machine Learning as a Service (MLaaS). Generally, the model is deployed at the cloud, while providing a publicly accessible Application Programming Interface (API) for paid queries to obtain benefits. However, model stealing attacks have posed security threats to this model monetizing scheme as they steal the model without paying for future extensive queries. Specifically, an adversary queries a targeted model to obtain input-output pairs and thus infer the model’s internal working mechanism by reverse-engineering a substitute model, which has deprived model owner’s business advantage and leaked the privacy of the model. In this work, we observe that the confidence vector or the top-1 confidence returned from the model under attack (MUA) varies in a relative large degree given different queried inputs. Therefore, rich internal information of the MUA is leaked to the attacker that facilities her reconstruction of a substitute model. We thus propose to leverage adversarial confidence perturbation to hide such varied confidence distribution given different queries, consequentially against model stealing attacks (dubbed as APMSA). In other words, the confidence vectors returned now is similar for queries from a specific category, considerably reducing information leakage of the MUA. To achieve this objective, through automated optimization, we constructively add delicate noise into per input query to make its confidence close to the decision boundary of the MUA. Generally, this process is achieved in a similar means of crafting adversarial examples but with a distinction that the hard label is preserved to be the same as the queried input. This retains the inference utility (i.e., without sacrificing the inference accuracy) for normal users but bounded the leaked confidence information to the attacker in a small constrained area (i.e., close to decision boundary). The later renders greatly deteriorated accuracy of the attacker’s substitute model. As the APMSA serves as a plug-in front-end and requires no change to the MUA, it is thus generic and easy to deploy. The high efficacy of APMSA is validated through experiments on datasets of CIFAR10 and GTSRB. Given a MUA model of ResNet-18 on the CIFAR10, our defense can degrade the accuracy of the stolen model by up to 15% (rendering the stolen model useless to a large extent) with 0% accuracy drop for normal user’s hard-label inference request.