Industrial Internet of Things (IIoT) is significant of building powerful industrial systems and applications. Deep learning has provided a promising opportunity to extract useful knowledge by utilizing vast amounts of data in IIoT. However, lacking of massive public datasets will lead to low performance and overfitting of the learned model. Therefore, the federated deep learning over distributed datasets has been proposed. Whereas, it inevitably introduces some new security challenges, i.e., disclosing participant's data privacy. However, existing methods cannot guarantee each participant's data privacy in a learning group. In this article, we propose two privacy-preserving asynchronous deep learning schemes [privacy-preserving and asynchronous deep learning via re-encryption (DeepPAR) and dynamic privacy-preserving and asynchronous deep learning (DeepDPA)]. Compared to the state-of-the-art work, DeepPAR protects each participant's input privacy while preserving dynamic update secrecy inherently. Meanwhile, DeepDPA enables to guarantee backward secrecy of group participants in a lightweight manner. Security analysis and performance evaluations on real dataset show that our proposed schemes are secure, efficient, and effective.
With the advent of artificial intelligence, machine learning has been well explored and extensively applied into numerous fields, such as pattern recognition, image processing and cloud computing. Very recently, machine learning hosted in a cloud service has gained more attentions due to the benefits from the outsourcing paradigm. Based on cloud-aided computation techniques, the heavy computation tasks involved in machine learning process can be off-loaded into the cloud server in a pay-per-use manner, whereas outsourcing large-scale collection of sensitive data risks privacy leakage since the cloud server is semi-honest. Therefore, privacy preservation for the client and verification for the returned results become two challenges to be dealt with. In this paper, we focus on designing a novel privacy-preserving single-layer perceptron training scheme which supports batch patterns training and verification for the training results on the client side. In addition, adopting classical secure two-party computation method, we design a novel lightweight privacy-preserving predictive algorithm. Both two participants learns nothing about other’s inputs, and the calculation result is only known by one party. Detailed security analysis shows that the proposed scheme can achieve the desired security properties. We also demonstrate the efficiency of our scheme by providing the experimental evaluation on two different real datasets.
29 December 2023 in IEEE Transactions on Dependable and Secure Computing
Training a deep learning model from scratch requires a great deal of available labeled data, computation resources, and expert knowledge. Thus, the time-consuming and complicated learning procedure catapulted the trained model to valuable intellectual property (IP), spurring interest from attackers in model copyright infringement and stealing. Recently, a new defense approach leverages watermarking techniques to inject watermarks into the training procedure and verify model ownership when necessary. To our best knowledge, there is no research work on model ownership stealing attacks in federated learning, and the existing defense or mitigation methods can not be directly used for federated learning scenarios. In this paper, we introduce watermarking neural networks in asynchronous federated learning and propose a novel model privacy attack, dubbed model ownership deprivation attack (MODA). MODA is launched by an inside adversarial participant, targeting occupying and depriving the remaining participants' (victims) copyright to achieve his maximum profit. The extensive experimental results on five benchmark datasets (MNIST, Fashion-MNIST, GTSRB, SVHN, CIFAR10) show that MODA is highly effective in a two-participant learning scenario with a minor impact on model's performance. When extending MODA into multiple participants scenario, MODA still maintains high attack success rate and classification accuracy. Compared to the state-of-the-art works, MODA has a higher attack success rate than the black-box solution and comparable efficacy with the approach in the white-box scenario.
As concerns over privacy protection grow and relevant laws come into effect, machine unlearning (MU) has emerged as a pivotal research area. Due to the complexity of the forgetting data distribution, the sample-wise MU is still open challenges. Gradient ascent, as the inverse of gradient descent, is naturally applied to machine unlearning, which is also the inverse process of machine learning. However, the straightforward gradient ascent MU method suffers from the trade-off between effectiveness, fidelity, and efficiency. In this work, we analyze the gradient ascent MU process from a multi-task learning (MTL) view. This perspective reveals two problems that cause the trade-off, i.e., the gradient direction problem and the gradient dominant problem. To address these problems, we propose a novel MU method, namely GDR-GMA, consisting of Gradient Direction Rectification (GDR) and Gradient Magnitude Adjustment (GMA). For the gradient direction problem, GDR rectifies the direction between the conflicting gradients by projecting a gradient onto the orthonormal plane of the conflicting gradient. For the gradient dominant problem, GMA dynamically adjusts the magnitude of the update gradients by assigning the dynamic magnitude weight parameter to the update gradients. Furthermore, we evaluate GDR-GMA against several baseline methods in three sample-wise MU scenarios: random data forgetting, sub-class forgetting, and class forgetting. Extensive experimental results demonstrate the superior performance of GDR-GMA in effectiveness, fidelity, and efficiency.
11 April 2025 in Proceedings of the AAAI Conference on Artificial Intelligence
The core challenge of high-dimensional and expensive black-box optimization (BBO) is how to obtain better performance faster with little function evaluation cost. The essence of the problem is how to design an efficient optimization strategy tailored to the target task. This paper designs a powerful optimization framework to automatically learn the optimization strategies from the target or cheap surrogate task without human intervention. However, current methods are weak for this due to poor representation of optimization strategy. To achieve this, 1) drawing on the mechanism of genetic algorithm, we propose a deep neural network framework called B2Opt, which has a stronger representation of optimization strategies based on survival of the fittest; 2) B2Opt can utilize the cheap surrogate functions of the target task to guide the design of the efficient optimization strategies. Compared to the state-of-the-art BBO baselines, B2Opt can achieve multiple orders of magnitude performance improvement with less function evaluation cost.