Public intelligent services enabled by machine learning algorithms are vulnerable to model extraction attacks that can steal confidential information of the learning models through public queries. Though there are some protection options such as differential privacy (DP) and monitoring, which are considered promising techniques to mitigate this attack, we still find that the vulnerability persists. In this article, we propose an adaptive query-flooding parameter duplication (QPD) attack. The adversary can infer the model information with black-box access and no prior knowledge of any model parameters or training data via QPD. We also develop a defense strategy using DP called monitoring-based DP (MDP) against this new attack. In MDP, we first propose a novel real-time model extraction status assessment scheme called Monitor to evaluate the situation of the model. Then, we design a method to guide the differential privacy budget allocation called APBA adaptively. Finally, all DP-based defenses with MDP could dynamically adjust the amount of noise added in the model response according to the result from Monitor and effectively defends the QPD attack. Furthermore, we thoroughly evaluate and compare the QPD attack and MDP defense performance on real-world models with DP and monitoring protection.

Network intrusion detection systems (IDS) are often considered effective to thwart cyber attacks. Currently, state-of-the-art (SOTA) IDSs are mainly based on machine learning (ML) including deep learning (DL) models, which suffer from their own security issues, especially evasion attacks by using adversarial examples. However, previous studies mostly focus on extracted features rather than the traffic sample itself, and/or assume that the adversary knows the information of the target model more or less, which severely restricts attack feasibility in practice. In this paper, we re-investigate this problem in a more realistic label-only black-box scenario and propose a practical evasion attack strategy to solve the above limitations. In this newly considered case that the adversary morphs the traffic sample and only obtains the results accepted or rejected without other knowledge, we successfully leverage the model extraction and transfer attack to evade the detection. The entire attack strategy is automated and a comprehensive evaluation is performed. Final results show that the proposed strategy effectively evades seven typical ML-based IDSs and one SOTA DL-based IDS with an average success rate of over 75%. We also discuss the corresponding countermeasures against our attack, which finally highlight the need for effective defenses against our attack.

The equation-solving model extraction attack is an intuitively simple but devastating attack to steal confidential information of regression models through a sufficient number of queries. Complete mitigation is difficult. Thus, the development of countermeasures is focused on degrading the attack effectiveness as much as possible without losing the model utilities. We investigate a novel personalized local differential privacy mechanism to defend against the attack. We obfuscate the model by adding high-dimensional Gaussian noise on model coefficients. Our solution can adaptively produce the noise to protect the model on the fly. We thoroughly evaluate the performance of our mechanisms using real-world datasets. The experiment shows that the proposed scheme outperforms the existing differential-privacy-enabled solution, i.e. 4 times more queries are required to achieve the same attack result. We also publish the relevant codes to the community for further research.

Federated learning (FL), an attractive and promising distributed machine learning paradigm, has sparked extensive interest in exploiting tremendous data stored on ubiquitous mobile devices. However, conventional FL suffers severely from resource heterogeneity, as clients with weak computational and communication capabilities may be unable to complete local training using the same local training hyper-parameters. In this article, we propose Dap-FL, a deep deterministic policy gradient (DDPG)-assisted adaptive FL system, in which local learning rates and local training epochs are adaptively adjusted by all resource-heterogeneous clients through locally deployed DDPG-assisted adaptive hyper-parameter selection schemes. Particularly, the rationality of the proposed hyper-parameter selection scheme is confirmed through rigorous mathematical proof. Besides, due to the thoughtlessness of security consideration of adaptive FL systems in previous studies, we introduce the Paillier cryptosystem to aggregate local models in a secure and privacy-preserving manner. Rigorous analyses show that the proposed Dap-FL system could protect clients’ private local models against chosen-plaintext attacks and chosen-message attacks in a widely used honest-but-curious participants and active adversaries security model. More importantly, through ingenious and extensive experiments, the proposed Dap-FL achieves higher model prediction accuracy than two state-of-the-art RL-assisted FL methods, i.e., 6.03% higher than DDPG-based FL and 7.85% higher than DQN-based FL. In addition, experimental results also show that the proposed Dap-FL achieves higher global model prediction accuracy and faster convergence rates than conventional FL, and the comprehensiveness of the adjusted local training hyper-parameters is validated.

Image retrieval techniques can be easily abused to violate personal privacy with images containing individuals' sensitive information. For example, people's identity information can be inferred from their face photos. Therefore, images should be sanitized before being shared or transmitted. However, previous works on image privacy protection suffer from either no provable privacy protection or poor utility with privacy guarantee. In this work, we propose CODER , a privacy protection mechanism in image retrieval, with provable privacy guarantee as well as improved utility. In particular, CODER achieves metric differential privacy and adopts a newly proposed distortion metric definition which measures the distance more precisely to improve utility. The novel distortion metric can be applied to an arbitrary k-dimensional metric space with stronger image privacy protection. We theoretically analyze the privacy guarantee and rigorous utility bound of CODER . We also experimentally compare its performance with two state-of-the-art works on two widely used face datasets. The results show that CODER significantly improves the utility of the protected images and demonstrates its superiority in terms of the privacy-utility trade-off over the compared works. Finally, we perform reliability verification on both discriminative and generative models to demonstrate the practicality of CODER

Federated recommender system (RS), a prevailing distributed paradigm, has been spawning significant interest in exploiting locally stored but tremendous data to predict items best aligned with clients. However, federated RS suffers severely from a single point of failure due to the dependency on the central server, leading to potential denial of service (DoS) attacks. To address this security weakness, in this paper, we propose a decentralized privacy-preserving federated graph convolutional network for RS, dubbed DeFedGCN. Specifically, DeFedGCN aggregates local updates by a decentralized consensus-reaching process and customizes local models for personalized recommendation, where the aggregation is enhanced by local differential privacy to resist model inversion attacks. More importantly, to promote the recommendation performance, DeFedGCN conducts a sub-graph expansion based on the private set interaction to explore high-order interactions among clients and items. Theoretical analysis confirms the effectiveness and privacy guarantee of DeFedGCN. Additionally, we conduct extensive experiments on four widespread real-world databases. The recommendation performance of DeFedGCN outperforms the state-of-the-art federated RS algorithms without security protection against DoS attacks by up to 7.4%. We release the codes at https://github.com/kqkq926/DeFedGCN.