15 February 2021 in IEEE Transactions on Automatic Control
This article studies the problem of distributed optimization in multiagent systems where each agent seeks to minimize the sum of all agents’ objective functions using only local information. Under the requirement of security, each agent needs to keep its objective function private from other agents and potential eavesdroppers. We first prove the impossibility of guaranteeing convergence and differential privacy simultaneously by perturbing states in exact distributed optimization algorithms. Motivated by this result, we design a completely distributed algorithm, Distributed algorithm via Direction and State Perturbation (DiaDSP), that achieves differential privacy by perturbing both states and directions with decaying Laplace noise. Different from most of the existing works that require decaying stepsizes to ensure convergence, we show that our DiaDSP algorithm converges in mean and almost surely even with a constant stepsize. In particular, we prove linear convergence in mean by only assuming that the sum of all cost functions is strongly convex. The R-linear convergence is proved under the assumption of Lipschitz gradients instead of that of bounded gradients. The optimal stepsize for the fastest convergence rate is also established. Moreover, we describe the privacy properties and characterize the tradeoff between differential privacy and convergence accuracy. Simulations are conducted on a typical sensor fusion problem to validate the theoretical results.
3 July 2020 in IEEE Transactions on Signal Processing
Differential privacy is a formal mathematical frame- work for quantifying the degree of individual privacy in a statistical database. To guarantee differential privacy, a typical method is to add random noise to the original data for data release. In this paper, we investigate the conditions of differential privacy (single-dimensional case) considering the general random noise adding mechanism, and then apply the obtained results for privacy analysis of the privacy-preserving consensus algorithm. Specifically, we obtain a necessary and sufficient condition of differential privacy, and the sufficient conditions of (δ,ε)- differential privacy. We apply them to analyze various random noises. For the special cases with known results, our theory not only matches with the literature, but also provides an efficient approach to the privacy parameters’ estimation; for other cases that are unknown, our approach provides a simple and effective tool for differential privacy analysis. Applying the obtained theory on privacy-preserving consensus algorithm, we obtain the necessary condition and the sufficient condition to ensure differential privacy.
31 May 2018 in IEEE Transactions on Information Theory
Network systems often rely on distributed algorithms to achieve a global computation goal with iterative local information exchanges between neighbor nodes. To preserve data privacy, a node may add a random noise to its original data for information exchange at each iteration. Nevertheless, an eavesdropping node can estimate other’s original data based on the information it received. The estimation accuracy and data privacy can be measured in terms of (ϵ,δ)-data-privacy, defined as the probability of ϵ-accurate estimate (the difference of an estimation and the original data is within ϵ) is no larger than δ (the disclosure probability). How to optimize the estimation and analyze data privacy is a critical and open issue. In this paper, a theoretical framework is developed to investigate how to optimize the estimation of neighbor’s original data using the local information received, named optimal distributed estimation. Then, we study the disclosure probability under the optimal estimation for data privacy analysis. We further apply the developed framework to analyze the data privacy of the privacypreserving average consensus algorithm and identify the optimal noises for the algorithm.
19 November 2019 in IEEE Transactions on Automatic Control
As the cyber-attack is becoming one of the most challenging threats faced by cyber-physical systems, investigating the effect of cyber-attacks on distributed optimization and designing resilient algorithms are of both theoretical merits and practical values. Most existing works are established on the assumption that the maximum tolerable number of attacks, which depends on the network connectivity, is known by all normal agents. All normal agents will use the maximum number of attacks to decide whether the received information will be used for iterations. In this article, we relax this assumption and propose a novel resilient distributed optimization algorithm. The proposed algorithm exploits the trusted agents which cannot be compromised by adversarial attacks and form a connected dominating set in the original graph to constrain effects of adversarial attacks. It is shown that local variables of all normal and trusted agents converge to the same value under the proposed algorithm. Further, the final solution belongs to the convex set of minimizers of the weighted average of local cost functions of all trusted agents. The upper bound of the distance between the final solution and the optimal one has also been provided. Numerical results are presented to demonstrate the effectiveness of the proposed algorithm.
23 December 2020 in IEEE Internet of Things Journal
A mass of data generated by the widespread smart devices is transmitted through wireless communication networks for estimation and control in the industrial internet of things (IIoT) systems. The frequent data transmission needs to meet the high reliability and real-time demand of IIoT applications because the freshness of status updates influences the system performance. In this work, we use the age of information (AoI) to characterize the information freshness since it is a powerful metric to capture the randomness of state updates. Meanwhile, AoI is very useful in the control and communication co-design to improve the control performance considering communication disturbance. In order to analyze the control cost, we first derive the specific expression average AoI under the packet loss with finite retransmission times. Then, we investigate the influence of average AoI on control performance and obtain the joint cost combining communication energy consumption and control cost. According to the certainty equivalent principle, we design the optimal control law separately. Besides, we prove that the optimal joint infinite horizon cost is bounded by the linear function of average AoI. The communication policy, including the data inter-arrival rate and code length, is designed by optimizing the mixed integer nonlinear programming (MINP) problem. Finally, simulation results reveal that, by employing the optimal data inter-arrival rate and code length, the joint control and communication cost is significantly reduced.
20 August 2018 in IEEE Transactions on Signal and Information Processing over Networks
Privacy-preserving average consensus aims to guarantee the privacy of initial states and asymptotic consensus on the exact average of the initial values. In the existing work, it is achieved by adding variance decaying and zero-sum random noises to the consensus process. However, there is lack of theoretical analysis to quantify the degree of the data privacy protection. In this paper, we introduce the maximum disclosure probability that other nodes can infer one node's initial state within a given small interval to quantify the data privacy. We develop a novel privacy definition, named (α,β)-data-privacy, to depict the relationship between the maximum disclosure probability and the estimation accuracy. Then, we prove that the general privacy-preserving average consensus (GPAC) provides (α,β)-data-privacy, and obtain the closed-form expression of the relationship between α and β given the noise distribution. We reveal that the added noise with a uniform distribution is optimal in terms of achieving the highest (α,β)-data-privacy. We also prove that under what condition, the data-privacy will be compromised. Finally, an optimal privacy-preserving average consensus (OPAC) algorithm is proposed to achieve the highest (α,β)-data-privacy. Simulations are conducted to verify the analytical results.