Distributed machine learning algorithms are increasingly used in multirobot systems and are prone to Byzantine attacks. In this article, we consider a distributed implementation of the stochastic gradient descent (SGD) algorithm in a cooperative network, where networked agents optimize a global loss function using SGD on the local data and aggregation of the estimates of immediate neighbors. Byzantine agents can send arbitrary estimates to their neighbors, which may disrupt the convergence of normal agents to the optimum state. We show that if every normal agent combines its neighbors’ estimates (states) such that the aggregated state is in the convex hull of its normal neighbors’ states, then the resilient convergence is guaranteed. To assure this sufficient condition, we propose a resilient aggregation rule based on the notion of centerpoint, which is a generalization of the median in the higher-dimensional Euclidean space. We evaluate our results using examples of target pursuit and pattern recognition in multirobot systems. The evaluation results demonstrate that distributed learning with average, coordinate-wise median, and geometric median-based aggregation rules fail to converge to the optimum state, whereas the centerpoint-based aggregation rule is resilient in the same scenario.
17 February 2023 in Neural Computing and Applications
In this paper, we demonstrate that there is a conflicting relation between two crucial properties—controllability and robustness—in linear dynamical networks of diffusively coupled agents. In particular, for any given number of nodes N and diameter D, we identify networks that are maximally robust using the notion of Kirchhoff index and then analyze their strong structural controllability by computing the minimum number of leaders (nodes receiving external control inputs) needed to make such networks controllable under all feasible coupling weights. Then, for any N and D, we obtain a sharp upper bound on the minimum number of leaders needed to design strong structurally controllable networks with N nodes and diameter D. We also discuss that the bound is best possible for arbitrary values of N and D. Moreover, we construct a family of graphs for any N and D such that the graphs have maximal edge sets while being strong structurally controllable with a number of leaders in the proposed bound. We then analyze the robustness of this graph family. The results suggest that optimizing robustness increases the number of leaders needed for strong structural controllability.