Tao Sun

Tao Sun

Auto-generated profile, based on publicly available information

Tao Sun

National University of Defense Technology

Research Focus & Metrics

Academic Career Length

13Years (2014-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 August 2022 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
TSTao Sun
Tao Sun
DLDongsheng Li
Dongsheng Li
BWBao Wang
Bao Wang
Federated averaging (FedAvg) is a communication-efficient algorithm for distributed training with an enormous number of clients. In FedAvg, clients keep their data locally for privacy protection; a central parameter server is used to communicate between clients. This central server distributes the parameters to each client and collects the updated parameters from clients. FedAvg is mostly studied in centralized fashions, requiring massive communications between the central server and clients, which leads to possible channel blocking. Moreover, attacking the central server can break the whole system's privacy. Indeed, decentralization can significantly reduce the communication of the busiest node (the central one) because all nodes only communicate with their neighbors. To this end, in this paper, we study the decentralized FedAvg with momentum (DFedAvgM), implemented on clients that are connected by an undirected graph. In DFedAvgM, all clients perform stochastic gradient descent with momentum and communicate with their neighbors only. To further reduce the communication cost, we also consider the quantized DFedAvgM. The proposed algorithm involves the mixing matrix, momentum, client training with multiple local iterations, and quantization, introducing extra items in the Lyapunov analysis. Thus, the analysis of this paper is much more challenging than previous decentralized (momentum) SGD or FedAvg. We prove convergence of the (quantized) DFedAvgM under trivial assumptions; the convergence rate can be improved to sublinear when the loss function satisfies the PŁ property. Numerically, we find that the proposed algorithm outperforms FedAvg in both convergence speed and communication cost.

3 September 2018 in IEEE Transactions on Signal Processing
Verified
Journal Article
Closed access
TSTao Sun
Tao Sun
HJHao Jiang
Hao Jiang
LCLizhi Cheng
Lizhi Cheng
WZWei Zhu
Wei Zhu
In this paper, we consider solving a class of nonconvex and nonsmooth problems frequently appearing in signal processing and machine learning research. The traditional alternating direction method of multipliers encounters troubles in both mathematics and computations in solving the nonconvex and nonsmooth subproblem. In view of this, we propose a reweighted alternating direction method of multipliers. In this algorithm, all subproblems are convex and easy to solve. We also provide several guarantees for the convergence and prove that the algorithm globally converges to a critical point of an auxiliary function with the help of the Kurdyka-Łojasiewicz property. Several numerical results are presented to demonstrate the efficiency of the proposed algorithm.

31 March 2022 in International Journal of Intelligent Systems
Verified
Journal Article
Closed access
KWKang Wang
Kang Wang
YDYong Dou
Yong Dou
TSTao Sun
Tao Sun
PQPeng Qiao
Peng Qiao
DWDong Wen
Dong Wen

16 August 2017 in Advances in Computational Mathematics
Verified
Journal Article
Closed access
PDF file
TSTao Sun
Tao Sun
PYPenghang Yin
Penghang Yin
LCLizhi Cheng
Lizhi Cheng
HJHao Jiang
Hao Jiang

25 August 2017 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
TSTao Sun
Tao Sun
HJHao Jiang
Hao Jiang
LCLizhi Cheng
Lizhi Cheng
The nonsmooth and nonconvex regularization has many applications in imaging science and machine learning research due to its excellent recovery performance. A proximal iteratively reweighted nuclear norm algorithm has been proposed for the nonsmooth and nonconvex matrix minimizations. In this paper, we aim to investigate the convergence of the algorithm. With the Kurdyka-Łojasiewicz property, we prove the algorithm globally converges to a critical point of the objective function. The numerical results presented in this paper coincide with our theoretical findings.

10 March 2017 in Journal of Global Optimization
Verified
Journal Article
Closed access
PDF file
TSTao Sun
Tao Sun
HJHao Jiang
Hao Jiang
LCLizhi Cheng
Lizhi Cheng