Debraj Basu

Debraj Basu

Auto-generated profile, based on publicly available information

Debraj Basu

Adobe, Inc., USA, and also with the University of California, Los Angeles (UCLA), USA

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 April 2020 in IEEE Journal on Selected Areas in Information Theory
Verified
Journal Article
Closed access
DBDebraj Basu
Debraj Basu
DDDeepesh Data
Deepesh Data
CKCan Karakus
Can Karakus
SDSuhas N. Diggavi
Suhas N. Diggavi
Communication bottleneck has been identified as a significant issue in distributed optimization of large-scale learning models. Recently, several approaches to mitigate this problem have been proposed, including different forms of gradient compression or computing local models and mixing them iteratively. In this paper, we propose Qsparse-local-SGD algorithm, which combines aggressive sparsification with quantization and local computation along with error compensation, by keeping track of the difference between the true and compressed gradients. We propose both synchronous and asynchronous implementations of Qsparse-local-SGD. We analyze convergence for Qsparse-local-SGD in the distributed setting for smooth non-convex and convex objective functions. We demonstrate that Qsparse-local-SGD converges at the same rate as vanilla distributed SGD for many important classes of sparsifiers and quantizers. We use Qsparse-local-SGD to train ResNet-50 on ImageNet and show that it results in significant savings over the state-of-the-art, in the number of bits transmitted to reach target accuracy.