Sunwoo Lee

Sunwoo Lee

Auto-generated profile, based on publicly available information

Sunwoo Lee

Department of Computer Engineering, Inha University, Incheon, Republic of Korea

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 June 2023
Verified
Conference Paper
Open access
TZTuo Zhang
Tuo Zhang
LGLei Gao
Lei Gao
SLSunwoo Lee
Sunwoo Lee
MZMi Zhang
Mi Zhang
SASalman Avestimehr
Salman Avestimehr
In cross-device Federated Learning (FL) environments, scaling synchronous FL methods is challenging as stragglers hinder the training process. Moreover, the availability of each client to join the training is highly variable over time due to system heterogeneities and intermittent connectivity. Recent asynchronous FL methods (e.g., FedBuff [22]) have been proposed to overcome these issues by allowing slower users to continue their work on local training based on stale models and to contribute to aggregation when ready. However, we show empirically that this method can lead to a substantial drop in training accuracy as well as a slower convergence rate. The primary reason is that fast-speed devices contribute to many more rounds of aggregation while others join more intermittently or not at all, and with stale model updates. To overcome this barrier, we propose TimelyFL, a heterogeneity-aware asynchronous FL framework with adaptive partial training. During the training, TimelyFL adjusts the local training workload based on the real-time resource capabilities of each client, aiming to allow more available clients to join in the global update without staleness. We demonstrate the performance benefits of TimelyFL by conducting extensive experiments on various datasets (e.g., CIFAR-10, Google Speech, and Reddit) and models (e.g., ResNet20, VGG11, and ALBERT). In comparison with the state-of-the-art (i.e., FedBuff), our evaluations reveal that TimelyFL improves participation rate by 21.13%, harvests 1.28× - 2.89× more efficiency on convergence rate, and provides a 6.25% increment on test accuracy.

11 November 2022 in Neural Networks
Verified
Journal Article
Closed access
SLSunwoo Lee
Sunwoo Lee
CHChaoyang He
Chaoyang He
SASalman Avestimehr
Salman Avestimehr

8 August 2023 in Neurocomputing
Verified
Journal Article
Closed access
SLSunwoo Lee
Sunwoo Lee
ASAnit Kumar Sahu
Anit Kumar Sahu
CHChaoyang He
Chaoyang He
SASalman Avestimehr
Salman Avestimehr

5 June 2020 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
QKQiao Kang
Qiao Kang
SLSunwoo Lee
Sunwoo Lee
KHKaiyuan Hou
Kaiyuan Hou
RRRobert Ross
Robert Ross
AAAnkit Agrawal
Ankit Agrawal
WLWei-Keng Liao
Wei-Keng Liao
Two-phase I/O is a well-known strategy for implementing collective MPI-IO functions. It redistributes I/O requests among the calling processes into a form that minimizes the file access costs. As modern parallel computers continue to grow into the exascale era, the communication cost of such request redistribution can quickly overwhelm collective I/O performance. This effect has been observed from parallel jobs that run on multiple compute nodes with a high count of MPI processes on each node. To reduce the communication cost, we present a new design for collective I/O by adding an extra communication layer that performs request aggregation among processes within the same compute nodes. This approach can significantly reduce inter-node communication contention when redistributing the I/O requests. We evaluate the performance and compare it with the original two-phase I/O on Cray XC40 parallel computers (Theta and Cori) with Intel KNL and Haswell processors. Using I/O patterns from two large-scale production applications and an I/O benchmark, we show our proposed method effectively reduces the communication cost and hence maintains the scalability for a large number of processes.

22 April 2024 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
SLSunwoo Lee
Sunwoo Lee
TZTuo Zhang
Tuo Zhang
SPSaurav Prakash
Saurav Prakash
YNYue Niu
Yue Niu
SASalman Avestimehr
Salman Avestimehr
In Federated Learning (FL), clients may have weak devices that cannot train the full model or even hold it in their memory space. To implement large-scale FL applications, thus, it is crucial to develop a distributed learning method that enables the participation of such weak clients. We propose $\mathtt{EmbracingFL}$ , a general FL framework that allows all available clients to join the distributed training regardless of their system resource capacity. The framework is built upon a novel form of partial model training method in which each client trains as many consecutive output-side layers as its system resources allow. Our study demonstrates that $\mathtt{EmbracingFL}$ encourages each layer to have similar data representations across clients, improving FL efficiency. The proposed partial model training method guarantees convergence to a neighbor of stationary points for non-convex and smooth problems. We evaluate the efficacy of $\mathtt{EmbracingFL}$ under a variety of settings with a mixed number of strong, moderate ( $\sim\! 40\%$ memory), and weak ( $\sim\! 15\%$ memory) clients, datasets (CIFAR-10, FEMNIST, and IMDB), and models (ResNet20, CNN, and LSTM). Our empirical study shows that $\mathtt{EmbracingFL}$ consistently achieves high accuracy as like all clients are strong, outperforming the state-of-the-art width reduction methods (i.e., HeteroFL and FjORD).

1 May 2022 in Parallel Computing
Verified
Journal Article
Closed access
SLSunwoo Lee
Sunwoo Lee
KHKai-Yuan Hou
Kai-Yuan Hou
KWKewei Wang
Kewei Wang
SSSaba Sehrish
Saba Sehrish
MPMarc Paterno
Marc Paterno
WLWei-Keng Liao
Wei-Keng Liao