Milad Khademi Nori

Milad Khademi Nori

Auto-generated profile, based on publicly available information

Milad Khademi Nori

Department of Electrical and Computer Engineering, Queen's University, Kingston, Canada

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
24 May 2021 in IEEE Transactions on Communications
Verified
Journal Article
Closed access
MNMilad Khademi Nori
Milad Khademi Nori
SYSangseok Yun
Sangseok Yun
IKILMIN KIM
ILMIN KIM
Federated Learning (FL) has recently received a lot of attention for large-scale privacy-preserving machine learning. However, high communication overheads due to frequent gradient transmissions decelerate FL. To mitigate the communication overheads, two main techniques have been studied: (i) local update of weights characterizing the trade-off between communication and computation and (ii) gradient compression characterizing the trade-off between communication and precision. To the best of our knowledge, studying and balancing those two trade-offs jointly and dynamically while considering their impacts on convergence has remained unresolved even though it promises significantly faster FL. In this paper, we first formulate our problem to minimize learning error with respect to two variables: local update coefficients and sparsity budgets of gradient compression who characterize trade-offs between communication and computation/precision, respectively. We then derive an upper bound of the learning error in a given wall-clock time considering the interdependency between the two variables. Based on this theoretical analysis, we propose an enhanced FL scheme, namely Fast FL (FFL), that jointly and dynamically adjusts the two variables to minimize the learning error. We demonstrate that FFL consistently achieves higher accuracies faster than similar schemes existing in the literature.

27 April 2020 in Wireless Networks
Verified
Journal Article
Closed access
PDF file
MNMilad Khademi Nori
Milad Khademi Nori
SSSaeed Sharifian
Saeed Sharifian

21 November 2023 in IEEE Access
Verified
Journal Article
Open access
MNMilad Khademi Nori
Milad Khademi Nori
YGYiqun Ge
Yiqun Ge
IKILMIN KIM
ILMIN KIM
Neurons of the brain never produce the same output twice even when the same stimuli are presented; this is because of their internal noisy biological processes. Likewise, in this work, we study activation noise jointly during training, and inference, in a generative energy-based modeling setting, and observe that the joint activation noise yields noteworthy outcomes: while it is known that applying activation noise during training serves the purpose of regularization and data augmentation, activation noise is not quite well-understood during inference, and most importantly, the relationship between the former and the latter, when we adopt both at the same time, is yet unknown. In this work, therefore, we analyze the role of activation noise at inference time and demonstrate it to be utilizing sampling. Then, we prove that the best performance is achieved when the activation noise follows the same distribution both during training and inference. Leveraging the proof of the optimal joint activation noise during training and inference, we achieve significant improvement in performance (classification accuracy). To help understand this phenomenon, we provide theoretical results that illuminate the roles of activation noise during training, inference, and their mutual influence on performance. To further confirm our theoretical results, we conduct extensive experiments on autoencoder architecture for five datasets and seven different distributions of activation noise.