Haozhao Wang

Prof. Dr. Haozhao Wang

orcid

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 May 2022 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
JZJie Zhang
Jie Zhang
SGSong Guo
Song Guo
ZQZhihao Qu
Zhihao Qu
DZDeze Zeng
Deze Zeng
Haozhao WangHaozhao Wang
Haozhao Wang
Albert ZomayaAlbert Zomaya
Albert Zomaya
Most of the existing FL systems focus on a data-parallel architecture where training data are partitioned by samples among several parties. In some real-life applications, however, partitioning by features is also of practical relevance and the number of features is usually unbalanced among parties. The corresponding learning framework is referred to as Vertical Federated Learning (VFL). Though some pioneering work focused on VFL, the convergence properties of VFL on unbalanced features, especially when parties conduct different numbers of local updates concerning heterogeneous computational capabilities are still unknown. In this article, we propose a new learning framework to improve the training efficiency of VFL on unbalanced features. Given the number of features and the computational capability owned by each party, our thorough theoretical analysis exhibits that the number of local updates conducted by each party has a great effect on the convergence rate and the computational complexity, both of which jointly determine the overall training efficiency in an interrelated and sophisticated way. Based on our theoretical findings, we formulate an optimization problem and derive the optimal solution by selecting an adaptive number of local training rounds for each party. Extensive experiments on various datasets and models demonstrate that our approach significantly improves the training efficiency of VFL.

10 September 2021 in IEEE Internet of Things Journal
Verified
Journal Article
Closed access
Haozhao WangHaozhao Wang
Haozhao Wang
ZQZhihao Qu
Zhihao Qu
QZQihua Zhou
Qihua Zhou
HZHaobo Zhang
Haobo Zhang
BLBoyuan Luo
Boyuan Luo
Ruixuan LiRuixuan Li
Ruixuan Li
The ever-growing artificial intelligence (AI) applications have greatly reshaped our world in many areas, e.g., smart home, computer vision, natural language processing, etc. Behind these applications are usually machine learning (ML) models with extremely large size, which require huge data sets for accurate training to mine the value contained in the big data. Large ML models, however, can consume tremendous computing resources to achieve decent performance and thus, it is difficult to train them in resource-constrained Internet of Things (IoT) environments, which would prevent further development and application of AI techniques in the future. To deal with such challenges, there are many efforts on accelerating the training process for large ML models in IoT. In this article, we provide a comprehensive review on the recent advances toward reducing the computing cost during the training stage while maintaining comparable model accuracy. Specifically, the optimization algorithms that aim to improve the convergence rate are emphasized over various distributed learning architectures that exploit ubiquitous computing resources. Then, the article elaborates the computation hardware acceleration and communication optimization for collaborative training among multiple learning entities. Finally, the remaining challenges, future opportunities, and possible directions are discussed.

24 May 2021 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
ZQZhihao Qu
Zhihao Qu
SGSong Guo
Song Guo
Haozhao WangHaozhao Wang
Haozhao Wang
BYBaoliu Ye
Baoliu Ye
YWYi Wang
Yi Wang
BTBin Tang
Bin Tang
Federated Learning (FL) is a promising machine learning paradigm to cooperatively train a global model with highly distributed data located on mobile devices. Aiming to optimize the communication efficiency for gradient aggregation and model synchronization among large-scale devices, we propose a relay-assisted FL framework. By breaking the traditional transmission-order constraint and exploiting the broadcast characteristic of relay nodes, we design a novel synchronization scheme named Partial Synchronization Parallel (PSP), in which models and gradients are transmitted simultaneously and aggregated at relay nodes, resulting in traffic reduction. We prove that PSP has the same convergence rate as the sequential synchronization approaches via rigorous analysis. To further accelerate the training process, we integrate PSP with any unbiased and error-bounded compression technologies and prove that the convergence properties of the resulting scheme still hold. Extensive experiments are conducted in a distributed cluster environment with real-world datasets and the results demonstrate that our proposed approach reduces the training time up to 37 percent compared to state-of-the-art methods.

2 August 2024 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
YLYichen Li
Yichen Li
WXWenchao Xu
Wenchao Xu
YQYining Qi
Yining Qi
Haozhao WangHaozhao Wang
Haozhao Wang
Ruixuan LiRuixuan Li
Ruixuan Li
SGSong Guo
Song Guo
Federated Learning (FL) is to allow multiple clients to collaboratively train a model while keeping their data locally. However, existing FL approaches typically assume that the data in each client is static and fixed, which cannot account for incremental data with domain shift, leading to catastrophic forgetting on previous domains, particularly when clients are common edge devices that may lack enough storage to retain full samples of each domain. To tackle this challenge, we propose Federated Domain-Incremental Learning via Synergistic Replay (SR-FDIL), which alleviates catastrophic forgetting by coordinating all clients to cache samples and replay them. More specifically, when new data arrives, each client selects the cached samples based not only on their importance in the local dataset but also on their correlation with the global dataset. Moreover, to achieve a balance between learning new data and memorizing old data, we propose a novel client selection mechanism by jointly considering the importance of both old and new data. We conducted extensive experiments on several datasets of which the results demonstrate that SR-FDIL outperforms state-of-theart methods by up to 4.05% in terms of average accuracy of all domains.

17 March 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
YLYichen Li
Yichen Li
Haozhao WangHaozhao Wang
Haozhao Wang
YQYining Qi
Yining Qi
WLWei Liu
Wei Liu
Ruixuan LiRuixuan Li
Ruixuan Li
Federated learning (FL) has emerged as a significant distributed machine learning paradigm. It allows the training of a global model through user collaboration without the necessity of sharing their original data. Traditional FL generally assumes that each client’s data remains fixed or static. However, in real-world scenarios, data typically arrives incrementally, leading to a dynamically expanding data domain. In this study, we examine catastrophic forgetting within Federated Incremental Learning (FIL) and focus on the training resources, where edge clients may not have sufficient storage to keep all data or computational budget to implement complex algorithms designed for the server-based environment. We propose a general and low-cost framework for FIL named Re-Fed+, which is designed to help clients cache important samples for replay. Specifically, when a new task arrives, each client initially caches selected previous samples based on their global and local significance. The client then trains the local model using both the cached samples and the new task samples. From a theoretical perspective, we analyze how effectively Re-Fed+ can identify significant samples for replay to alleviate the catastrophic forgetting issue. Empirically, we show that Re-Fed+ achieves competitive performance compared to state-of-the-art methods.

9 May 2025 in IEEE Communications Surveys & Tutorials
Verified
Journal Article
Closed access
YLYichen Li
Yichen Li
Haozhao WangHaozhao Wang
Haozhao Wang
WXWenchao Xu
Wenchao Xu
TXTianzhe Xiao
Tianzhe Xiao
HLHong Liu
Hong Liu
Ruixuan LiRuixuan Li
Ruixuan Li
Non-Centralized Continual Learning (NCCL) has become an emerging paradigm for enabling distributed devices such as vehicles and servers to handle streaming data from a joint non-stationary environment. To achieve high reliability and scalability in deploying this paradigm in distributed systems, it is essential to overcome challenges stemming from both spatial and temporal dimensions, manifesting as distribution shifts, catastrophic forgetting, heterogeneity, and privacy issues. This survey focuses on a comprehensive examination of the development of the non-centralized continual learning algorithms and the real-world deployment across distributed devices. We begin with an introduction to the background and fundamentals of non-centralized learning and continual learning. Then, we review existing solutions from three levels to represent how existing techniques alleviate the catastrophic forgetting and distribution shift. Additionally, we delve into the various types of heterogeneity issues, security, and privacy attributes, as well as real-world applications across three prevalent scenarios. Furthermore, we establish a large-scale benchmark to revisit this problem and analyze the performance of the state-of-the-art NCCL approaches. Finally, we discuss the important challenges and future research directions in NCCL.