Miao Zhang

Dr. Miao Zhang

National University of Defence Technology
orcid

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 November 2025 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
WLWeishi Li
Weishi Li
YPYong Peng
Yong Peng
MZMiao Zhang
Miao Zhang
LDLiang Ding
Liang Ding
HHHan Hu
Han Hu
LSLi Shen
Li Shen
Deep model fusion/merging is an emerging technique that integrates parameters or predictions from multiple deep learning (DL) models into a unified framework. It combines the abilities of different models to compensate for the biases and errors of an individual model, improving overall performance. However, deep model fusion, especially on large-scale DL models such as large language models (LLMs) and foundation models, faces several challenges, including high computational cost and interference between different heterogeneous models. In order to understand it better, we present a comprehensive survey to summarize the recent progress. We categorize existing model fusion methods as fourfold: 1) weight average (WA) averages the parameters of multiple models to obtain results closer to the optimal solution; 2) considering that direct averaging of models often yields suboptimal results, “mode connectivity” connects networks via paths of nonincreasing loss in weight spaces before the fusion. Along these paths, initial models are transformed into forms with consistent functions and better fusion effects; 3) similarly, for models with poor direct fusion results, “alignment” matches the corresponding units and merges these models, thus fully exploiting the corresponding relationships between the models; and 4) in addition to the above-mentioned methods of parameter fusion, “ensemble learning” fuses the outputs of multiple models in the inference stage to improve the accuracy and robustness of networks. In addition, we analyze the challenges of deep model fusion and illuminate the possible research directions in the future.

3 March 2022 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
MZMiao Zhang
Miao Zhang
PYPeng Yong
Peng Yong
ZJZhu Jiancheng
Zhu Jiancheng
YQYin Quanjun
Yin Quanjun
Edge computing is a good complement to cloud computing for deploying large-scale geo-distributed simulation applications, which are very sensitive to the communication delay among different simulation components (also called tasks in this paper) and users. We mainly focus on the efficient scheduling of simulation components in collaborative edge and cloud environments. As components should be deployed jointly with the consideration of capacity constraints of hosts, it is actually an NP-complete multi-dimensional bin packing problem. Meanwhile, dynamic changes of component and host states require the low deployment latency of scheduling algorithms. Unfortunately, most of the existing schedulers for modern clusters are queue-based, in which tasks are scheduled sequentially, thus lacking the ability to process tightly coupled tasks jointly. Other batching-based placement algorithms are usually time-consuming. This paper describes Pond, a novel flow-based scheduler with the awareness of interactions among tasks and users as well as heterogeneous multi-dimensional resources. First, characteristics of distributed simulation tasks are analysed and the scheduling problem is formulated as a min-cost max-flow (MCMF) problem over the flow network by mapping the communication overhead among tasks and users to the costs of arcs in the network. Considering the inherent defects of existing flow-based schedulers in dealing with multi-dimensional resources, a new method based on dominant resource is proposed and some problem specific heuristics are also designed. Extensive simulation experiments based on Alibaba production trace and some random synthetic parameters are conducted. Results show that Pond can reduce the average communication cost for each task significantly in a quite low deployment latency compared with some baselines.

9 May 2025 in IEEE Transactions on Computers
Verified
Journal Article
Closed access
QLQinglun Li
Qinglun Li
MZMiao Zhang
Miao Zhang
NYNan Yin
Nan Yin
QYQuanjun Yin
Quanjun Yin
LSLi Shen
Li Shen
XCXiaochun Cao
Xiaochun Cao
To address the communication burden and privacy concerns associated with the centralized server in Federated Learning (FL), Decentralized Federated Learning (DFL) has emerged, which discards the server with a peer-to-peer (P2P) communication framework, significantly expanding the application scenarios of FL. However, most existing DFL algorithms are based on symmetric topologies, such as ring and grid topology, which can easily lead to deadlocks and are susceptible to the impact of network link quality in practice. To address these issues, we propose DFedSGPSM, a transitional framework that converts symmetric DFL optimizers into asymmetric variants. By adopting the Push-Sum protocol in asymmetric network topologies, our framework successfully circumvents the deadlock and link-quality issues prevalent in symmetric configurations. To further validate the effectiveness of our algorithm framework, we integrate the local momentum (in DFedAvgM) and SAM (in DFedSAM) from existing symmetric DFL optimizer into DFedSGPSM to accelerate training and pursue smooth local minimum, which enables existing symmetric DFL optimizers to be seamlessly integrated into asymmetric DFL. Theoretical analysis proves that DFedSGPSM achieves a linear speedup rate of $\mathcal{O}(\frac{1}{\sqrt{nT}})$ in the non-convex setting. This analysis also reveals crucial issues such as tighter upper bounds achieved with improved topological connectivity. Empirically, extensive experiments conducted on the MNIST, CIFAR10&100 datasets demonstrate the superior performance of our proposed algorithm compared to several existing SOTA optimizers in terms of generalization.


28 August 2024 in IEEE Transactions on Services Computing
Verified
Journal Article
Closed access
HCHailiang Chen
Hailiang Chen
ZXZhu Xiang
Zhu Xiang
LYLujia Yin
Lujia Yin
MZMiao Zhang
Miao Zhang
QYQuanjun Yin
Quanjun Yin
Low-frequency periodic requests are common in cloud-based enterprise applications. These infrequent requests often leave microservices idle for extended periods, leading to low resource utilization. Furthermore, the randomness of response times may decrease the reliability of the cloud platform. Intuitively, the periodic nature of requests allows for the agile deployment of microservices to promptly free up occupied computing resources. Thus, the key lies in designing low-cost, high-reliability microservice deployment schemes. Traditional approaches relying on specialized expertise are impractical because of intricate interdependencies within microservice frameworks. To address this, the Microservice Deployment Problem for Low-frequency Periodic Requests (MDP-LPR) is formulated, and a Mixed Integer Programming (MIP) model is developed. A deployment framework leveraging statistical analysis and Monte Carlo simulation is proposed to ensure high reliability. Furthermore, a two-stage heuristic algorithm named Relaxation and Precision Mixed Algorithm (RPMA) is introduced to generate low-cost deployment schemes. Finally, experiments are conducted on real-world workflows. The results show that the RPMA outperforms its counterparts in generating low-cost deployment schemes, and the proposed deployment framework enables the automatic acquisition of low-cost, high-reliability deployment schemes.

27 September 2023 in arXiv
Verified
Preprint
Open access
PDF file
WLWeishi Li
Weishi Li
YPYong Peng
Yong Peng
MZMiao Zhang
Miao Zhang
LDLiang Ding
Liang Ding
HHHan Hu
Han Hu
LSLi Shen
Li Shen
Deep model fusion/merging is an emerging technique that merges the parameters or predictions of multiple deep learning models into a single one. It combines the abilities of different models to make up for the biases and errors of a single model to achieve better performance. However, deep model fusion on large-scale deep learning models (e.g., LLMs and foundation models) faces several challenges, including high computational cost, high-dimensional parameter space, interference between different heterogeneous models, etc. Although model fusion has attracted widespread attention due to its potential to solve complex real-world tasks, there is still a lack of complete and detailed survey research on this technique. Accordingly, in order to understand the model fusion method better and promote its development, we present a comprehensive survey to summarize the recent progress. Specifically, we categorize existing deep model fusion methods as four-fold: (1) "Mode connectivity", which connects the solutions in weight space via a path of non-increasing loss, in order to obtain better initialization for model fusion; (2) "Alignment" matches units between neural networks to create better conditions for fusion; (3) "Weight average", a classical model fusion method, averages the weights of multiple models to obtain more accurate results closer to the optimal solution; (4) "Ensemble learning" combines the outputs of diverse models, which is a foundational technique for improving the accuracy and robustness of the final model. In addition, we analyze the challenges faced by deep model fusion and propose possible research directions for model fusion in the future. Our review is helpful in deeply understanding the correlation between different model fusion methods and practical application methods, which can enlighten the research in the field of deep model fusion.