Miao Zhang

Miao Zhang

Auto-generated profile, based on publicly available information

Miao Zhang

Harbin Institute of Technology, Shenzhen, China

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
21 August 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
HCHongrong Chen
Hongrong Chen
MZMiao Zhang
Miao Zhang
Qinfeng ShiQinfeng Shi
Qinfeng Shi
Modern deep neural networks, particularly recent large language models, come with massive model sizes that require significant computational and storage resources. To enable the deployment of modern models on resource-constrained environments and to accelerate inference time, researchers have increasingly explored pruning techniques as a popular research direction in neural network compression. More than three thousand pruning papers have been published from 2020 to 2024. However, there is a dearth of up-to-date comprehensive review papers on pruning. To address this issue, in this survey, we provide a comprehensive review of existing research works on deep neural network pruning in a taxonomy of 1) universal/specific speedup, 2) when to prune, 3) how to prune, and 4) fusion of pruning and other compression techniques. We then provide a thorough comparative analysis of eight pairs of contrast settings for pruning (e.g., unstructured/structured, one-shot/iterative, data-free/data-driven, initialized/pre-trained weights, etc.) and explore several emerging topics, including pruning for large language models, vision transformers, diffusion models, and large multimodal models, post-training pruning, and different levels of supervision for pruning to shed light on the commonalities and differences of existing methods and lay the foundation for further method development. Finally, we provide some valuable recommendations on selecting pruning methods and prospect several promising research directions for neural network pruning. To facilitate future research on deep neural network pruning, we summarize broad pruning applications (e.g., adversarial robustness, natural language understanding, etc.) and build a curated collection of datasets, networks, and evaluations on different applications. We maintain a repository on https://github.com/hrcheng1066/awesome-pruning that serves as a comprehensive resource for neural network pruning papers and corresponding open-source codes. We will keep updating this repository to include the latest advancements in the field.

13 June 2023 in Proceedings of the ACM on Management of Data
Verified
Journal Article
Open access
DCDavid Campos
David Campos
MZMiao Zhang
Miao Zhang
BYBin Yang
Bin Yang
TKTung Kieu
Tung Kieu
CGChenjuan Guo
Chenjuan Guo
CJChristian S. Jensen
Christian S. Jensen
Due to the sweeping digitalization of processes, increasingly vast amounts of time series data are being produced. Accurate classification of such time series facilitates decision making in multiple domains. State-of-the-art classification accuracy is often achieved by ensemble learning where results are synthesized from multiple base models. This characteristic implies that ensemble learning needs substantial computing resources, preventing their use in resource-limited environments, such as in edge devices. To extend the applicability of ensemble learning, we propose the LightTS framework that compresses large ensembles into lightweight models while ensuring competitive accuracy. First, we propose adaptive ensemble distillation that assigns adaptive weights to different base models such that their varying classification capabilities contribute purposefully to the training of the lightweight model. Second, we propose means of identifying Pareto optimal settings w.r.t. model accuracy and model size, thus enabling users with a space budget to select the most accurate lightweight model. We report on experiments using 128 real-world time series sets and different types of base models that justify key decisions in the design of LightTS and provide evidence that LightTS is able to outperform competitors.

19 February 2021 in IEEE Transactions on Evolutionary Computation
Verified
Journal Article
Closed access
MZMiao Zhang
Miao Zhang
Huiqi LiHuiqi Li
Huiqi Li
SPShirui Pan
Shirui Pan
JLJuan Lyu
Juan Lyu
SLSteve Ling
Steve Ling
Steven SuSteven Su
Steven Su
This article investigates deep neural networks (DNNs)-based lung nodule classification with hyperparameter optimization. Hyperparameter optimization in DNNs is a computationally expensive problem, and a surrogate-assisted evolutionary algorithm has been recently introduced to automatically search for optimal hyperparameter configurations of DNNs, by applying computationally efficient surrogate models to approximate the validation error function of hyperparameter configurations. Different from existing surrogate models adopting stationary covariance functions (kernels) to measure the difference between hyperparameter points, this article proposes a nonstationary kernel that allows the surrogate model to adapt to functions whose smoothness varies with the spatial location of inputs. A multilevel convolutional neural network (ML-CNN) is built for lung nodule classification, and the hyperparameter configuration is optimized by the proposed nonstationary kernel-based Gaussian surrogate model. Our algorithm searches with a surrogate for optimal setting via a hyperparameter importance-based evolutionary strategy, and the experiments demonstrate our algorithm outperforms manual tuning and several well-established hyperparameter optimization methods, including random search, grid search, the tree-structured parzen estimator (TPE) approach, Gaussian processes (GP) with stationary kernels, and the recently proposed hyperparameter optimization via RBF and dynamic (HORD) coordinate search.

3 November 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
MZMiao Zhang
Miao Zhang
Huiqi LiHuiqi Li
Huiqi Li
SPShirui Pan
Shirui Pan
XCXiaojun Chang
Xiaojun Chang
CZChuan Zhou
Chuan Zhou
Steven SuSteven Su
Steven Su
One-shot neural architecture search (NAS) has recently become mainstream in the NAS community because it significantly improves computational efficiency through weight sharing. However, the supernet training paradigm in one-shot NAS introduces catastrophic forgetting. To overcome this problem of catastrophic forgetting, we formulate supernet training for one-shot NAS as a constrained continual learning optimization problem such that learning the current architecture does not degrade the validation accuracy of previous architectures. The key to solving this constrained optimization problem is a novelty search based architecture selection (NSAS) loss function that regularizes the supernet training by using a greedy novelty search method to find the most representative subset. We applied the NSAS loss function to two one-shot NAS baselines and extensively tested them on both a common search space and a NAS benchmark dataset. We further derive three variants based on the NSAS loss function, the NSAS with depth constrain (NSAS-C) to improve the transferability, and NSAS-G and NSAS-LG to handle the situation with a limited number of constraints. The experiments on the common NAS search space demonstrate that NSAS and it variants improve the predictive ability of supernet training in one-shot NAS baselines.

1 December 2023 in Proceedings of the VLDB Endowment
Verified
Journal Article
Open access
KZKai Zhao
Kai Zhao
CGChenjuan Guo
Chenjuan Guo
YCYunyao Cheng
Yunyao Cheng
PHPeng Han
Peng Han
MZMiao Zhang
Miao Zhang
BYBin Yang
Bin Yang
Multiple time series forecasting plays an essential role in many applications. Solutions based on graph neural network (GNN) that deliver state-of-the-art forecasting performance use the relation graph which can capture historical correlations among time series. However, in real world, it is common that correlations among time series evolve across time, resulting in dynamic relation graph, where the future correlations may be different from those in history. To address this problem, we propose multiple time series forecasting with dynamic graph modeling (MTSF-DG) that is able to learn historical relation graphs and predicting future relation graphs to capture the dynamic correlations. We also propose a causal GNN to extract features from both kinds of relation graphs efficiently. Then we propose a reasoning network to explicitly learn the variant influence from historical timestamps to future timestamps for final forecasting. Extensive experiments on six benchmark datasets show that MTSF-DG consistently outperforms state-of-the-art baselines, and justify our design with dynamic relation graph modeling.

1 July 2026 in IEEE Transactions on Knowledge and Data Engineering
Verified
Journal Article
Closed access
XZXin Zheng
Xin Zheng
YWYi Wang
Yi Wang
YLYixin Liu
Yixin Liu
MLMing Li
Ming Li
MZMiao Zhang
Miao Zhang
SPShirui Pan
Shirui Pan