Hui Guan

Hui Guan

Auto-generated profile, based on publicly available information

Hui Guan

College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, USA

Research Focus & Metrics

Academic Career Length

14Years (2013-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 April 2024
Verified
Conference Paper
Open access
SASohaib Ahmad
Sohaib Ahmad
HGHui Guan
Hui Guan
BFBrian D. Friedman
Brian D. Friedman
TWThomas Williams
Thomas Williams
RSRamesh K. Sitaraman
Ramesh K. Sitaraman
TWThomas Woo
Thomas Woo
Existing machine learning inference-serving systems largely rely on hardware scaling by adding more devices or using more powerful accelerators to handle increasing query demands. However, hardware scaling might not be feasible for fixed-size edge clusters or private clouds due to their limited hardware resources. A viable alternate solution is accuracy scaling, which adapts the accuracy of ML models instead of hardware resources to handle varying query demands. This work studies the design of a high-throughput inference-serving system with accuracy scaling that can meet throughput requirements while maximizing accuracy. To achieve the goal, this work proposes to identify the right amount of accuracy scaling by jointly optimizing three sub-problems: how to select model variants, how to place them on heterogeneous devices, and how to assign query workloads to each device. It also proposes a new adaptive batching algorithm to handle variations in query arrival times and minimize SLO violations. Based on the proposed techniques, we build an inference-serving system called Proteus and empirically evaluate it on real-world and synthetic traces. We show that Proteus reduces accuracy drop by up to 3× and latency timeouts by 2--10× with respect to baseline schemes, while meeting throughput requirements.

10 July 2023
Verified
Conference Paper
Open access
JHJin Huang
Jin Huang
HGHui Guan
Hui Guan
DGDeepak Ganesan
Deepak Ganesan
While a number of recent efforts have explored the use of "cloud offload" to enable deep learning on IoT devices, these have not assumed the use of duty-cycled radios like BLE. We argue that radio duty-cycling significantly diminishes the performance of existing cloud-offload methods. We tackle this problem by leveraging a previously unexplored opportunity to use early-exit offload enhanced with prioritized communication, dynamic pooling, and dynamic fusion of features. We show that our system, FLEET, achieves significant benefits in accuracy, latency, and compute budget compared to state-of-art local early exit, remote processing, and model partitioning schemes across a range of DNN models, datasets, and IoT platforms.

3 June 2024
Verified
Conference Paper
Open access
SASohaib Ahmad
Sohaib Ahmad
HGHui Guan
Hui Guan
RSRamesh K. Sitaraman
Ramesh K. Sitaraman
The rapid adoption of machine learning (ML) has underscored the importance of serving ML models with high throughput and resource efficiency. Traditional approaches to managing increasing query demands have predominantly focused on hardware scaling, which involves increasing server count or computing power. However, this strategy can often be impractical due to limitations in the available budget or compute resources. As an alternative, accuracy scaling offers a promising solution by adjusting the accuracy of ML models to accommodate fluctuating query demands. Yet, existing accuracy scaling techniques target independent ML models and tend to underperform while managing inference pipelines. Furthermore, they lack integration with hardware scaling, leading to potential resource inefficiencies during low-demand periods. To address the limitations, this paper introduces Loki, a system designed for serving inference pipelines effectively with both hardware and accuracy scaling. Loki incorporates an innovative theoretical framework for optimal resource allocation and an effective query routing algorithm, aimed at improving system accuracy and minimizing latency deadline violations. Our empirical evaluation demonstrates that through accuracy scaling, the effective capacity of a fixed-size cluster can be enhanced by more than 2.7× compared to relying solely on hardware scaling. When compared with state-of-the-art inference-serving systems, Loki achieves up to a 10× reduction in Service Level Objective (SLO) violations, with minimal compromises on accuracy and while fulfilling throughput demands.

22 April 2024
Verified
Conference Paper
Open access
QYQizheng Yang
Qizheng Yang
TYTianyi Yang
Tianyi Yang
MXMingcan Xiang
Mingcan Xiang
LZLijun Zhang
Lijun Zhang
HWHaoliang Wang
Haoliang Wang
HGHui Guan
Hui Guan

3 June 2024
Verified
Conference Paper
Open access
MRMohammad Mehdi Rastikerdar
Mohammad Mehdi Rastikerdar
JHJin Huang
Jin Huang
SFShiwei Fang
Shiwei Fang
HGHui Guan
Hui Guan
DGDeepak Ganesan
Deepak Ganesan
While existing strategies to execute deep learning-based classification on low-power platforms assume the models are trained on all classes of interest, this paper posits that adopting context-awareness i.e. narrowing down a classification task to the current deployment context consisting of only recent inference queries can substantially enhance performance in resource-constrained environments. We propose a new paradigm, CACTUS, for scalable and efficient context-aware classification where a micro-classifier recognizes a small set of classes relevant to the current context and, when context change happens (e.g., a new class comes into the scene), rapidly switches to another suitable micro-classifier. CACTUS features several innovations, including optimizing the training cost of context-aware classifiers, enabling on-the-fly context-aware switching between classifiers, and balancing context switching costs and performance gains via simple yet effective switching policies. We show that CACTUS achieves significant benefits in accuracy, latency, and compute budget across a range of datasets and IoT platforms.

25 April 2025
Verified
Conference Paper
Open access
KLKylie Lin
Kylie Lin
SRSean Sheng-Tse Ru
Sean Sheng-Tse Ru
DRDavid N. Rapp
David N. Rapp
HGHui Guan
Hui Guan
Cindy XiongCindy Xiong
Cindy Xiong