Zinuo Cai

Zinuo Cai

Auto-generated profile, based on publicly available information

Zinuo Cai

Shanghai Jiao Tong University, Shanghai, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 September 2023 in IEEE Transactions on Sustainable Computing
Verified
Journal Article
Closed access
SPShanxing Pan
Shanxing Pan
HZHongyu Zhao
Hongyu Zhao
ZCZinuo Cai
Zinuo Cai
DLDongmei Li
Dongmei Li
Ruhui MaRuhui Ma
Ruhui Ma
HGHaibing Guan
Haibing Guan
In recent years, serverless computing has garnered significant attention owing to its high scalability, pay-as-you-go billing model, and efficient resource management provided by cloud service providers. Optimal resource scheduling of serverless computing has become imperative to reduce energy consumption and enable sustainable computing. However, existing serverless platforms encounter two significant challenges: the cold-start problem of containers and the absence of an effective resource allocation strategy for serverless workflows. Existing pre-warm strategies are associated with high computational overhead, while current resource scheduling techniques inadequately account for the intricate structure of serverless workflows. To address these challenges, we present SSC, a pre-warming and automatic resource allocation framework designed explicitly for serverless workflows. We introduce an innovative gradient-based algorithm for pre-warming containers, significantly reducing cold start hit rates. Moreover, leveraging a critical path and priority queue-based algorithm, SSC enables efficient allocation of resources for serverless workflows. In our experimental evaluation, SSC reduces the cold start hit rate by nearly $50\%$ and achieves substantial cost savings of approximately $30\%$ .

5 March 2025 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
JLJunhan Liu
Junhan Liu
ZCZinuo Cai
Zinuo Cai
YLYumou Liu
Yumou Liu
HLHao Li
Hao Li
ZZZongpu Zhang
Zongpu Zhang
RBRajkumar Buyya
Rajkumar Buyya
Deep learning (DL) clusters allow machine learning practitioners to submit their computation-intensive tasks, with GPUs accelerating their execution process. However, GPUs in current deep learning clusters are often under-utilized, which hampers the job performance and overall cluster throughput. It is urgent to improve GPU utilization, but existing works lack research on fine-grained allocation for GPU resources, as it typically allocates GPUs as indivisible units. Serverless computing reveals an opportunity to optimize utilization with fine-grained resource allocation methods, but it requires addressing three main challenges: co-location performance degradation, service level objectives guarantee of serverless functions, and cold start overhead. We propose SMore, a framework based on serverless computing to optimize GPU resource utilization of DL clusters. SMore dynamically predicts the possible co-location performance degradation and leverages a degradation-aware scheduling algorithm to ensure that the co-location decisions do not impact workload performance. It also dynamically preloads or offloads DL models by predicting the request numbers of the subsequent period to address the cold start issue. Through actual trace testing on the prototype of SMore, we find that the average GPU utilization can be increased by 34% with degradation being controlled effectively.

21 August 2023 in IEEE Transactions on Intelligent Transportation Systems
Verified
Journal Article
Closed access
ZCZinuo Cai
Zinuo Cai
ZCZebin Chen
Zebin Chen
ZLZihan Liu
Zihan Liu
QXQuanmin Xie
Quanmin Xie
Ruhui MaRuhui Ma
Ruhui Ma
HGHaibing Guan
Haibing Guan
Modern transportation big data features high Volume, Velocity, and Variety, making it more and more challenging to develop an intelligent transportation system for data analysis. Current transportation systems resort to cloud computing to deploy their applications but face two bottlenecks, lack of real-time processing and under-utilization of intelligent roadside devices. We observe that dispersed computing—an emerging paradigm of cloud computing—well fits the requirements of modern transportation systems. It provides real-time response by alleviating data transmission between data sources and cloud servers and fully utilizes smart devices by exploring their computing capacity. Therefore, we design RIDIC, an intelligent transportation system with dispersed computing to provide a real-time response when processing transportation big data. RIDIC abstracts all the heterogeneous smart roadside devices as actors, and its workflow consists of three stages, Actor Registration, Resource Application and Task Execution. We conduct experiments on two real-life traffic scenarios—road vehicle detection and traffic signal recognition—and the results show that RIDIC can utilize edge devices to process transportation big data faster while reducing the demand for device computing resources.

17 July 2024 in IEEE Transactions on Artificial Intelligence
Verified
Journal Article
Closed access
ZCZinuo Cai
Zinuo Cai
RMRongbo Ma
Rongbo Ma
YFYicheng Fu
Yicheng Fu
Weishan ZhangWeishan Zhang
Weishan Zhang
Ruhui MaRuhui Ma
Ruhui Ma
HGHaibing Guan
Haibing Guan
In recent years, the emergence of large language models has profoundly transformed our production and lifestyle. These models have shown tremendous potential in fields such as natural language processing, speech recognition, and recommendation systems, and are increasingly playing crucial roles in applications like human-computer interaction and intelligent customer service. Efficient inference solutions for large language models in data centers have been extensively researched, with a focus on meeting users’ Quality of Service requirements. In this paper, we focus on two additional requirements that responsible large language model inference should meet under QoS conditions: security throughout the model execution process and low maintenance requirements for the inference system. Therefore, we propose LLMaaS , a trusted model inference platform based on a serverless computing platform, aimed at providing inference as a service for large language models. First, we design a trusted serverless computing platform based on SGX, which includes distributed identity verification and SGX device plugins to ensure the security and trustworthiness of the inference process. Additionally, to reduce the maintenance requirements of the system, we enhance the SGX-based deep learning computing framework, including replacing PyTorch and using a greedy algorithm for graph partitioning. We conduct tests on four typical large models, and the experimental results demonstrate that, with minimal overhead and user code modifications, we can ensure the security of model execution.

21 June 2024 in IEEE Transactions on Cybernetics
Verified
Journal Article
Closed access
ZCZinuo Cai
Zinuo Cai
ZCZebin Chen
Zebin Chen
XCXinglei Chen
Xinglei Chen
Ruhui MaRuhui Ma
Ruhui Ma
HGHaibing Guan
Haibing Guan
RBRajkumar Buyya
Rajkumar Buyya
With the advance of smart manufacturing and information technologies, the volume of data to process is increasing accordingly. Current solutions for big data processing resort to distributed stream processing systems, such as Apache Flink and Spark. However, such frameworks face challenges of resource underutilization and high latency in big data application scenarios. In this article, we propose SPSC, a serverless-based stream computing framework where events are discretized into the atomic stream and stateless Lambda functions are taken as context-irrelevant operators, achieving task parallelism and inherent data parallelism in processing. Also, we implement a prototype of the framework on Amazon Web service (AWS) using AWS Lambda, AWS simple queue service, and AWS DynamoDB. The evaluation shows that compared with Alibaba’s real-time computing Flink version, SPSC outperforms by 10.12% when the overhead is close.

11 July 2023 in IEEE Transactions on Network Science and Engineering
Verified
Journal Article
Closed access
HZHongyu Zhao
Hongyu Zhao
SPShanxing Pan
Shanxing Pan
ZCZinuo Cai
Zinuo Cai
XCXinglei Chen
Xinglei Chen
LJLingxiao Jin
Lingxiao Jin
HGHaibing Guan
Haibing Guan
The prosperity of the Internet has made network traffic analysis increasingly indispensable in network operation. With the development of machine learning, more researchers and engineers are using deep learning models for network traffic analysis. However, the rapidly growing data size and model complexity make resource scheduling a serious limitation, which is why cloud computing services are typically required for network analysis. To leverage the advantages of serverless platforms, we propose faaShark , an end-to-end network traffic analysis system based on a serverless computing platform. faaShark adapts distributed training to fully utilize the flexibility of serverless platforms. Additionally, we design a cold start optimization algorithm to reduce the hit rate of cold start when serving pretrained models to handle network analysis requests. Our extensive experiments evaluate the impact of several parameters of distributed training and confirm the effectiveness of our cold start optimization algorithm when building such a network analysis system atop serverless computing frameworks.