Rujia Wang

Rujia Wang

Auto-generated profile, based on publicly available information

Rujia Wang

Microsoft, Redmond, USA

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 July 2024
Verified
Conference Paper
Open access
XZXuchao Zhang
Xuchao Zhang
SGSupriyo Ghosh
Supriyo Ghosh
CBChetan Bansal
Chetan Bansal
RWRujia Wang
Rujia Wang
MMMinghua Ma
Minghua Ma
SRSaravan Rajmohan
Saravan Rajmohan
Root Cause Analysis (RCA) plays a pivotal role in the incident diagnosis process for cloud services, requiring on-call engineers to identify the primary issues and implement corrective actions to prevent future recurrences. Improving the incident RCA process is vital for minimizing service downtime, customer impact and manual toil. Recent advances in artificial intelligence have introduced state-of-the-art Large Language Models (LLMs) like GPT-4, which have proven effective in tackling various AIOps problems, ranging from code authoring to incident management. Nonetheless, the GPT-4 model’s immense size presents challenges when trying to fine-tune it on user data because of the significant GPU resource demand and the necessity for continuous model fine-tuning with the emergence of new data. To address the high cost of fine-tuning LLM, we propose an in-context learning approach for automated root causing, which eliminates the need for fine-tuning. We conduct extensive study over 100,000 production incidents from Microsoft, comparing several large language models using multiple metrics. The results reveal that our in-context learning approach outperforms the previous fine-tuned large language models such as GPT-3 by an average of 24.8% across all metrics, with an impressive 49.7% improvement over the zero-shot model. Moreover, human evaluation involving actual incident owners demonstrates its superiority over the fine-tuned model, achieving a 43.5% improvement in correctness and an 8.7% enhancement in readability. The impressive results demonstrate the viability of utilizing a vanilla GPT model for the RCA task, thereby avoiding the high computational and maintenance costs associated with a fine-tuned model.

17 March 2021 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
FZFeng Zhang
Feng Zhang
JSJiya Su
Jiya Su
WLWeifeng Liu
Weifeng Liu
BHBingsheng He
Bingsheng He
RWRuofan Wu
Ruofan Wu
RWRujia Wang
Rujia Wang
Sparse triangular solves (SpTRSVs) are widely used in linear algebra domains, and several GPU-based SpTRSV algorithms have been developed. Synchronization-free SpTRSVs, due to their short preprocessing time and high performance, are currently the most popular SpTRSV algorithms. However, we observe that the performance of those SpTRSV algorithms on different matrices can vary greatly by 845 times. Our further studies show that when the average number of components per level is high and the average number of nonzero elements per row is low, those SpTRSVs exhibit extremely low performance. The reason is that, they use a warp on the GPU to process a row in sparse matrices, and such warp-level designs have severe underutilization of the GPU. To solve this problem, we propose YuenyeungSpTRSV, a thread-level and wrap-level fusion synchronization-free SpTRSV algorithm, which handles the rows with a large number of nonzero elements at warp-level while the rows with a low number of nonzero elements at thread-level. Particularly, YuenyeungSpTRSV has three novel features. First, unlike the previous studies, YuenyeungSpTRSV does not need long preprocessing time to calculate levels. Second, YuenyeungSpTRSV exhibits high performance on matrices that previous SpTRSVs cannot handle efficiently. Third, YuenyeungSpTRSV's optimization does not rely on the specific sparse matrix storage format. Instead, it can achieve very good performance on the most popular sparse matrix storage, compressed sparse row (CSR) format, and thus users do not need to conduct format conversion. We evaluate YuenyeungSpTRSV with 245 matrices from the Florida Sparse Matrix Collection on four GPU platforms, and experiments show that our YuenyeungSpTRSV exhibits 7.14 GFLOPS/s, which is 5.98x speedup over the state-of-the-art synchronization-free SpTRSV algorithm, and 4.83x speedup over the SpTRSV in cuSPARSE.

30 March 2025
Verified
Conference Paper
Open access
KJKunal Jain
Kunal Jain
APAnjaly Parayil
Anjaly Parayil
AMAnkur Mallick
Ankur Mallick
ECEsha Choukse
Esha Choukse
XQXiaoting Qin
Xiaoting Qin
SRSaravan Rajmohan
Saravan Rajmohan

2 December 2025 in Proceedings of the ACM on Measurement and Analysis of Computing Systems
Verified
Journal Article
Open access
SJShashwat Jaiswal
Shashwat Jaiswal
KJKunal Jain
Kunal Jain
YSYogesh Simmhan
Yogesh Simmhan
APAnjaly Parayil
Anjaly Parayil
AMAnkur Mallick
Ankur Mallick
SRSaravan Rajmohan
Saravan Rajmohan
Global cloud service providers handle inference workloads for Large Language Models (LLMs) that span latency-sensitive (e.g., chatbots) and insensitive (e.g., report writing) tasks, resulting in diverse and often conflicting Service Level Agreement (SLA) requirements. Managing such mixed workloads is challenging due to the complexity of the inference serving stack, which encompasses multiple models, GPU hardware, and global data centers. Existing solutions often silo such fast and slow tasks onto separate GPU resource pools with different SLAs, but this leads to significant under-utilization of expensive accelerators due to load mismatch. In this article, we characterize the LLM serving workloads at Microsoft Office 365, one of the largest users of LLMs within Microsoft Azure cloud with over 10 million requests per day, and highlight key observations across workloads in different data center regions and across time. This is one of the first such public studies of Internet-scale LLM workloads. We use these insights to propose SageServe , a comprehensive LLM serving framework that dynamically adapts to workload demands using multi-timescale control knobs. It combines short-term request routing to data centers with long-term scaling of GPU VMs and model placement with higher lead times, and co-optimizes the routing and resource allocation problem using a traffic forecast model and an Integer Linear Programming (ILP) solution. We evaluate SageServe through real runs and realistic simulations on 10 million production requests across three regions and four open-source models. We achieve up to 25% savings in GPU-hours compared to the current baseline deployment and reduce GPU-hour wastage due to inefficient auto-scaling by 80%, resulting in a potential monthly cost savings of up to $2.5 million, while maintaining tail latency and meeting SLAs. The workload traces, our simulator harness and the SageServe scheduler are available at https://github.com/shashwatj07/SageServe.

1 December 2020 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
YCYuezhi Che
Yuezhi Che
YYYuanzhou Yang
Yuanzhou Yang
AAAmro Awad
Amro Awad
RWRujia Wang
Rujia Wang
Emerging Non-Volatile Memories (NVMs) are entering the mainstream market. With attractive performance, high density, and near-zero idle power, emerging NVMs are promising contenders to build future memory systems. On the other hand, their limited write endurance and enabling data remanence attacks remain as main challenges that could hinder the wide adoption of NVMs in many sectors. With the limited write-endurance of NVMs, implementing major security primitives, such as Oblivious RAM (ORAM) for memory access pattern obfuscation, become more challenging due to a massive extra-write operation demand, which will exacerbate the write endurance problem. Wear-leveling techniques aim at mitigating the endurance problem of NVM by shuffling locations and continuously changing the mappings of memory addresses to the actual memory cells. However, such wear-leveling techniques do not provide the access pattern obfuscation for security. In this paper, we propose a lightweight memory access pattern obfuscation framework for NVM, called O-NVM, a novel design that achieves both wear-leveling and obfuscation with an appealing level of performance and security (obfuscation) guarantees. Our experimental results show that the NVM's lifetime can be improved by ${4-125\times}$ , and the randomness of address access patterns can achieve the passing rate of statistics tests over 99%.

1 October 2022
Verified
Conference Paper
Closed access
YCYuezhi Che
Yuezhi Che
RWRujia Wang
Rujia Wang
As deep neural networks (DNN) expand their attention into various domains and the high cost of training a model, the structure of a DNN model has become a valuable intellectual property and needs to be protected. However, reversing DNN models by exploiting side-channel leakage has been demonstrated in various ways. Even if the model is encrypted and the processing hardware units are trusted, the attacker can still extract the model’s structure and critical parameters through side channels, potentially posing significant commercial risks. In this paper, we begin by analyzing representative memory side-channel attacks on DNN models and identifying the primary causes of leakage. We also find that the full encryption used to protect model parameters could add extensive overhead. Based on our observations, we propose DNNCloak, a lightweight and secure framework aiming at mitigating reverse engineering attacks on common DNN architectures. DNNCloak includes a set of obfuscation schemes that increase the difficulty of reverse-engineering the DNN structure. Additionally, DNNCloak reduces the overhead of full weights encryption with an efficient matrix permutation scheme, resulting in reduced memory access time and enhanced security against retraining attacks on the model parameters. At last, we show how DNNCloak can defend DNN models from side-channel attacks effectively, with minimal performance overhead.