Pratyush Patel

Pratyush Patel

Auto-generated profile, based on publicly available information

Pratyush Patel

University of Washington, Seattle, WA, USA

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 June 2025 in IEEE Micro
Verified
Journal Article
Closed access
ECEsha Choukse
Esha Choukse
PPPratyush Patel
Pratyush Patel
CZChaojie Zhang
Chaojie Zhang
ASAashaka Shah
Aashaka Shah
ÍGÍñigo Goiri
Íñigo Goiri
RBRicardo Bianchini
Ricardo Bianchini
Generative Large Language Model (LLM) applications are rapidly growing, leading to widespread deployment of expensive, power-hungry GPUs. Growing power demands of AI in the cloud industry has become a global problem [5]. Our analysis shows that LLM inference involves two distinct phases: a compute-intensive prefill phase and a memory-intensive decode phase, each with different resource needs. Running them together introduces inefficient scheduling. Furthermore, unlike prefill phase, the decode phase can run on lower-cost and lowerpower hardware. Building on these insights, we propose Splitwise, a scheduling technique that splits prefill and decode phases across different machines to achieve better throughput. Additionally, Splitwise allows phase-specific hardware optimization. By efficiently transferring request state between machines, Splitwise achieves up to 2.35× more throughput within the same power and cost budgets, or 1.4× higher throughput at 20% lower cost and same power.

27 April 2024
Verified
Conference Paper
Open access
PPPratyush Patel
Pratyush Patel
ECEsha Choukse
Esha Choukse
CZChaojie Zhang
Chaojie Zhang
ÍGÍñigo Goiri
Íñigo Goiri
BWBrijesh Warrier
Brijesh Warrier
RBRicardo Bianchini
Ricardo Bianchini
Recent innovation in large language models (LLMs), and their myriad use cases have rapidly driven up the compute demand for datacenter GPUs. Several cloud providers and other enterprises plan to substantially grow their datacenter capacity to support these new workloads. A key bottleneck resource in datacenters is power, which LLMs are quickly saturating due to their rapidly increasing model sizes. We extensively characterize the power consumption patterns of a variety of LLMs and their configurations. We identify the differences between the training and inference power consumption patterns. Based on our analysis, we claim that the average and peak power utilization in LLM inference clusters should not be very high. Our deductions align with data from production LLM clusters, revealing that inference workloads offer substantial headroom for power oversubscription. However, the stringent set of telemetry and controls that GPUs offer in a virtualized environment make it challenging to build a reliable and robust power management framework. We leverage the insights from our characterization to identify opportunities for better power management. As a detailed use case, we propose a new framework called POLCA, which enables power oversubscription in LLM inference clouds. POLCA is robust, reliable, and readily deployable. Using open-source models to replicate the power patterns observed in production, we simulate POLCA and demonstrate that we can deploy 30% more servers in existing clusters with minimal performance loss.

22 May 2023 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
PPPratyush Patel
Pratyush Patel
ZGZibo Gong
Zibo Gong
SRSyeda Rizvi
Syeda Rizvi
ECEsha Choukse
Esha Choukse
PMPulkit Misra
Pulkit Misra
ASAkshitha Sriraman
Akshitha Sriraman
As modern server GPUs are increasingly power intensive, better power management mechanisms can significantly reduce the power consumption, capital costs, and carbon emissions in large cloud datacenters. This paper uses diverse datacenter workloads to study the power management capabilities of modern GPUs. We find that current GPU management mechanisms have limited compatibility and monitoring support under cloud virtualization. They have sub-optimal, imprecise, and non-intuitive implementations of Dynamic Voltage and Frequency Scaling (DVFS) and power capping. Consequently, efficient GPU power management is not widely deployed in clouds today. To address these limitations, we make actionable recommendations for GPU vendors and researchers.

1 April 2018
Verified
Conference Paper
Closed access
PPPratyush Patel
Pratyush Patel
IBIljoo Baek
Iljoo Baek
HKHyoseung Kim
Hyoseung Kim
RRRagunathan Rajkumar
Ragunathan Rajkumar
Hardware accelerators such as GP-GPUs and DSPs are being increasingly used in computationally-intensive real-time and multimedia systems. System efficiency is often increased when CPU tasks suspend while using these devices. In this paper, we extend the existing Multiprocessor Priority Ceiling Protocol (MPCP) schedulability analysis in this particular context. We present three methods to improve the traditional MPCP analysis that reduces pessimism in analyzing blocking times. Two of these methods, the request-driven and the job-driven approaches, are motivated by prior work and are adapted to MPCP. The third combines these two approaches in a novel way to consistently outperform either on its own. We note that our underlying observations are general, and that such methods can also be used for analyzing other real-time synchronization protocols. Experimental results indicate that our analytical improvements result in a significantly higher schedulability compared to the traditional recursion-based analysis, even when self-suspensions are not considered. Our approach is also competitive with and often outperforms the linear-programming-based FMLP+ analysis, while having a considerably lower runtime complexity. We further substantiate the practical feasibility of suspension-based MPCP and examine its benefits over the busy-waiting approach by presenting a case-study on an NVIDIA TX2 embedded platform using real-world vision applications.

1 August 2018 in Journal of Systems Architecture
Verified
Journal Article
Closed access
HKHyoseung Kim
Hyoseung Kim
PPPratyush Patel
Pratyush Patel
SWShige Wang
Shige Wang
RRRagunathan (Raj) Rajkumar
Ragunathan (Raj) Rajkumar

1 August 2017
Verified
Conference Paper
Closed access
HKHyoseung Kim
Hyoseung Kim
PPPratyush Patel
Pratyush Patel
SWShige Wang
Shige Wang
RRRagunathan Raj Rajkumar
Ragunathan Raj Rajkumar
We propose a server-based approach to manage a general-purpose graphics processing unit (GPU) in a predictable and efficient manner. Our proposed approach introduces a GPU server task that is dedicated to handling GPU requests from other tasks on their behalf. The GPU server ensures bounded time to access the GPU, and allows other tasks to suspend during their GPU computation to save CPU cycles. By doing so, we address the two major limitations of the existing real-time synchronization-based GPU management approach: busy waiting and long priority inversion. We implemented a prototype of the server-based approach on a real embedded platform. This case study demonstrates the practicality and effectiveness of the server-based approach. Experimental results indicate that the server-based approach yields significant improvements in task schedulability over the existing synchronization-based approach in most practical settings. Although we focus on a GPU in this paper, the server-based approach can also be used for other types of computational accelerators.