Vikram Nathan

Vikram Nathan

Auto-generated profile, based on publicly available information

Vikram Nathan

Amazon Web Services

Research Focus & Metrics

Representative Publications

View All Publications
9 June 2024
Verified
Conference Paper
Open access
ZWZiniu Wu
Ziniu Wu
RMRyan Marcus
Ryan Marcus
ZLZhengchun Liu
Zhengchun Liu
PNParimarjan Negi
Parimarjan Negi
VNVikram Nathan
Vikram Nathan
TKTim Kraska
Tim Kraska
Query performance (e.g., execution time) prediction is a critical component of modern DBMSes. As a pioneering cloud data warehouse, Amazon Redshift relies on an accurate execution time prediction for many downstream tasks, ranging from high-level optimizations, such as automatically creating materialized views, to low-level tasks on the critical path of query execution, such as admission, scheduling, and execution resource control. Unfortunately, many existing execution time prediction techniques, including those used in Redshift, suffer from cold start issues, inaccurate estimation, and are not robust against workload/data changes. In this paper, we propose a novel hierarchical execution time predictor: the Stage predictor. The Stage predictor is designed to leverage the unique characteristics and challenges faced by Redshift. The Stage predictor consists of three model states: an execution time cache, a lightweight local model optimized for a specific DB instance with uncertainty measurement, and a complex global model that is transferable across all instances in Redshift. We design a systematic approach to use these models that best leverages optimality (cache), instance-optimization (local model), and transferable knowledge about Redshift (global model). Experimentally, we show that the Stage predictor makes more accurate and robust predictions while maintaining a practical inference latency and memory overhead. Overall, the Stage predictor can improve the average query execution latency by 20% on these instances compared to the prior query performance predictor in Redshift.

9 June 2024
Verified
Conference Paper
Open access
VNVikram Nathan
Vikram Nathan
VSVikramank Singh
Vikramank Singh
ZLZhengchun Liu
Zhengchun Liu
MRMohammad Rahman
Mohammad Rahman
AKAndreas Kipf
Andreas Kipf
TKTim Kraska
Tim Kraska
Cloud-based data warehouses are built to be easy to use, requiring minimal intervention from customers as their workloads scale. However, there are still many dimensions of a workload that they do not scale with automatically. For example, in cloud-managed clusters, large ad-hoc queries and ETL workloads must use the same cluster size provisioned for the rest of the workload, and warehouse size does not automatically grow as the underlying data grows in size, causing queries to slow down. In this paper, we describe RAIS, the latest collection of AI-powered scaling and optimization techniques in Amazon Redshift, released in preview at re:Invent 2023, which enable it to scale both vertically and horizontally to adapt to all types of workload variability. RAIS dynamically provisions compute resources to run heavy queries efficiently and automatically optimizes warehouse size for the customer's workload, even as it shifts over time. We show that, depending on the workload, RAIS improves either cost or average query execution time by up to 7.6× and 14.2×, respectively, over existing baselines.

20 November 2024
Verified
Conference Paper
Open access
YDYanlei Diao
Yanlei Diao
DHDominik Horn
Dominik Horn
AKAndreas Kipf
Andreas Kipf
OSOleksandr Shchur
Oleksandr Shchur
IBInes Benito
Ines Benito
TKTim Kraska
Tim Kraska
There has been a growing demand for making modern cloud-based data analytics systems cost-effective and easy to use. AI-powered intelligent resource scaling is one such effort, aiming at automating scaling decisions for serverless offerings like Amazon Redshift Serverless. The foundation of intelligent resource scaling lies in the ability to forecast query workloads and their resource consumption accurately. Although the forecasting problem has been extensively studied across various domains, there is a lack of thorough analysis of existing forecasting algorithms for large-scale, real-world cloud query workloads. This paper fills this gap by providing an in-depth analysis of forecasting algorithms for real-world cloud workloads, covering the fundamental data characteristics that distinguish query workload forecasting from prior problems and evaluating the strengths and limitations of existing algorithms in this new domain. We anticipate that our findings will provide valuable insights in informing the design of an efficient and effective solution for production use, as well as in steering the forecasting community toward more effective algorithms of high real-world impact.