Yasharth Bajpai

Yasharth Bajpai

Auto-generated profile, based on publicly available information

Yasharth Bajpai

PROSE Team, Microsoft Corporation, Bangalore, India

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 November 2023
Verified
Conference Paper
Open access
PGPriyanshu Gupta
Priyanshu Gupta
AKAvishree Khare
Avishree Khare
YBYasharth Bajpai
Yasharth Bajpai
SCSaikat Chakraborty
Saikat Chakraborty
SGSumit Gulwani
Sumit Gulwani
ATAshish Tiwari
Ashish Tiwari
Developers spend a significant amount of time in editing code for a variety of reasons such as bug fixing or adding new features. Designing effective methods to predict code edits has been an active yet challenging area of research due to the diversity of code edits and the difficulty of capturing the developer intent. In this work, we address these challenges by endowing pre-trained large language models (LLMs) with the knowledge of relevant prior associated edits, which we call the Grace (Generation conditioned on Associated Code Edits) method. The generative capability of the LLMs helps address the diversity in code changes and conditioning code generation on prior edits helps capture the latent developer intent. We evaluate two well-known LLMs, codex and CodeT5, in zero-shot and fine-tuning settings respectively. In our experiments with two datasets, Grace boosts the performance of the LLMs significantly, enabling them to generate 29% and 54% more correctly edited code in top-1 suggestions relative to the current state-of-the-art symbolic and neural approaches, respectively.

31 October 2022 in Proceedings of the ACM on Programming Languages
Verified
Journal Article
Open access
YZYuhao Zhang
Yuhao Zhang
YBYasharth Bajpai
Yasharth Bajpai
PGPriyanshu Gupta
Priyanshu Gupta
AKAmeya Ketkar
Ameya Ketkar
MAMiltiadis Allamanis
Miltiadis Allamanis
ATAshish Tiwari
Ashish Tiwari
Integrated Development Environments (IDEs) provide tool support to automate many source code editing tasks. Traditionally, IDEs use only the spatial context, i.e., the location where the developer is editing, to generate candidate edit recommendations. However, spatial context alone is often not sufficient to confidently predict the developer’s next edit, and thus IDEs generate many suggestions at a location. Therefore, IDEs generally do not actively offer suggestions and instead, the developer is usually required to click on a specific icon or menu and then select from a large list of potential suggestions. As a consequence, developers often miss the opportunity to use the tool support because they are not aware it exists or forget to use it. To better understand common patterns in developer behavior and produce better edit recommendations, we can additionally use the temporal context, i.e., the edits that a developer was recently performing. To enable edit recommendations based on temporal context, we present Overwatch, a novel technique for learning edit sequence patterns from traces of developers’ edits performed in an IDE. Our experiments show that Overwatch has 78% precision and that Overwatch not only completed edits when developers missed the opportunity to use the IDE tool support but also predicted new edits that have no tool support in the IDE.

10 July 2024
Verified
Conference Paper
Open access
PBParam Biyani
Param Biyani
YBYasharth Bajpai
Yasharth Bajpai
ARArjun Radhakrishna
Arjun Radhakrishna
GSGustavo Soares
Gustavo Soares
SGSumit Gulwani
Sumit Gulwani
Evaluating conversational assistants, such as GitHub Copilot Chat, poses a significant challenge for tool builders in the domain of Software Engineering. These assistants rely on language models and chat-based user experiences, rendering their evaluation with respect to the quality of the Human-AI conversations complicated. Existing general-purpose metrics for measuring conversational quality found in literature are inadequate for appraising domain-specific dialogues due to their lack of contextual sensitivity. In this paper, we present RUBICON, a technique for evaluating domain-specific Human-AI conversations. RUBICON leverages large language models to generate candidate rubrics for assessing conversation quality and employs a selection process to choose the subset of rubrics based on their performance in scoring conversations. In our experiments, RUBICON effectively learns to differentiate conversation quality, achieving higher accuracy and yield rates than existing baselines.

9 December 2025 in ACM Transactions on Computer-Human Interaction
Verified
Journal Article
Open access
JLJenny T. Liang
Jenny T. Liang
AKAayush Kumar
Aayush Kumar
YBYasharth Bajpai
Yasharth Bajpai
SGSumit Gulwani
Sumit Gulwani
VLVu Le
Vu Le
GSGustavo Soares
Gustavo Soares
Spreadsheet programming is challenging. Programmers use spreadsheet programming knowledge (e.g., formulas) and problem-solving skills to combine actions into complex tasks. Advancements in large language models have introduced language agents that observe, plan, and perform tasks, showing promise for spreadsheet creation. We present TableTalk, a spreadsheet programming agent embodying three design principles—scaffolding, flexibility, and incrementality—derived from studies with seven spreadsheet programmers and 85 Excel templates. TableTalk guides programmers through structured plans based on professional workflows, generating three potential next steps to adapt plans to programmer needs. It uses pre-defined tools to generate spreadsheet components and incrementally build spreadsheets. In a study with 20 programmers, TableTalk produced higher-quality spreadsheets 2.3 times more likely to be preferred than the baseline. It reduced cognitive load and thinking time by 12.6%. From this, we derive design guidelines for agentic spreadsheet programming tools and discuss implications on spreadsheet programming, end-user programming, AI-assisted programming, and human-agent collaboration.