Mahima Pushkarna

Mahima Pushkarna

Auto-generated profile, based on publicly available information

Mahima Pushkarna

Google Research, United States

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 April 2023
Verified
Conference Paper
Open access
NYNur Yildirim
Nur Yildirim
MPMahima Pushkarna
Mahima Pushkarna
NGNitesh Goyal
Nitesh Goyal
MWMartin Wattenberg
Martin Wattenberg
FVFernanda Viégas
Fernanda Viégas
Artificial intelligence (AI) presents new challenges for the user experience (UX) of products and services. Recently, practitioner-facing resources and design guidelines have become available to ease some of these challenges. However, little research has investigated if and how these guidelines are used, and how they impact practice. In this paper, we investigated how industry practitioners use the People + AI Guidebook. We conducted interviews with 31 practitioners (i.e., designers, product managers) to understand how they use human-AI guidelines when designing AI-enabled products. Our findings revealed that practitioners use the guidebook not only for addressing AI’s design challenges, but also for education, cross-functional communication, and for developing internal resources. We uncovered that practitioners desire more support for early phase ideation and problem formulation to avoid AI product failures. We discuss the implications for future resources aiming to help practitioners in designing AI products.

11 May 2024
Verified
Conference Paper
Open access
MCMichelle Cohn
Michelle Cohn
MPMahima Pushkarna
Mahima Pushkarna
GOGbolahan O. Olanubi
Gbolahan O. Olanubi
JMJoseph M. Moran
Joseph M. Moran
DPDaniel Padgett
Daniel Padgett
CHCourtney Heldreth
Courtney Heldreth
People now regularly interface with Large Language Models (LLMs) via speech and text (e.g., Bard) interfaces. However, little is known about the relationship between how users anthropomorphize an LLM system (i.e., ascribe human-like characteristics to a system) and how they trust the information the system provides. Participants (n=2,165; ranging in age from 18-90 from the United States) completed an online experiment, where they interacted with a pseudo-LLM that varied in modality (text only, speech + text) and grammatical person (“I” vs. “the system”) in its responses. Results showed that the “speech + text” condition led to higher anthropomorphism of the system overall, as well as higher ratings of accuracy of the information the system provides. Additionally, the first-person pronoun (“I”) led to higher information accuracy and reduced risk ratings, but only in one context. We discuss these findings for their implications for the design of responsible, human–generative AI experiences.

2 May 2024
Verified
Conference Paper
Open access
MKMinsuk Kahng
Minsuk Kahng
ITIan Tenney
Ian Tenney
MPMahima Pushkarna
Mahima Pushkarna
MLMichael Xieyang Liu
Michael Xieyang Liu
JWJames Wexler
James Wexler
LDLucas Dixon
Lucas Dixon
Automatic side-by-side evaluation has emerged as a promising approach to evaluating the quality of responses from large language models (LLMs). However, analyzing the results from this evaluation approach raises scalability and interpretability challenges. In this paper, we present LLM Comparator, a novel visual analytics tool for interactively analyzing results from automatic side-by-side evaluation. The tool supports interactive workflows for users to understand when and why a model performs better or worse than a baseline model, and how the responses from two models are qualitatively different. We iteratively designed and developed the tool by closely working with researchers and engineers at Google. This paper details the user challenges we identified, the design and development of the tool, and an observational study with participants who regularly evaluate their models.

18 March 2024
Verified
Conference Paper
Open access
SPSavvas Petridis
Savvas Petridis
BWBenjamin D Wedin
Benjamin D Wedin
JWJames Wexler
James Wexler
MPMahima Pushkarna
Mahima Pushkarna
ADAaron Donsbach
Aaron Donsbach
MTMichael Terry
Michael Terry

10 July 2023
Verified
Conference Paper
Open access
NANarges Ashtari
Narges Ashtari
RMRyan Mullins
Ryan Mullins
CQCrystal Qian
Crystal Qian
JWJames Wexler
James Wexler
ITIan Tenney
Ian Tenney
MPMahima Pushkarna
Mahima Pushkarna
Models are interpretable when machine learning (ML) practitioners can readily understand the reasoning behind their predictions. Ironically, little is known about the ML practitioners’ experience of discovering and adopting novel interpretability techniques in production settings. In a qualitative study with 18 practitioners at a large technology company working with text data, we found that despite varied tasks, practitioners experienced nearly identical challenges related to interpretability methods in model analysis workflows. These stem from problem formulation, the social nature of interpretability investigations, and non-standard practices in cross-functional organizational contexts. A follow-up examination of early-stage design probes with seven practitioners suggests that self-reported experts are “perpetual intermediates”, who can benefit from regular, responsive, and in-situ education about interpretability methods across workflows, regardless of prior experience with models, analysis tools, or interpretability techniques. From these findings, we emphasize the need for multi-stage support for learning of interpretability methods for real-world NLP applications.