Umang Bhatt

Dr. Umang Bhatt


Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 July 2023 in Patterns
Verified
Journal Article
Open access
VCValerie Chen
Valerie Chen
UBUmang Bhatt
Umang Bhatt
HHHoda Heidari
Hoda Heidari
AWAdrian Weller
Adrian Weller
ATAmeet Talwalkar
Ameet Talwalkar
Machine learning (ML) practitioners are increasingly tasked with developing models that are aligned with non-technical experts’ values and goals. However, there has been insufficient consideration of how practitioners should translate domain expertise into ML updates. In this review, we consider how to capture interactions between practitioners and experts systematically. We devise a taxonomy to match expert feedback types with practitioner updates. A practitioner may receive feedback from an expert at the observation or domain level and then convert this feedback into updates to the dataset, loss function, or parameter space. We review existing work from ML and human-computer interaction to describe this feedback-update taxonomy and highlight the insufficient consideration given to incorporating feedback from non-technical experts. We end with a set of open questions that naturally arise from our proposed taxonomy and subsequent survey.

11 April 2025 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
UBUmang Bhatt
Umang Bhatt
VCValerie Chen
Valerie Chen
KCKatherine M. Collins
Katherine M. Collins
PKParameswaran Kamalaruban
Parameswaran Kamalaruban
EKEmma Kallina
Emma Kallina
ATAmeet Talwalkar
Ameet Talwalkar
Individual human decision-makers may benefit from different forms of support to improve decision outcomes, but when will each form of support yield better outcomes? In this work, we posit that personalizing access to decision support tools can be an effective mechanism for instantiating the appropriate use of AI assistance. Specifically, we propose the general problem of learning a decision support policy that, for a given input, chooses which form of support to provide to decision-makers for whom we initially have no prior information. We develop Modiste, an interactive tool to learn personalized decision support policies. Modiste leverages stochastic contextual bandit techniques to personalize a decision support policy for each decision-maker. In our computational experiments, we characterize the expertise profiles of decision-makers for whom personalized policies will outperform offline policies, including population-wide baselines. Our experiments include realistic forms of support (e.g., expert consensus and predictions from a large language model) on vision and language tasks. Our human subject experiments add nuance to and bolster our computational experiments, demonstrating the practical utility of personalized policies when real users benefit from accessing support across tasks.

5 October 2021
Verified
Book Chapter
Closed access
PDF file
UBUmang Bhatt
Umang Bhatt
ZSZohreh Shams
Zohreh Shams

18 March 2021 in arXiv
Verified
Preprint
Open access
PDF file
JAJavier Antorán
Javier Antorán
UBUmang Bhatt
Umang Bhatt
TATameem Adel
Tameem Adel
AWAdrian Weller
Adrian Weller
JLJose Miguel Hernandez Lobato
Jose Miguel Hernandez Lobato
Both uncertainty estimation and interpretability are important factors for trustworthy machine learning systems. However, there is little work at the intersection of these two areas. We address this gap by proposing a novel method for interpreting uncertainty estimates from differentiable probabilistic models, like Bayesian Neural Networks (BNNs). Our method, Counterfactual Latent Uncertainty Explanations (CLUE), indicates how to change an input, while keeping it on the data manifold, such that a BNN becomes more confident about the input's prediction. We validate CLUE through 1) a novel framework for evaluating counterfactual explanations of uncertainty, 2) a series of ablation experiments, and 3) a user study. Our experiments show that CLUE outperforms baselines and enables practitioners to better understand which input patterns are responsible for predictive uncertainty.

14 October 2022 in arXiv
Verified
Preprint
Open access
PDF file
YXYuxin Xiao
Yuxin Xiao
PLPaul Pu Liang
Paul Pu Liang
UBUmang Bhatt
Umang Bhatt
WNWillie Neiswanger
Willie Neiswanger
RSRuslan Salakhutdinov
Ruslan Salakhutdinov
LMLouis-Philippe Morency
Louis-Philippe Morency
Pre-trained language models (PLMs) have gained increasing popularity due to their compelling prediction performance in diverse natural language processing (NLP) tasks. When formulating a PLM-based prediction pipeline for NLP tasks, it is also crucial for the pipeline to minimize the calibration error, especially in safety-critical applications. That is, the pipeline should reliably indicate when we can trust its predictions. In particular, there are various considerations behind the pipeline: (1) the choice and (2) the size of PLM, (3) the choice of uncertainty quantifier, (4) the choice of fine-tuning loss, and many more. Although prior work has looked into some of these considerations, they usually draw conclusions based on a limited scope of empirical studies. There still lacks a holistic analysis on how to compose a well-calibrated PLM-based prediction pipeline. To fill this void, we compare a wide range of popular options for each consideration based on three prevalent NLP classification tasks and the setting of domain shift. In response, we recommend the following: (1) use ELECTRA for PLM encoding, (2) use larger PLMs if possible, (3) use Temp Scaling as the uncertainty quantifier, and (4) use Focal Loss for fine-tuning.

3 December 2021 in arXiv
Verified
Preprint
Open access
PDF file
DLDan Ley
Dan Ley
UBUmang Bhatt
Umang Bhatt
AWAdrian Weller
Adrian Weller
To interpret uncertainty estimates from differentiable probabilistic models, recent work has proposed generating Counterfactual Latent Uncertainty Explanations (CLUEs). However, for a single input, such approaches could output a variety of explanations due to the lack of constraints placed on the explanation. Here we augment the original CLUE approach, to provide what we call $\delta$-CLUE. CLUE indicates $\it{one}$ way to change an input, while remaining on the data manifold, such that the model becomes more confident about its prediction. We instead return a $\it{set}$ of plausible CLUEs: multiple, diverse inputs that are within a $\delta$ ball of the original input in latent space, all yielding confident predictions.