Mert Yuksekgonul

Mert Yuksekgonul

Auto-generated profile, based on publicly available information

Mert Yuksekgonul

Stanford University

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 August 2023 in Nature Medicine
Verified
Journal Article
Closed access
PDF file
ZHZhi Huang
Zhi Huang
FBFederico Bianchi
Federico Bianchi
MYMert Yuksekgonul
Mert Yuksekgonul
TMThomas J. Montine
Thomas J. Montine
JZJames Y Zou
James Y Zou

19 March 2025 in Nature
Verified
Journal Article
Closed access
PDF file
MYMert Yuksekgonul
Mert Yuksekgonul
FBFederico Bianchi
Federico Bianchi
JBJoseph Boen
Joseph Boen
SLSheng Liu
Sheng Liu
PLPan Lu
Pan Lu
JZJames Y Zou
James Y Zou

1 April 2023 in bioRxiv
Verified
Preprint
Open access
PDF file
ZHZhi Huang
Zhi Huang
FBFederico Bianchi
Federico Bianchi
MYMert Yuksekgonul
Mert Yuksekgonul
TMThomas Montine
Thomas Montine
JZJames Y Zou
James Y Zou
The lack of annotated publicly available medical images is a major barrier for innovations. At the same time, many de-identified images and much knowledge are shared by clinicians on public forums such as medical Twitter. Here we harness these crowd platforms to curate OpenPath, a large dataset of 208,414 pathology images paired with natural language descriptions. This is the largest public dataset for pathology images annotated with natural text. We demonstrate the value of this resource by developing PLIP, a multimodal AI with both image and text understanding, which is trained on OpenPath. PLIP achieves state-of-the-art zero-shot and transfer learning performances for classifying new pathology images across diverse tasks. Moreover, PLIP enables users to retrieve similar cases by either image or natural language search, greatly facilitating knowledge sharing. Our approach demonstrates that publicly shared medical information is a tremendous resource that can be harnessed to advance biomedical AI.

23 February 2026 in Nature Machine Intelligence
Verified
Journal Article
Closed access
PDF file
NTNitya Thakkar
Nitya Thakkar
MYMert Yuksekgonul
Mert Yuksekgonul
JSJake Silberg
Jake Silberg
AGAnimesh Garg
Animesh Garg
NPNanyun Peng
Nanyun Peng
JZJames Y Zou
James Y Zou

1 February 2023 in arXiv
Verified
Preprint
Open access
PDF file
MYMert Yuksekgonul
Mert Yuksekgonul
MWMaggie Wang
Maggie Wang
JZJames Y Zou
James Y Zou
Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts (``the bottleneck'') and use the concepts to make predictions. A concept bottleneck enhances interpretability since it can be investigated to understand what concepts the model "sees" in an input and which of these concepts are deemed important. However, CBMs are restrictive in practice as they require dense concept annotations in the training data to learn the bottleneck. Moreover, CBMs often do not match the accuracy of an unrestricted neural network, reducing the incentive to deploy them in practice. In this work, we address these limitations of CBMs by introducing Post-hoc Concept Bottleneck models (PCBMs). We show that we can turn any neural network into a PCBM without sacrificing model performance while still retaining the interpretability benefits. When concept annotations are not available on the training data, we show that PCBM can transfer concepts from other datasets or from natural language descriptions of concepts via multimodal models. A key benefit of PCBM is that it enables users to quickly debug and update the model to reduce spurious correlations and improve generalization to new distributions. PCBM allows for global model edits, which can be more efficient than previous works on local interventions that fix a specific prediction. Through a model-editing user study, we show that editing PCBMs via concept-level feedback can provide significant performance gains without using data from the target domain or model retraining.

24 March 2023 in arXiv
Verified
Preprint
Open access
PDF file
MYMert Yuksekgonul
Mert Yuksekgonul
FBFederico Bianchi
Federico Bianchi
PKPratyusha Kalluri
Pratyusha Kalluri
DJDan Jurafsky
Dan Jurafsky
JZJames Y Zou
James Y Zou
Despite the success of large vision and language models (VLMs) in many downstream applications, it is unclear how well they encode compositional information. Here, we create the Attribution, Relation, and Order (ARO) benchmark to systematically evaluate the ability of VLMs to understand different types of relationships, attributes, and order. ARO consists of Visual Genome Attribution, to test the understanding of objects' properties; Visual Genome Relation, to test for relational understanding; and COCO & Flickr30k-Order, to test for order sensitivity. ARO is orders of magnitude larger than previous benchmarks of compositionality, with more than 50,000 test cases. We show where state-of-the-art VLMs have poor relational understanding, can blunder when linking objects to their attributes, and demonstrate a severe lack of order sensitivity. VLMs are predominantly trained and evaluated on large datasets with rich compositional structure in the images and captions. Yet, training on these datasets has not been enough to address the lack of compositional understanding, and evaluating on these datasets has failed to surface this deficiency. To understand why these limitations emerge and are not represented in the standard tests, we zoom into the evaluation and training procedures. We demonstrate that it is possible to perform well on retrieval over existing datasets without using the composition and order information. Given that contrastive pretraining optimizes for retrieval on datasets with similar shortcuts, we hypothesize that this can explain why the models do not need to learn to represent compositional information. This finding suggests a natural solution: composition-aware hard negative mining. We show that a simple-to-implement modification of contrastive learning significantly improves the performance on tasks requiring understanding of order and compositionality.