Michael Burnham

Michael Burnham

Auto-generated profile, based on publicly available information

Michael Burnham

Princeton University

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 September 2024 in Political Science Research and Methods
Verified
Journal Article
Open access
PDF file
MBMichael Burnham
Michael Burnham
Stance detection is identifying expressed beliefs in a document. While researchers widely use sentiment analysis for this, recent research demonstrates that sentiment and stance are distinct. This paper advances text analysis methods by precisely defining stance detection and outlining three approaches: supervised classification, natural language inference, and in-context learning. I discuss how document context and trade-offs between resources and workload should inform your methods. For all three approaches I provide guidance on application and validation techniques, as well as coding tutorials for implementation. Finally, I demonstrate how newer classification approaches can replicate supervised classifiers.

2 June 2022 in Social Science & Medicine
Verified
Journal Article
Closed access
RBRay Block
Ray Block
MBMichael Burnham
Michael Burnham
KKKayla Kahn
Kayla Kahn
RPRachel Peng
Rachel Peng
JSJeremy Seeman
Jeremy Seeman
CSChristopher Seto
Christopher Seto

15 December 2025 in Political Analysis
Verified
Journal Article
Open access
PDF file
MBMichael Burnham
Michael Burnham
KKKayla Kahn
Kayla Kahn
RWRyan Yang Wang
Ryan Yang Wang
RPRachel X. Peng
Rachel X. Peng
Social scientists have quickly adopted large language models (LLMs) for their ability to annotate documents without supervised training, an ability known as zero-shot classification. However, due to their computational demands, cost, and often proprietary nature, these models are frequently at odds with open science standards. This article introduces the Political Domain Enhanced BERT-based Algorithm for Textual Entailment (DEBATE) language models: Foundation models for zero-shot, few-shot, and supervised classification of political documents. As zero-shot classifiers, the models are designed to be used for common, well-defined tasks, such as topic and opinion classification. When used in this context, the DEBATE models are not only as good as state-of-the-art LLMs at zero-shot classification, but are orders of magnitude more efficient and completely open source. We further demonstrate that the models are effective few-shot learners. With a simple random sample of 10–25 documents, they can outperform supervised classifiers trained on hundreds or thousands of documents and state-of-the-art generative models. Additionally, we release the PolNLI dataset used to train these models—a corpus of over 200,000 political documents with highly accurate labels across over 800 classification tasks.

1 October 2024 in Research & Politics
Verified
Journal Article
Open access
MBMichael Burnham
Michael Burnham
Sentiment analysis is one of the most widely used text analysis methods in social science. Recent advancements with large language models have made it more accurate and accessible than ever, allowing researchers to classify text with only a plain English prompt. However, “sentiment” entails a wide variety of concepts depending on the domain and tools used. It has been used to mean emotion, opinions, market movements, or simply a general “good-bad” dimension. This raises a question: What exactly are language models doing when prompted to label documents by sentiment? This paper first overviews how sentiment is defined across different contexts, highlighting that it is a confounded measurement construct that entails multiple concepts, particularly emotional valence and opinion, without disentangling them. I then test three language models across two data sets with prompts requesting sentiment, valence, and stance classification. I find that language models generally understand sentiment in terms of emotional valence rather than opinion. I find that classification generally improves when researchers more precisely specify their dimension, but that even when classification metrics are similar the different prompts are biased in different ways. I conclude by encouraging researchers to move beyond sentiment when feasible and use more precise measurement constructs.

3 May 2024 in arXiv
Verified
Preprint
Open access
PDF file
MBMichael Burnham
Michael Burnham
This paper introduces "Semantic Scaling," a novel method for ideal point estimation from text. I leverage large language models to classify documents based on their expressed stances and extract survey-like data. I then use item response theory to scale subjects from these data. Semantic Scaling significantly improves on existing text-based scaling methods, and allows researchers to explicitly define the ideological dimensions they measure. This represents the first scaling approach that allows such flexibility outside of survey instruments and opens new avenues of inquiry for populations difficult to survey. Additionally, it works with documents of varying length, and produces valid estimates of both mass and elite ideology. I demonstrate that the method can differentiate between policy preferences and in-group/out-group affect. Among the public, Semantic Scaling out-preforms Tweetscores according to human judgement; in Congress, it recaptures the first dimension DW-NOMINATE while allowing for greater flexibility in resolving construct validity challenges.

3 May 2024 in arXiv
Verified
Preprint
Open access
PDF file
MBMichael Burnham
Michael Burnham
Sentiment analysis is one of the most widely used techniques in text analysis. Recent advancements with Large Language Models have made it more accurate and accessible than ever, allowing researchers to classify text with only a plain English prompt. However, "sentiment" entails a wide variety of concepts depending on the domain and tools used. It has been used to mean emotion, opinions, market movements, or simply a general ``good-bad'' dimension. This raises a question: What exactly are language models doing when prompted to label documents by sentiment? This paper first overviews how sentiment is defined across different contexts, highlighting that it is a confounded measurement construct in that it entails multiple variables, such as emotional valence and opinion, without disentangling them. I then test three language models across two data sets with prompts requesting sentiment, valence, and stance classification. I find that sentiment labels most strongly correlate with valence labels. I further find that classification improves when researchers more precisely specify their dimension of interest rather than using the less well-defined concept of sentiment. I conclude by encouraging researchers to move beyond "sentiment" when feasible and use a more precise measurement construct.