Allison Koenecke

Allison Koenecke

Auto-generated profile, based on publicly available information

Allison Koenecke

Cornell University, Ithaca, New York, USA

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 March 2020 in Proceedings of the National Academy of Sciences
Verified
Journal Article
Open access
PDF file
AKAllison Koenecke
Allison Koenecke
ANAndrew Nam
Andrew Nam
ELEmily Lake
Emily Lake
JNJoe Nudell
Joe Nudell
MQMinnie Quartey
Minnie Quartey
SGSharad Goel
Sharad Goel
Automated speech recognition (ASR) systems, which use sophisticated machine-learning algorithms to convert spoken language to text, have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems. There is concern, however, that these tools do not work equally well for all subgroups of the population. Here, we examine the ability of five state-of-the-art ASR systems—developed by Amazon, Apple, Google, IBM, and Microsoft—to transcribe structured interviews conducted with 42 white speakers and 73 black speakers. In total, this corpus spans five US cities and consists of 19.8 h of audio matched on the age and gender of the speaker. We found that all five ASR systems exhibited substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers. We trace these disparities to the underlying acoustic models used by the ASR systems as the race gap was equally large on a subset of identical phrases spoken by black and white individuals in our corpus. We conclude by proposing strategies—such as using more diverse training datasets that include African American Vernacular English—to reduce these performance differences and ensure speech recognition technology is inclusive.

1 July 2020 in Journal of Clinical Investigation
Verified
Journal Article
Open access
PDF file
Maximilian KonigMaximilian Konig
Maximilian Konig
MPMike Powell
Mike Powell
VSVerena Staedtke
Verena Staedtke
RBRenyuan Bai
Renyuan Bai
DTDavid L. Thomas
David L. Thomas
CBChetan Bettegowda
Chetan Bettegowda

3 June 2024
Verified
Conference Paper
Open access
AKAllison Koenecke
Allison Koenecke
ACAnna Seo Gyeong Choi
Anna Seo Gyeong Choi
KMKatelyn X. Mei
Katelyn X. Mei
HSHilke Schellmann
Hilke Schellmann
MSMona Sloane
Mona Sloane
Speech-to-text services aim to transcribe input audio as accurately as possible. They increasingly play a role in everyday life, for example in personal voice assistants or in customer-company interactions. We evaluate Open AI’s Whisper, a state-of-the-art automated speech recognition service outperforming industry competitors, as of 2023. While many of Whisper’s transcriptions were highly accurate, we find that roughly 1% of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio. We thematically analyze the Whisper-hallucinated content, finding that 38% of hallucinations include explicit harms such as perpetuating violence, making up inaccurate associations, or implying false authority. We then study why hallucinations occur by observing the disparities in hallucination rates between speakers with aphasia (who have a lowered ability to express themselves using speech and voice) and a control group. We find that hallucinations disproportionately occur for individuals who speak with longer shares of non-vocal durations—a common symptom of aphasia. We call on industry practitioners to ameliorate these language-model-based hallucinations in Whisper, and to raise awareness of potential biases amplified by hallucinations in downstream applications of speech-to-text models.


12 June 2023
Verified
Conference Paper
Open access
OPOrestis Papakyriakopoulos
Orestis Papakyriakopoulos
ACAnna Seo Gyeong Choi
Anna Seo Gyeong Choi
WTWilliam Thong
William Thong
DZDora Zhao
Dora Zhao
JAJerone Andrews
Jerone Andrews
AKAllison Koenecke
Allison Koenecke
Speech datasets are crucial for training Speech Language Technologies (SLT); however, the lack of diversity of the underlying training data can lead to serious limitations in building equitable and robust SLT products, especially along dimensions of language, accent, dialect, variety, and speech impairment—and the intersectionality of speech features with socioeconomic and demographic features. Furthermore, there is often a lack of oversight on the underlying training data—commonly built on massive web-crawling and/or publicly available speech—with regard to the ethics of such data collection. To encourage standardized documentation of such speech data components, we introduce an augmented datasheet for speech datasets1, which can be used in addition to “Datasheets for Datasets” [78]. We then exemplify the importance of each question in our augmented datasheet based on in-depth literature reviews of speech data used in domains such as machine learning, linguistics, and health. Finally, we encourage practitioners—ranging from dataset creators to researchers—to use our augmented datasheet to better define the scope, properties, and limits of speech datasets, while also encouraging consideration of data-subject protection and user community empowerment. Ethical dataset creation is not a one-size-fits-all process, but dataset creators can use our augmented datasheet to reflexively consider the social context of related SLT applications and data sources in order to foster more inclusive SLT products downstream.

11 June 2021 in eLife
Verified
Journal Article
Open access
AKAllison Koenecke
Allison Koenecke
MPMichael Powell
Michael Powell
RXRuoxuan Xiong
Ruoxuan Xiong
ZSZhu Shen
Zhu Shen
NFNicole Fischer
Nicole Fischer
SASusan Athey
Susan Athey
In severe viral pneumonia, including Coronavirus disease 2019 (COVID-19), the viral replication phase is often followed by hyperinflammation, which can lead to acute respiratory distress syndrome, multi-organ failure, and death. We previously demonstrated that alpha-1 adrenergic receptor (⍺1-AR) antagonists can prevent hyperinflammation and death in mice. Here, we conducted retrospective analyses in two cohorts of patients with acute respiratory distress (ARD, n = 18,547) and three cohorts with pneumonia (n = 400,907). Federated across two ARD cohorts, we find that patients exposed to ⍺1-AR antagonists, as compared to unexposed patients, had a 34% relative risk reduction for mechanical ventilation and death (OR = 0.70, p = 0.021). We replicated these methods on three pneumonia cohorts, all with similar effects on both outcomes. All results were robust to sensitivity analyses. These results highlight the urgent need for prospective trials testing whether prophylactic use of ⍺1-AR antagonists ameliorates lower respiratory tract infection-associated hyperinflammation and death, as observed in COVID-19.