Mark Hasegawa-Johnson

Dr. Mark Hasegawa-Johnson

orcid

Research Focus & Metrics

Academic Career Length

29Years (1998-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 February 2010 in Pattern Recognition Letters
Verified
Journal Article
Closed access
XZXiaodan Zhuang
Xiaodan Zhuang
XZXi Zhou
Xi Zhou
MHMark Hasegawa-Johnson
Mark Hasegawa-Johnson
THThomas S. Huang
Thomas S. Huang

1 January 2010 in Laboratory Phonology
Verified
Journal Article
Closed access
JCJennifer Cole
Jennifer Cole
YMYoonsook Mo
Yoonsook Mo
MHMark Hasegawa-Johnson
Mark Hasegawa-Johnson
The perception of prosodic prominence in spontaneous speech is investigated through an online task of prosody transcription using untrained listeners. Prominence is indexed through a probabilistic prominence score assigned to each word based on the proportion of transcribers who perceived the word as prominent. Correlation and regression analyses between perceived prominence, acoustic measures and measures of a word's information status are conducted to test three hypotheses: (i) prominence perception is signal-driven, influenced by acoustic factors reflecting speakers' productions; (ii) perception is expectation-driven, influenced by the listener's prior experience of word frequency and repetition; (iii) any observed influence of word frequency on perceived prominence is mediated through the acoustic signal. Results show correlates of perceived prominence in acoustic measures, in word log-frequency and in the repetition index of a word, consistent with both signal-driven and expectation-driven hypotheses of prominence perception. But the acoustic correlates of perceived prominence differ somewhat from the correlates of word frequency, suggesting an independent effect of frequency on prominence perception. A speech processing account is offered as a model of signal-driven and expectation-driven effects on prominence perception, where prominence ratings are a function of the ease of lexical processing, as measured through the activation levels of lexical and sub-lexical units.

12 October 2010 in Folia Phoniatrica et Logopaedica
Verified
Journal Article
Closed access
PDF file
HKHeejin Kim
Heejin Kim
MHMark Hasegawa-Johnson
Mark Hasegawa-Johnson
APAdrienne Perlman
Adrienne Perlman
Background/Aims: This study examined the spectral characteristics of American English vowels in dysarthria associated with cerebral palsy (CP), and investigated the relationship between a speaker’s overall speech intelligibility and vowel contrast. Methods: The data were collected from 12 American English native speakers (9 speakers with a diagnosis of CP and 3 controls). Primary measures were F1 and F2 frequencies of 3 corner vowels /i, a, u/ and 3 noncorner vowels /I, 3, */. Six acoustic variables were derived from the formant measures, and were regressed against intelligibility: corner vowel space, noncorner vowel space, mean distance between vowels, F1 and F2 variability, and overlap degree among vowels. Results: First, the effect of vowel was significant for both F1 and F2 measures for all speakers, but post hoc analysis revealed a reduced distinction at lower intelligibility. Second, regression functions relating intelligibility and acoustic variables were significant for overlap degree among vowels, F1 variability, corner vowel space and mean distance between vowels. Overlap degree among vowels accounted for the greatest amount of variance in intelligibility scores. Conclusion: A speaker’s overall intelligibility in dysarthric speech is better represented by the overlap degree among vowels than by the vowel space.

7 November 2024 in Journal of Speech, Language, and Hearing Research
Verified
Journal Article
Open access
MHMark Hasegawa-Johnson
Mark Hasegawa-Johnson
XZXiuwen Zheng
Xiuwen Zheng
HKHeejin Kim
Heejin Kim
CMClarion Mendes
Clarion Mendes
MDMeg Dickinson
Meg Dickinson
BMBob MacDonald
Bob MacDonald
Purpose: The Speech Accessibility Project (SAP) intends to facilitate research and development in automatic speech recognition (ASR) and other machine learning tasks for people with speech disabilities. The purpose of this article is to introduce this project as a resource for researchers, including baseline analysis of the first released data package. Method: The project aims to facilitate ASR research by collecting, curating, and distributing transcribed U.S. English speech from people with speech and/or language disabilities. Participants record speech from their place of residence by connecting their personal computer, cell phone, and assistive devices, if needed, to the SAP web portal. All samples are manually transcribed, and 30 per participant are annotated using differential diagnostic pattern dimensions. For purposes of ASR experiments, the participants have been randomly assigned to a training set, a development set for controlled testing of a trained ASR, and a test set to evaluate ASR error rate. Results: The SAP 2023-10-05 Data Package contains the speech of 211 people with dysarthria as a correlate of Parkinson's disease, and the associated test set contains 42 additional speakers. A baseline ASR, with a word error rate of 3.4% for typical speakers, transcribes test speech with a word error rate of 36.3%. Fine-tuning reduces the word error rate to 23.7%. Conclusions: Preliminary findings suggest that a large corpus of dysarthric and dysphonic speech has the potential to significantly improve speech technology for people with disabilities. By providing these data to researchers, the SAP intends to significantly accelerate research into accessible speech technology. Supplemental Material: https://doi.org/10.23641/asha.27078079

10 November 2021 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Open access
LSLeda Sarı
Leda Sarı
MHMark Hasegawa-Johnson
Mark Hasegawa-Johnson
CYChang D. Yoo
Chang D. Yoo
Widelyused automatic speech recognition (ASR) systems have been empirically demonstrated in various studies to be unfair, having higher error rates for some groups of users than others. One way to define fairness in ASR is to require that changing the demographic group affiliation of any individual (e.g., changing their gender, age, education or race) should not change the probability distribution across possible speech-to-text transcriptions. In the paradigm of counterfactual fairness, all variables independent of group affiliation (e.g., the text being read by the speaker) remain unchanged, while variables dependent on group affiliation (e.g., the speaker’s voice) are counterfactually modified. Hence, we approach the fairness of ASR by training the ASR to minimize change in its outcome probabilities despite a counterfactual change in the individual’s demographic attributes. Starting from the individualized counterfactual equal odds criterion, we provide relaxations to it and compare their performances for connectionist temporal classification (CTC) based end-to-end ASR systems. We perform our experiments on the Corpus of Regional African American Language (CORAAL) and the LibriSpeech dataset to accommodate for differences due to gender, age, education, and race. We show that with counterfactual training, we can reduce average character error rates while achieving lower performance gap between demographic groups, and lower error standard deviation among individuals.

13 February 2020 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
OSOdette Scharenborg
Odette Scharenborg
LBLaurent Besacier
Laurent Besacier
ABAlan W. Black
Alan W. Black
MHMark Hasegawa-Johnson
Mark Hasegawa-Johnson
FMFlorian Metze
Florian Metze
EDEmmanuel Dupoux
Emmanuel Dupoux
Speech technology plays an important role in our everyday life. Among others, speech is used for human-computer interaction, for instance for information retrieval and on-line shopping. In the case of an unwritten language, however, speech technology is unfortunately difficult to create, because it cannot be created by the standard combination of pre-trained speech-to-text and text-to-speech subsystems. The research presented in this paper takes the first steps towards speech technology for unwritten languages. Specifically, the aim of this work was 1) to learn speech-to-meaning representations without using text as an intermediate representation, and 2) to test the sufficiency of the learned representations to regenerate speech or translated text, or to retrieve images that depict the meaning of an utterance in an unwritten language. The results suggest that building systems that go directly from speech-to-meaning and from meaning-to-speech, bypassing the need for text, is possible.