Hiroaki Hayashi

Hiroaki Hayashi

Auto-generated profile, based on publicly available information

Hiroaki Hayashi

Language Technologies Institute Carnegie Mellon University

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 January 2023 in ACM Computing Surveys
Verified
Journal Article
Open access
PLPengfei Liu
Pengfei Liu
WYWeizhe Yuan
Weizhe Yuan
JFJinlan Fu
Jinlan Fu
ZJZhengbao Jiang
Zhengbao Jiang
HHHiroaki Hayashi
Hiroaki Hayashi
GNGraham Neubig
Graham Neubig
This article surveys and organizes research works in a new paradigm in natural language processing, which we dub “prompt-based learning.” Unlike traditional supervised learning, which trains a model to take in an input x and predict an output y as P ( y|x ), prompt-based learning is based on language models that model the probability of text directly. To use these models to perform prediction tasks, the original input x is modified using a template into a textual string prompt x′ that has some unfilled slots, and then the language model is used to probabilistically fill the unfilled information to obtain a final string x̂ , from which the final output y can be derived. This framework is powerful and attractive for a number of reasons: It allows the language model to be pre-trained on massive amounts of raw text, and by defining a new prompting function the model is able to perform few-shot or even zero-shot learning, adapting to new scenarios with few or no labeled data. In this article, we introduce the basics of this promising paradigm, describe a unified set of mathematical notations that can cover a wide variety of existing work, and organize existing work along several dimensions, e.g., the choice of pre-trained language models, prompts, and tuning strategies. To make the field more accessible to interested beginners, we not only make a systematic review of existing works and a highly structured typology of prompt-based concepts but also release other resources, e.g., a website NLPedia–Pretrain including constantly updated survey and paperlist.

11 March 2021 in Transactions of the Association for Computational Linguistics
Verified
Journal Article
Open access
PDF file
HHHiroaki Hayashi
Hiroaki Hayashi
PBPrashant Budania
Prashant Budania
PWPeng Wang
Peng Wang
CAChris Ackerson
Chris Ackerson
RNRaj Neervannan
Raj Neervannan
GNGraham Neubig
Graham Neubig
Aspect-based summarization is the task of generating focused summaries based on specific points of interest. Such summaries aid efficient analysis of text, such as quickly understanding reviews or opinions from different angles. However, due to large differences in the type of aspects for different domains (e.g., sentiment, product features), the development of previous models has tended to be domain-specific. In this paper, we propose WikiAsp, 1 a large-scale dataset for multi-domain aspect- based summarization that attempts to spur research in the direction of open-domain aspect-based summarization. Specifically, we build the dataset using Wikipedia articles from 20 different domains, using the section titles and boundaries of each article as a proxy for aspect annotation. We propose several straightforward baseline models for this task and conduct experiments on the dataset. Results highlight key challenges that existing summarization models face in this setting, such as proper pronoun handling of quoted sources and consistent explanation of time-sensitive events.

27 October 2017 in Machine Translation
Verified
Journal Article
Closed access
PDF file
PLPatrick Littell
Patrick Littell
TTTian Tian
Tian Tian
RXRuochen Xu
Ruochen Xu
ZSZaid Sheikh
Zaid Sheikh
DMDavid Mortensen
David Mortensen
EHEduard Hovy
Eduard Hovy

19 April 2021 in arXiv
Verified
Preprint
Open access
PDF file
ZDZi-Yi Dou
Zi-Yi Dou
PLPengfei Liu
Pengfei Liu
HHHiroaki Hayashi
Hiroaki Hayashi
ZJZhengbao Jiang
Zhengbao Jiang
GNGraham Neubig
Graham Neubig
Neural abstractive summarization models are flexible and can produce coherent summaries, but they are sometimes unfaithful and can be difficult to control. While previous studies attempt to provide different types of guidance to control the output and increase faithfulness, it is not clear how these strategies compare and contrast to each other. In this paper, we propose a general and extensible guided summarization framework (GSum) that can effectively take different kinds of external guidance as input, and we perform experiments across several different varieties. Experiments demonstrate that this model is effective, achieving state-of-the-art performance according to ROUGE on 4 popular summarization datasets when using highlighted sentences as guidance. In addition, we show that our guided model can generate more faithful summaries and demonstrate how different types of guidance generate qualitatively different summaries, lending a degree of controllability to the learned models.

16 November 2020 in arXiv
Verified
Preprint
Open access
PDF file
HHHiroaki Hayashi
Hiroaki Hayashi
PBPrashant Budania
Prashant Budania
PWPeng Wang
Peng Wang
CAChris Ackerson
Chris Ackerson
RNRaj Neervannan
Raj Neervannan
GNGraham Neubig
Graham Neubig
Aspect-based summarization is the task of generating focused summaries based on specific points of interest. Such summaries aid efficient analysis of text, such as quickly understanding reviews or opinions from different angles. However, due to large differences in the type of aspects for different domains (e.g., sentiment, product features), the development of previous models has tended to be domain-specific. In this paper, we propose WikiAsp, a large-scale dataset for multi-domain aspect-based summarization that attempts to spur research in the direction of open-domain aspect-based summarization. Specifically, we build the dataset using Wikipedia articles from 20 different domains, using the section titles and boundaries of each article as a proxy for aspect annotation. We propose several straightforward baseline models for this task and conduct experiments on the dataset. Results highlight key challenges that existing summarization models face in this setting, such as proper pronoun handling of quoted sources and consistent explanation of time-sensitive events.

30 August 2019 in arXiv
Verified
Preprint
Open access
PDF file
DKDongyeop Kang
Dongyeop Kang
HHHiroaki Hayashi
Hiroaki Hayashi
ABAlan W Black
Alan W Black
EHEduard Hovy
Eduard Hovy
Generating a long, coherent text such as a paragraph requires a high-level control of different levels of relations between sentences (e.g., tense, coreference). We call such a logical connection between sentences as a (paragraph) flow. In order to produce a coherent flow of text, we explore two forms of intersentential relations in a paragraph: one is a human-created linguistical relation that forms a structure (e.g., discourse tree) and the other is a relation from latent representation learned from the sentences themselves. Our two proposed models incorporate each form of relations into document-level language models: the former is a supervised model that jointly learns a language model as well as discourse relation prediction, and the latter is an unsupervised model that is hierarchically conditioned by a recurrent neural network (RNN) over the latent information. Our proposed models with both forms of relations outperform the baselines in partially conditioned paragraph generation task. Our codes and data are publicly available.