Paul McNamee

Paul McNamee

Auto-generated profile, based on publicly available information

Paul McNamee

Johns Hopkins University, Baltimore, MD, USA

Research Focus & Metrics

Academic Career Length

26Years (2001-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 April 2022
Verified
Book Chapter
Closed access
PDF file
SNSuraj Nair
Suraj Nair
EYEugene Yang
Eugene Yang
DLDawn Lawrie
Dawn Lawrie
KDKevin Duh
Kevin Duh
PMPaul McNamee
Paul McNamee
DODouglas W. Oard
Douglas W. Oard

10 July 2024
Verified
Conference Paper
Open access
JMJames Mayfield
James Mayfield
EYEugene Yang
Eugene Yang
DLDawn Lawrie
Dawn Lawrie
SMSean MacAvaney
Sean MacAvaney
PMPaul McNamee
Paul McNamee
NHNoah Hibbler
Noah Hibbler
Large Language Models (LLMs) have enabled new ways to satisfy information needs. Although great strides have been made in applying them to settings like document ranking and short-form text generation, they still struggle to compose complete, accurate, and verifiable long-form reports. Reports with these qualities are necessary to satisfy the complex, nuanced, or multi-faceted information needs of users. In this perspective paper, we draw together opinions from industry and academia, and from a variety of related research areas, to present our vision for automatic report generation, and---critically---a flexible framework by which such reports can be evaluated. In contrast with other summarization tasks, automatic report generation starts with a detailed description of an information need, stating the necessary background, requirements, and scope of the report. Further, the generated reports should be complete, accurate, and verifiable. These qualities, which are desirable---if not required---in many analytic report-writing settings, require rethinking how to build and evaluate systems that exhibit these qualities. To foster new efforts in building these systems, we present an evaluation framework that draws on ideas found in various evaluations. To test completeness and accuracy, the framework uses nuggets of information, expressed as questions and answers, that need to be part of any high-quality generated report. Additionally, evaluation of citations that map claims made in the report to their source documents ensures verifiability.

1 September 2001 in Journal of the American Dietetic Association
Verified
Journal Article
Closed access
LKL.R. Kletzky
L.R. Kletzky
GEG.B. Espinosa-Hall
G.B. Espinosa-Hall
PMP.P. McNamee
P.P. McNamee