Pradeep Dasigi

Pradeep Dasigi

Auto-generated profile, based on publicly available information

Pradeep Dasigi

Allen Institute for AI

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024
Verified
Conference Paper
Open access
PDF file
APArkil Patel
Arkil Patel
SRSiva Reddy
Siva Reddy
DBDzmitry Bahdanau
Dzmitry Bahdanau
PDPradeep Dasigi
Pradeep Dasigi
Arkil Patel, Siva Reddy, Dzmitry Bahdanau, Pradeep Dasigi. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

1 January 2020
Verified
Conference Paper
Open access
PDF file
QNQiang Ning
Qiang Ning
HWHao Wu
Hao Wu
PDPradeep Dasigi
Pradeep Dasigi
DDDheeru Dua
Dheeru Dua
MGMatt Gardner
Matt Gardner
ZNZhen Nie
Zhen Nie
Qiang Ning, Hao Wu, Pradeep Dasigi, Dheeru Dua, Matt Gardner, Robert L. Logan IV, Ana Marasović, Zhen Nie. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2020.

13 July 2025
Verified
Conference Paper
Open access
RRRevanth Gangi Reddy
Revanth Gangi Reddy
PDPradeep Dasigi
Pradeep Dasigi
ASArafat Sultan
Arafat Sultan
ACArman Cohan
Arman Cohan
ASAvirup Sil
Avirup Sil
HHHannaneh Hajishirzi
Hannaneh Hajishirzi

30 January 2023 in arXiv
Verified
Preprint
Open access
PDF file
KKKalpesh Krishna
Kalpesh Krishna
EBErin Bransom
Erin Bransom
BKBailey Kuehl
Bailey Kuehl
MIMohit Iyyer
Mohit Iyyer
PDPradeep Dasigi
Pradeep Dasigi
KLKyle Lo
Kyle Lo
While human evaluation remains best practice for accurately judging the faithfulness of automatically-generated summaries, few solutions exist to address the increased difficulty and workload when evaluating long-form summaries. Through a survey of 162 papers on long-form summarization, we first shed light on current human evaluation practices surrounding long-form summaries. We find that 73% of these papers do not perform any human evaluation on model-generated summaries, while other works face new difficulties that manifest when dealing with long documents (e.g., low inter-annotator agreement). Motivated by our survey, we present LongEval, a set of guidelines for human evaluation of faithfulness in long-form summaries that addresses the following challenges: (1) How can we achieve high inter-annotator agreement on faithfulness scores? (2) How can we minimize annotator workload while maintaining accurate faithfulness scores? and (3) Do humans benefit from automated alignment between summary and source snippets? We deploy LongEval in annotation studies on two long-form summarization datasets in different domains (SQuALITY and PubMed), and we find that switching to a finer granularity of judgment (e.g., clause-level) reduces inter-annotator variance in faithfulness scores (e.g., std-dev from 18.5 to 6.8). We also show that scores from a partial annotation of fine-grained units highly correlates with scores from a full annotation workload (0.89 Kendall's tau using 50% judgments). We release our human judgments, annotation templates, and our software as a Python library for future research.

25 May 2023 in arXiv
Verified
Preprint
Open access
PDF file
HIHamish Ivison
Hamish Ivison
NSNoah A. Smith
Noah A. Smith
HHHannaneh Hajishirzi
Hannaneh Hajishirzi
PDPradeep Dasigi
Pradeep Dasigi
Obtaining labeled data to train a model for a task of interest is often expensive. Prior work shows training models on multitask data augmented with task descriptions (prompts) effectively transfers knowledge to new tasks. Towards efficiently building task-specific models, we assume access to a small number (32-1000) of unlabeled target-task examples and use those to retrieve the most similar labeled examples from a large pool of multitask data augmented with prompts. Compared to the current practice of finetuning models on uniformly sampled prompted multitask data (e.g.: FLAN, T0), our approach of finetuning on cross-task nearest neighbors is significantly more data-efficient. Using only 2% of the data from the P3 pool without any labeled target-task data, our models outperform strong baselines trained on all available data by 3-30% on 12 out of 14 datasets representing held-out tasks including legal and scientific document QA. Similarly, models trained on cross-task nearest neighbors from SuperNaturalInstructions, representing about 5% of the pool, obtain comparable performance to state-of-the-art models on 12 held-out tasks from that pool. Moreover, the models produced by our approach also provide a better initialization than single multitask finetuned models for few-shot finetuning on target-task data, as shown by a 2-23% relative improvement over few-shot finetuned T0-3B models on 8 datasets.

13 November 2020 in arXiv
Verified
Preprint
Open access
PDF file
JFJames Ferguson
James Ferguson
MGMatt Gardner
Matt Gardner
HHHannaneh Hajishirzi
Hannaneh Hajishirzi
TKTushar Khot
Tushar Khot
PDPradeep Dasigi
Pradeep Dasigi
Humans often have to read multiple documents to address their information needs. However, most existing reading comprehension (RC) tasks only focus on questions for which the contexts provide all the information required to answer them, thus not evaluating a system's performance at identifying a potential lack of sufficient information and locating sources for that information. To fill this gap, we present a dataset, IIRC, with more than 13K questions over paragraphs from English Wikipedia that provide only partial information to answer them, with the missing information occurring in one or more linked documents. The questions were written by crowd workers who did not have access to any of the linked documents, leading to questions that have little lexical overlap with the contexts where the answers appear. This process also gave many questions without answers, and those that require discrete reasoning, increasing the difficulty of the task. We follow recent modeling work on various reading comprehension datasets to construct a baseline model for this dataset, finding that it achieves 31.1% F1 on this task, while estimated human performance is 88.4%. The dataset, code for the baseline system, and a leaderboard can be found at https://allennlp.org/iirc.