Eduard Hovy

Eduard Hovy

Auto-generated profile, based on publicly available information

Eduard Hovy

Carnegie Mellon University

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 December 2021 in Transactions of the Association for Computational Linguistics
Verified
Journal Article
Open access
PDF file
YEYanai Elazar
Yanai Elazar
NKNora Kassner
Nora Kassner
SRShauli Ravfogel
Shauli Ravfogel
ARAbhilasha Ravichander
Abhilasha Ravichander
EHEduard Hovy
Eduard Hovy
YGYoav Goldberg
Yoav Goldberg
Consistency of a model—that is, the invariance of its behavior under meaning-preserving alternations in its input—is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge? To this end, we create ParaRel🤘, a high-quality resource of cloze-style query English paraphrases. It contains a total of 328 paraphrases for 38 relations. Using ParaRel🤘, we show that the consistency of all PLMs we experiment with is poor— though with high variance between relations. Our analysis of the representational spaces of PLMs suggests that they have a poor structure and are currently not suitable for representing knowledge robustly. Finally, we propose a method for improving model consistency and experimentally demonstrate its effectiveness. 1

1 January 2022
Verified
Conference Paper
Open access
PDF file
ZZZhisong Zhang
Zhisong Zhang
ESEmma Strubell
Emma Strubell
EHEduard Hovy
Eduard Hovy
Zhisong Zhang, Emma Strubell, Eduard Hovy. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.

1 January 2019
Verified
Conference Paper
Open access
PDF file
VVVaibhav Vaibhav
Vaibhav Vaibhav
RMRaghuram Mandyam
Raghuram Mandyam
EHEduard Hovy
Eduard Hovy
Vaibhav Vaibhav, Raghuram Mandyam, Eduard Hovy. Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13). 2019.

1 January 2014
Verified
Conference Paper
Closed access
SPSameer Pradhan
Sameer Pradhan
XLXiaoqiang Luo
Xiaoqiang Luo
MRMarta Recasens
Marta Recasens
EHEduard Hovy
Eduard Hovy
VNVincent Ng
Vincent Ng
MSMichael Strube
Michael Strube
The definitions of two coreference scoring metrics— B3 and CEAF—are underspecified with respect to predicted, as opposed to key (or gold) mentions. Several variations have been proposed that manipulate either, or both, the key and predicted mentions in order to get a one-to-one mapping. On the other hand, the metric BLANC was, until recently, limited to scoring partitions of key mentions. In this paper, we (i) argue that mention manipulation for scoring predicted mentions is unnecessary, and potentially harmful as it could produce unintuitive results; (ii) illustrate the application of all these measures to scoring predicted mentions; (iii) make available an open-source, thoroughly-tested reference implementation of the main coreference evaluation measures; and (iv) rescore the results of the CoNLL-2011/2012 shared task systems with this implementation. This will help the community accurately measure and compare new end-to-end coreference resolution algorithms.

1 January 2019
Verified
Conference Paper
Open access
PDF file
ARAbhilasha Ravichander
Abhilasha Ravichander
ANAakanksha Naik
Aakanksha Naik
CRCarolyn Rose
Carolyn Rose
EHEduard Hovy
Eduard Hovy
Abhilasha Ravichander, Aakanksha Naik, Carolyn Rose, Eduard Hovy. Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). 2019.

1 January 2014
Verified
Conference Paper
Open access
JLJiwei Li
Jiwei Li
ARAlan Ritter
Alan Ritter
CCClaire Cardie
Claire Cardie
EHEduard Hovy
Eduard Hovy