Jheng-Hong Yang

Jheng-Hong Yang

Auto-generated profile, based on publicly available information

Jheng-Hong Yang

University of Waterloo, Waterloo, Canada

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 July 2024
Verified
Conference Paper
Open access
EKEhsan Kamalloo
Ehsan Kamalloo
NTNandan Thakur
Nandan Thakur
CLCarlos Lassance
Carlos Lassance
XMXueguang Ma
Xueguang Ma
JYJheng-Hong Yang
Jheng-Hong Yang
JLJimmy Lin
Jimmy Lin
BEIR is a benchmark dataset originally designed for zero-shot evaluation of retrieval models across 18 different domain/task combinations. In recent years, we have witnessed the growing popularity of models based on representation learning, which naturally begs the question: How effective are these models when presented with queries and documents that differ from the training data? While BEIR was designed to answer this question, our work addresses two shortcomings that prevent the benchmark from achieving its full potential: First, the sophistication of modern neural methods and the complexity of current software infrastructure create barriers to entry for newcomers. To this end, we provide reproducible reference implementations that cover learned dense and sparse models. Second, comparisons on BEIR are performed by reducing scores from heterogeneous datasets into a single average that is difficult to interpret. To remedy this, we present meta-analyses focusing on effect sizes across datasets that are able to accurately quantify model differences. By addressing both shortcomings, our work facilitates future explorations in a range of interesting research questions.

1 January 2022
Verified
Conference Paper
Open access
PDF file
WZWei Zhong
Wei Zhong
JYJheng-Hong Yang
Jheng-Hong Yang
YXYuqing Xie
Yuqing Xie
JLJimmy Lin
Jimmy Lin
Wei Zhong, Jheng-Hong Yang, Yuqing Xie, Jimmy Lin. Findings of the Association for Computational Linguistics: EMNLP 2022. 2022.

18 July 2023
Verified
Conference Paper
Open access
JYJheng-Hong Yang
Jheng-Hong Yang
CLCarlos Lassance
Carlos Lassance
RRRafael Sampaio De Rezende
Rafael Sampaio De Rezende
KSKrishna Srinivasan
Krishna Srinivasan
MRMiriam Redi
Miriam Redi
JLJimmy Lin
Jimmy Lin
This paper presents the AToMiC (Authoring Tools for Multi media Content) dataset, designed to advance research in image/text cross-modal retrieval. While vision--language pretrained transformers have led to significant improvements in retrieval effectiveness, existing research has relied on image-caption datasets that feature only simplistic image--text relationships and underspecified user models of retrieval tasks. To address the gap between these oversimplified settings and real-world applications for multimedia content creation, we introduce a new approach for building retrieval test collections. We leverage hierarchical structures and diverse domains of texts, styles, and types of images, as well as large-scale image--document associations embedded in Wikipedia. We formulate two tasks based on a realistic user model and validate our dataset through retrieval experiments using baseline models. AToMiC offers a testbed for scalable, diverse, and reproducible multimedia retrieval research. Finally, our dataset provides the basis for a dedicated track at the 2023 Text Retrieval Conference (TREC), and is publicly available at https://github.com/TREC-AToMiC/AToMiC.


13 July 2025
Verified
Conference Paper
Open access
JLJimmy Lin
Jimmy Lin
ACArthur Haonan Chen
Arthur Haonan Chen
CLCarlos Lassance
Carlos Lassance
XMXueguang Ma
Xueguang Ma
RPRonak Pradeep
Ronak Pradeep
VZVincent Zhong
Vincent Zhong