Naghmeh Farzi

Naghmeh Farzi

Auto-generated profile, based on publicly available information

Naghmeh Farzi

University of New Hampshire, Durham, NH, USA

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 August 2024
Verified
Conference Paper
Open access
NFNaghmeh Farzi
Naghmeh Farzi
LDLaura Dietz
Laura Dietz
Current IR evaluation paradigms are challenged by large language models (LLMs) and retrieval-augmented generation (RAG) methods. Furthermore, evaluation either resorts to expensive human judgments or lead to an over-reliance on LLMs. To remedy this situation, we introduce the RUBRIC metric, which puts information retrieval systems to the proverbial test. This metric leverages a bank of query-related test questions to quantify relevant information content that is contained in the systems' responses. The process involves (1) decomposing the query into detailed questions, and (2) checking each for answerability using passages in the system response. Using three TREC benchmarks, we demonstrate that our LLM-based RUBRIC approach works successfully. Unlike previous LLM-based evaluation measures, our paradigm lends itself for incorporating a human-in-the-loop while avoiding some pitfalls of over-reliance on AI or resorting to expensive manual passage-level judgments. Moreover, our evaluation is repeatable and extensible and can be scored with existing evaluation tools. Data and code at https://github.com/TREMA-UNH/rubric-evaluation/

18 July 2025
Verified
Conference Paper
Open access
NFNaghmeh Farzi
Naghmeh Farzi
LDLaura Dietz
Laura Dietz

13 July 2025
Verified
Conference Paper
Open access
NFNaghmeh Farzi
Naghmeh Farzi
LDLaura Dietz
Laura Dietz

22 March 2026
Verified
Conference Paper
Open access
NFNaghmeh Farzi
Naghmeh Farzi
LDLaura Dietz
Laura Dietz
DLDavid D. Lewis
David D. Lewis

19 July 2026
Verified
Conference Paper
Open access
NFNaghmeh Farzi
Naghmeh Farzi
THTim Hagen
Tim Hagen
EYEugene Yang
Eugene Yang
MFMaik Fröbe
Maik Fröbe
RPRonak Pradeep
Ronak Pradeep
LDLaura Dietz
Laura Dietz

19 July 2026
Verified
Conference Paper
Open access
LDLaura Dietz
Laura Dietz
NFNaghmeh Farzi
Naghmeh Farzi
EYEugene Yang
Eugene Yang
DLDawn Lawrie
Dawn Lawrie