Stefano Mizzaro

Stefano Mizzaro

Auto-generated profile, based on publicly available information

Stefano Mizzaro

University of Udine, Italy

Research Focus & Metrics

Academic Career Length

30Years (1997-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 September 1997 in Journal of the American Society for Information Science
Verified
Journal Article
Closed access
PDF file
SMStefano Mizzaro
Stefano Mizzaro

1 June 1998 in Interacting with Computers
Verified
Journal Article
Closed access
SMStefano Mizzaro
Stefano Mizzaro

1 November 2012 in Information Processing & Management
Verified
Journal Article
Closed access
OAOmar Alonso
Omar Alonso
SMStefano Mizzaro
Stefano Mizzaro

3 February 2024 in Information Processing & Management
Verified
Journal Article
Open access
MSMichael Soprano
Michael Soprano
KRKevin Roitero
Kevin Roitero
DBDavid La Barbera
David La Barbera
DCDavide Ceolin
Davide Ceolin
Damiano SpinaDamiano Spina
Damiano Spina
SMStefano Mizzaro
Stefano Mizzaro

4 January 2017 in ACM Transactions on Information Systems
Verified
Journal Article
Open access
EMEddy Maddalena
Eddy Maddalena
SMStefano Mizzaro
Stefano Mizzaro
FSFalk Scholer
Falk Scholer
ATAndrew Turpin
Andrew Turpin
Magnitude estimation is a psychophysical scaling technique for the measurement of sensation, where observers assign numbers to stimuli in response to their perceived intensity. We investigate the use of magnitude estimation for judging the relevance of documents for information retrieval evaluation, carrying out a large-scale user study across 18 TREC topics and collecting over 50,000 magnitude estimation judgments using crowdsourcing. Our analysis shows that magnitude estimation judgments can be reliably collected using crowdsourcing, are competitive in terms of assessor cost, and are, on average, rank-aligned with ordinal judgments made by expert relevance assessors. We explore the application of magnitude estimation for IR evaluation, calibrating two gain-based effectiveness metrics, nDCG and ERR, directly from user-reported perceptions of relevance. A comparison of TREC system effectiveness rankings based on binary, ordinal, and magnitude estimation relevance shows substantial variation; in particular, the top systems ranked using magnitude estimation and ordinal judgments differ substantially. Analysis of the magnitude estimation scores shows that this effect is due in part to varying perceptions of relevance: different users have different perceptions of the impact of relative differences in document relevance. These results have direct implications for IR evaluation, suggesting that current assumptions about a single view of relevance being sufficient to represent a population of users are unlikely to hold.

29 September 2013
Verified
Conference Paper
Open access
PDF file
LBLuca Busin
Luca Busin
SMStefano Mizzaro
Stefano Mizzaro
The evaluation of retrieval effectiveness has played and is playing a central role in Information Retrieval (IR). A specific issue is that there are literally dozens (most likely more than one hundred) IR effectiveness metrics, and counting. In this paper we propose an axiomatic approach to IR effectiveness metrics. We build on the notions of measure, measurement, and similarity; they allow us to provide a general definition of IR effectiveness metric. On this basis, we then propose and justify some axioms that every effectiveness metric should satisfy, and we derive some theorems from the axioms. We also discuss some future developments.