Federica Sarro

Federica Sarro

Auto-generated profile, based on publicly available information

Federica Sarro

University College London, London, United Kingdom

Research Focus & Metrics

Academic Career Length

17Years (2010-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 June 2024 in ACM Journal on Responsible Computing
Verified
Journal Article
Open access
MHMax Hort
Max Hort
ZCZhenpeng Chen
Zhenpeng Chen
JZJie M. Zhang
Jie M. Zhang
MHMark Harman
Mark Harman
FSFederica Sarro
Federica Sarro
This article provides a comprehensive survey of bias mitigation methods for achieving fairness in Machine Learning (ML) models. We collect a total of 341 publications concerning bias mitigation for ML classifiers. These methods can be distinguished based on their intervention procedure (i.e., pre-processing, in-processing, post-processing) and the technique they apply. We investigate how existing bias mitigation methods are evaluated in the literature. In particular, we consider datasets, metrics, and benchmarking. Based on the gathered insights (e.g., What is the most popular fairness metric? How many datasets are used for evaluating bias mitigation methods?), we hope to support practitioners in making informed choices when developing and evaluating new bias mitigation methods.

4 June 2024 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
ZCZhenpeng Chen
Zhenpeng Chen
JZJie M. Zhang
Jie M. Zhang
MHMax Hort
Max Hort
MHMark Harman
Mark Harman
FSFederica Sarro
Federica Sarro
Unfair behaviors of Machine Learning (ML) software have garnered increasing attention and concern among software engineers. To tackle this issue, extensive research has been dedicated to conducting fairness testing of ML software, and this article offers a comprehensive survey of existing studies in this field. We collect 100 papers and organize them based on the testing workflow (i.e., how to test) and testing components (i.e., what to test). Furthermore, we analyze the research focus, trends, and promising directions in the realm of fairness testing. We also identify widely adopted datasets and open-source tools for fairness testing.

27 May 2023 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
ZCZhenpeng Chen
Zhenpeng Chen
JZJie M. Zhang
Jie M. Zhang
FSFederica Sarro
Federica Sarro
MHMark Harman
Mark Harman
Software bias is an increasingly important operational concern for software engineers. We present a large-scale, comprehensive empirical study of 17 representative bias mitigation methods for Machine Learning (ML) classifiers, evaluated with 11 ML performance metrics (e.g., accuracy), 4 fairness metrics, and 20 types of fairness-performance tradeoff assessment, applied to 8 widely-adopted software decision tasks. The empirical coverage is much more comprehensive, covering the largest numbers of bias mitigation methods, evaluation metrics, and fairness-performance tradeoff measures compared to previous work on this important software property. We find that (1) the bias mitigation methods significantly decrease ML performance in 53% of the studied scenarios (ranging between 42%∼66% according to different ML performance metrics); (2) the bias mitigation methods significantly improve fairness measured by the 4 used metrics in 46% of all the scenarios (ranging between 24%∼59% according to different fairness metrics); (3) the bias mitigation methods even lead to decrease in both fairness and ML performance in 25% of the scenarios; (4) the effectiveness of the bias mitigation methods depends on tasks, models, the choice of protected attributes, and the set of metrics used to assess fairness and ML performance; (5) there is no bias mitigation method that can achieve the best tradeoff in all the scenarios. The best method that we find outperforms other methods in 30% of the scenarios. Researchers and practitioners need to choose the bias mitigation method best suited to their intended application scenario(s).

6 April 2021 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
MHMax Hort
Max Hort
MKMaria Kechagia
Maria Kechagia
FSFederica Sarro
Federica Sarro
MHMark Harman
Mark Harman
To ensure user satisfaction and success of mobile applications, it is important to provide highly performant applications. This is particularly important for resource-constrained systems such as mobile devices. Thereby, non-functional performance characteristics, such as energy and memory consumption, play an important role for user satisfaction. This paper provides a comprehensive survey of non-functional performance optimization for Android applications. We collected 156 unique publications, published between 2008 and 2020, that focus on the optimization of performance of mobile applications. We target our search at four performance characteristics: responsiveness, launch time, memory and energy consumption. For each performance characteristic, we categorize optimization approaches based on the method used in the corresponding publications. Furthermore, we identify research gaps in the literature for future work.

9 January 2019 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
AAAfnan A. Al-Subaihin
Afnan A. Al-Subaihin
FSFederica Sarro
Federica Sarro
SBSue Black
Sue Black
LCLicia Capra
Licia Capra
MHMark Harman
Mark Harman

18 August 2021
Verified
Conference Paper
Open access
MHMax Hort
Max Hort
JZJie M. Zhang
Jie M. Zhang
FSFederica Sarro
Federica Sarro
MHMark Harman
Mark Harman
The increasingly wide uptake of Machine Learning (ML) has raised the significance of the problem of tackling bias (i.e., unfairness), making it a primary software engineering concern. In this paper, we introduce Fairea, a model behaviour mutation approach to benchmarking ML bias mitigation methods. We also report on a large-scale empirical study to test the effectiveness of 12 widely-studied bias mitigation methods. Our results reveal that, surprisingly, bias mitigation methods have a poor effectiveness in 49% of the cases. In particular, 15% of the mitigation cases have worse fairness-accuracy trade-offs than the baseline established by Fairea; 34% of the cases have a decrease in accuracy and an increase in bias. Fairea has been made publicly available for software engineers and researchers to evaluate their bias mitigation methods.