Explainability for artificial intelligence (AI) in medicine is a hotly debated topic. Our paper presents a review of the key arguments in favor and against explainability for AI-powered Clinical Decision Support System (CDSS) applied to a concrete use case, namely an AI-powered CDSS currently used in the emergency call setting to identify patients with life-threatening cardiac arrest. More specifically, we performed a normative analysis using socio-technical scenarios to provide a nuanced account of the role of explainability for CDSSs for the concrete use case, allowing for abstractions to a more general level. Our analysis focused on three layers: technical considerations, human factors, and the designated system role in decision-making. Our findings suggest that whether explainability can provide added value to CDSS depends on several key questions: technical feasibility, the level of validation in case of explainable algorithms, the characteristics of the context in which the system is implemented, the designated role in the decision-making process, and the key user group(s). Thus, each CDSS will require an individualized assessment of explainability needs and we provide an example of how such an assessment could look like in practice.
29 July 2022 in IEEE Transactions on Technology and Society
The paper’s main contributions are twofold: to demonstrate how to apply the general European Union’s High-Level Expert Group’s (EU HLEG) guidelines for trustworthy AI in practice for the domain of healthcare; and to investigate the research question of what does “trustworthy AI” mean at the time of the COVID-19 pandemic. To this end, we present the results of a post-hoc self-assessment to evaluate the trustworthiness of an AI system for predicting a multi-regional score conveying the degree of lung compromise in COVID-19 patients, developed and verified by an interdisciplinary team with members from academia, public hospitals, and industry in time of pandemic. The AI system aims to help radiologists to estimate and communicate the severity of damage in a patient’s lung from Chest X-rays. It has been experimentally deployed in the radiology department of the ASST Spedali Civili clinic in Brescia (Italy) since December 2020 during pandemic time. The methodology we have applied for our post-hoc assessment, called Z-Inspection, uses socio-technical scenarios to identify ethical, technical and domain-specific issues in the use of the AI system in the context of the pandemic.
18 January 2018 in Journal of Law and the Biosciences
Gene-editing technology, such as CRISPR/Cas9, holds great promise for the advancement of science and many useful applications technology. This foundational technology enables modification of the genetic structure of any living organisms with unprecedented precision. Yet, in order to enhance its potential for societal benefit, it is necessary to adapt rules and produce adequate regulations. This requires an interdisciplinary effort in legal thinking. Any legislative initiative needs to consider both the benefits and the problematic aspects of gene editing, from a broader societal and value-based perspective. This paper stems from an interdisciplinary research project seeking to identify and discuss some of the most pressing legal implications of gene-editing technology and how to address these. While the questions raised by gene editing are global, laws and regulations are to a great extent bound by national borders. This paper presents a European perspective, written for a global audience, and intends to contribute to the global debate. The analysis will include brief references to corresponding USA rules in order to place these European debates in the broader international context. Our legal analysis incorporates interdisciplinary contributes concerning the scientific state of the art, philosophical thinking regarding the precautionary principle and dual-use issues as well as the importance of communication, social perception, and public debate. Focusing mainly in the main regulatory and patent law issues, we will argue that (a) general moratoriums and blank prohibitions do a disservice to science and innovation; (b) it is crucial to carefully consider a complex body of international and European fundamental rights norms applicable to gene editing; (c) these require further developments grounded in consistent and coherent implementation and interpretation; (d) legal development should follow a critical contextual approach capable of integrating interdisciplinary contributions and broad multilevel societal dialog.
Recent work on algorithmic fairness has largely focused on the fairness of discrete decisions, or classifications. While such decisions are often based on risk score models, the fairness of the risk models themselves has received considerably less attention. Risk models are of interest for a number of reasons, including the fact that they communicate uncertainty about the potential outcomes to users, thus representing a way to enable meaningful human oversight. Here, we address fairness desiderata for risk score models. We identify the provision of similar epistemic value to different groups as a key desideratum for risk score fairness, and we show how even fair risk scores can lead to unfair risk-based rankings. Further, we address how to assess the fairness of risk score models quantitatively, including a discussion of metric choices and meaningful statistical comparisons between groups. In this context, we also introduce a novel calibration error metric that is less sample size-biased than previously proposed metrics, enabling meaningful comparisons between groups of different sizes. We illustrate our methodology – which is widely applicable in many other settings – in two case studies, one in recidivism risk prediction, and one in risk of major depressive disorder (MDD) prediction.