The increasing use of artificial intelligence (AI) technologies across application domains has prompted our society to pay closer attention to AI's trustworthiness, fairness, interpretability, and accountability. In order to foster trust in AI, it is important to consider the potential of interactive visualization, and how such visualizations help build trust in AI systems. This manifesto discusses the relevance of interactive visualizations and makes the following four claims: i) trust is not a technical problem, ii) trust is dynamic, iii) visualization cannot address all aspects of trust, and iv) visualization is crucial for human agency in AI.
Algorithms used in the public sector, e.g., for allocating social benefits or predicting fraud, often require involvement from multiple stakeholders at various phases of the algorithm’s life-cycle. This paper focuses on the communication issues between diverse stakeholders that can lead to misinterpretation and misuse of algorithmic systems. Ethnographic research was conducted via 11 semi-structured interviews with practitioners working on algorithmic systems in the Dutch public sector, at local and national levels. With qualitative coding analysis, we identify key elements of the communication processes that underlie fairness-related human decisions. More specifically, we analyze the division of roles and tasks, the required skills, and the challenges perceived by diverse stakeholders. Three general patterns emerge from the coding analysis: (1) Policymakers, civil servants, and domain experts are less involved compared to developers throughout a system’s life-cycle. This leads to developers taking on the role of decision-maker and policy advisor, while they potentially miss the required skills. (2) End-users and policy-makers often lack the technical skills to interpret a system’s output, and rely on actors having a developer role for making decisions concerning fairness issues. (3) Citizens are structurally absent throughout a system’s life-cycle. This may lead to unbalanced fairness assessments that do not include key input from relevant stakeholders. We formalize the underlying communication issues within such networks of stakeholders and introduce the phase-actor-role-task-skill (PARTS) model. PARTS can both (i) represent the communication patterns identified in the interviews, and (ii) explicitly outline missing elements in communication patterns such as actors who miss skills or collaborators for their tasks, or tasks that miss qualified actors. The PARTS model can be extended to other use cases and used to analyze and design the human organizations responsible for assessing fairness in algorithmic systems. It can be further extended to explore communication issues in other use cases, design potential solutions, and organize accountability with a common vocabulary.
In-situ video recording of underwater ecosystems is able to provide valuable information for biology research and natural resources management, e.g. changes in species abundance. Searching the videos manually, however, requires costly human effort. Our video analysis tool supports the key task of counting different species of fish, allowing marine biologists to query the video collection without watching the videos. To be suitable for scientific research on changes in species abundance, the video data must include data provenance information that reflects the potential biases introduced through the video processing.In order to trust the analyses made by the system, we need to provide expert users with sufficient information to allow them to interpret these potential biases. We conducted two user studies to design a user interface that includes data provenance information. Our qualitative analysis discusses the support for understanding the reliability of video analysis, and trusting the results it produces. Our main finding is that disclosing details about the video processing and provenance data allows biologists to compare the results with their traditional statistical methods, thus increasing their trust in the results.
Our study investigates the inter-dependencies between the technical components of AI-based fraud detection systems and fairness approaches. Here, fairness approaches refer to any method or process to address, specify, measure or mitigate unfairness in AI systems. Despite extensive research on both AI-based fraud detection systems and AI fairness, these domains are often treated separately, leading to a fragmented understanding of both domains. This fragmentation leads to inconsistent terminologies and concepts, and fails to identify the inter-dependencies between technical components of fraud detection systems and fairness approaches. These are the research gaps we address in this study. Through a literature review of 78 papers, we synthesize existing concepts and terminologies into two structured taxonomies that allow to document these inter-dependencies. One taxonomy describes the technical components of AI fraud detection systems and another describes fairness approaches. A joint evaluation of the taxonomies reveals how technical components are impacting fairness, and how fairness considerations should govern the design and implementation of these technical components. Our findings show that four technical components (regarding class imbalance, overlapping class distributions, data quality, and adaptive learning) are intertwined with fairness approaches in fraud detection. The proposed taxonomies, and the inter-dependencies we identify, support AI practitioners and decision-makers in designing fair fraud detection systems. Additionally, the taxonomies provide structure to research investigating how technical components and technical decisions impact fairness and, vice versa, investigating how fairness approaches impact the technical design of fraud detection systems.
AI algorithms used in the public sector, e.g., for allocating social benefits or predicting fraud, often involve multiple public and private stakeholders at various phases of the algorithm's life-cycle. Communication issues between these diverse stakeholders can lead to misinterpretation and misuse of algorithms. We investigate the communication processes for AI fairness-related decisions by conducting interviews with practitioners working on algorithmic systems in the public sector. By applying qualitative coding analysis, we identify key elements of communication processes that underlie fairness-related human decisions. We analyze the division of roles, tasks, skills, and challenges perceived by stakeholders. We formalize the underlying communication issues within a conceptual framework that i. represents the communication patterns ii. outlines missing elements, such as actors who miss skills for their tasks. The framework is used for describing and analyzing key organizational issues for fairness-related decisions. Three general patterns emerge from the analysis: 1. Policy-makers, civil servants, and domain experts are less involved compared to developers throughout a system's life-cycle. This leads to developers taking on extra roles such as advisor, while they potentially miss the required skills and guidance from domain experts. 2. End-users and policy-makers often lack the technical skills to interpret a system's limitations, and rely on developer roles for making decisions concerning fairness issues. 3. Citizens are structurally absent throughout a system's life-cycle, which may lead to decisions that do not include relevant considerations from impacted stakeholders.