23 January 2024 in BMC Medical Ethics
Background: While the theoretical benefits and harms of Artificial Intelligence (AI) have been widely discussed in academic literature, empirical evidence remains elusive regarding the practical ethical challenges of developing AI for healthcare. Bridging the gap between theory and practice is an essential step in understanding how to ethically align AI for healthcare. Therefore, this research examines the concerns and challenges perceived by experts in developing ethical AI that addresses the healthcare context and needs. Methods: We conducted semi-structured interviews with 41 AI experts and analyzed the data using reflective thematic analysis. Results: We developed three themes that expressed the considerations perceived by experts as essential for ensuring AI aligns with ethical practices within healthcare. The first theme explores the ethical significance of introducing AI with a clear and purposeful objective. The second theme focuses on how experts are concerned about the tension that exists between economic incentives and the importance of prioritizing the interests of doctors and patients. The third theme illustrates the need to develop context-sensitive AI for healthcare that is informed by its underlying theoretical foundations. Conclusions: The three themes collectively emphasized that beyond being innovative, AI must genuinely benefit healthcare and its stakeholders, meaning AI also aligns with intricate and context-specific healthcare practices. Our findings signal that instead of narrow product-specific AI guidance, ethical AI development may need a systemic, proactive perspective that includes the ethical considerations (objectives, actors, and context) and focuses on healthcare applications. Ethically developing AI involves a complex interplay between AI, ethics, healthcare, and multiple stakeholders.
4 June 2024 in Science and Engineering Ethics
While the technologies that enable Artificial Intelligence (AI) continue to advance rapidly, there are increasing promises regarding AI’s beneficial outputs and concerns about the challenges of human–computer interaction in healthcare. To address these concerns, institutions have increasingly resorted to publishing AI guidelines for healthcare, aiming to align AI with ethical practices. However, guidelines as a form of written language can be analyzed to recognize the reciprocal links between its textual communication and underlying societal ideas. From this perspective, we conducted a discourse analysis to understand how these guidelines construct, articulate, and frame ethics for AI in healthcare. We included eight guidelines and identified three prevalent and interwoven discourses: (1) AI is unavoidable and desirable; (2) AI needs to be guided with (some forms of) principles (3) trust in AI is instrumental and primary. These discourses signal an over-spillage of technical ideals to AI ethics, such as over-optimism and resulting hyper-criticism. This research provides insights into the underlying ideas present in AI guidelines and how guidelines influence the practice and alignment of AI with ethical, legal, and societal values expected to shape AI in healthcare.
30 October 2024 in Bioethics
Since its launch in November 2022, ChatGPT has become a global phenomenon, sparking widespread public interest in chatbot artificial intelligences (AIs) generally. While not approved for medical use, it is capable of passing all three United States medical licensing exams and offers diagnostic accuracy comparable to a human doctor. It seems inevitable that it, and tools like it, are and will be used by the general public to provide medical diagnostic information or treatment plans. Before we are taken in by the promise of a golden age for chatbot medical AIs, it would be wise to consider the implications of using these tools as either supplements to, or substitutes for, human doctors. With the rise of publicly available chatbot AIs, there has been a keen focus on research into the diagnostic accuracy of these tools. This, however, has left a notable gap in our understanding of the implications for health outcomes of these tools. Diagnosis accuracy is only part of good health care. For example, crucial to positive health outcomes is the doctor–patient relationship. This paper challenges the recent focus on diagnostic accuracy by drawing attention to the causal relationship between doctor–patient relationships and health outcomes arguing that chatbot AIs may even hinder outcomes in numerous ways including subtracting the elements of perception and observation that are crucial to clinical consultations. The paper offers brief suggestions to improve chatbot medical AIs so as to positively impact health outcomes.
26 October 2023 in Science and Engineering Ethics
The convergence of human and artificial intelligence is currently receiving considerable scholarly attention. Much debate about the resulting Hybrid Minds focuses on the integration of artificial intelligence into the human brain through intelligent brain-computer interfaces as they enter clinical use. In this contribution we discuss a complementary development: the integration of a functional in vitro network of human neurons into an in silico computing environment.To do so, we draw on a recent experiment reporting the creation of silico-biological intelligence as a case study (Kagan et al., 2022b). In this experiment, multielectrode arrays were plated with stem cell-derived human neurons, creating a system which the authors call DishBrain. By embedding the system into a virtual game-world, neural clusters were able to receive electrical input signals from the game-world and to respond appropriately with output signals from pre-assigned motor regions. Using this design, the authors demonstrate how the DishBrain self-organises and successfully learns to play the computer game ‘Pong’, exhibiting ‘sentient’ and intelligent behaviour in its virtual environment.The creation of such hybrid, silico-biological intelligence raises numerous ethical challenges. Following the neuroscientific framework embraced by the authors themselves, we discuss the arising ethical challenges in the context of Karl Friston’s Free Energy Principle, focusing on the risk of creating synthetic phenomenology. Following the DishBrain’s creator’s neuroscientific assumptions, we highlight how DishBrain’s design may risk bringing about artificial suffering and argue for a congruently cautious approach to such synthetic biological intelligence.
19 August 2024 in JMIR AI
Background: The discourse surrounding medical artificial intelligence (AI) often focuses on narratives that either hype the technology’s potential or predict dystopian futures. AI narratives have a significant influence on the direction of research, funding, and public opinion and thus shape the future of medicine. Objective: The paper aims to offer critical reflections on AI narratives, with a specific focus on medical AI, and to raise awareness as to how people working with medical AI talk about AI and discharge their “narrative responsibility.” Methods: Qualitative semistructured interviews were conducted with 41 participants from different disciplines who were exposed to medical AI in their profession. The research represents a secondary analysis of data using a thematic narrative approach. The analysis resulted in 2 main themes, each with 2 other subthemes. Results: Stories about the AI-physician interaction depicted either a competitive or collaborative relationship. Some participants argued that AI might replace physicians, as it performs better than physicians. However, others believed that physicians should not be replaced and that AI should rather assist and support physicians. The idea of excessive technological deferral and automation bias was discussed, highlighting the risk of “losing” decisional power. The possibility that AI could relieve physicians from burnout and allow them to spend more time with patients was also considered. Finally, a few participants reported an extremely optimistic account of medical AI, while the majority criticized this type of story. The latter lamented the existence of a “magical theory” of medical AI, identified with techno-solutionist positions. Conclusions: Most of the participants reported a nuanced view of technology, recognizing both its benefits and challenges and avoiding polarized narratives. However, some participants did contribute to the hype surrounding medical AI, comparing it to human capabilities and depicting it as superior. Overall, the majority agreed that medical AI should assist rather than replace clinicians. The study concludes that a balanced narrative (that focuses on the technology’s present capabilities and limitations) is necessary to fully realize the potential of medical AI while avoiding unrealistic expectations and hype.
17 November 2025 in npj Digital Medicine
The rapid emergence of patient-facing medical artificial intelligence (MAI) raises pressing questions about its design and impact on healthcare. Current anthropomorphic design strategies, which endow AIs with human-like features, are based on a reductivist understanding of healthcare and risks undermining the integrity of doctor–patient relationships (DPRs) that are foundational to positive health outcomes. This paper argues for a xenomorphic approach—designing MAIs with decidedly non-human, even alien-like characteristics. By distinguishing AIs from human physicians as far as possible, xenomorphic design may avoid direct competition with human relational roles, protecting the ontological distinctiveness of DPRs. Importantly, we suggest that such designs may foster a new class of therapeutic relationships with patients: non-human but nonetheless health-affirming. Analogous to the established benefits of animal-assisted therapies, xenomorphic MAIs could provide complementary sources of trust, comfort, and support without supplanting essential human care. The xenomorphic approach may therefore not only avoid the conceptual and practical challenges of anthropomorphism but also offer an innovative path to expand therapeutic possibilities and improve health outcomes.