Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint—a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

One Digital Health is a proposed unified structure. The conceptual framework of the One Digital Health Steering Wheel is built around two keys (ie, One Health and digital health), three perspectives (ie, individual health and well-being, population and society, and ecosystem), and five dimensions (ie, citizens’ engagement, education, environment, human and veterinary health care, and Healthcare Industry 4.0). One Digital Health aims to digitally transform future health ecosystems, by implementing a systemic health and life sciences approach that takes into account broad digital technology perspectives on human health, animal health, and the management of the surrounding environment. This approach allows for the examination of how future generations of health informaticians can address the intrinsic complexity of novel health and care scenarios in digitally transformed health ecosystems. In the emerging hybrid landscape, citizens and their health data have been called to play a central role in the management of individual-level and population-level perspective data. The main challenges of One Digital Health include facilitating and improving interactions between One Health and digital health communities, to allow for efficient interactions and the delivery of near–real-time, data-driven contributions in systems medicine and systems ecology. However, digital health literacy; the capacity to understand and engage in health prevention activities; self-management; and collaboration in the prevention, control, and alleviation of potential problems are necessary in systemic, ecosystem-driven public health and data science research. Therefore, people in a healthy One Digital Health ecosystem must use an active and forceful approach to prevent and manage health crises and disasters, such as the COVID-19 pandemic.

The administrative burden of clinical documentation contributes to health care practitioner burnout and diverts valuable time away from direct patient care. Ambient artificial intelligence (AI) scribes—also called “digital scribes” or “AI scribes”—are emerging as a promising solution, given their potential to automate clinical note generation and reduce clinician workload, and those specifically built on a large language model (LLM) are emerging as technologies for facilitating real-time clinical documentation tasks. This potentially transformative development has a foundation on longer-standing, AI-based transcription software, which uses automated speech recognition and/or natural language processing. Recent studies have highlighted the potential impact of ambient AI scribes on clinician well-being, workflow efficiency, documentation quality, user experience, and patient interaction. So far, limited evidence indicates that ambient AI scribes are associated with reduced clinician burnout, lower cognitive task load, and significant time savings in documentation, particularly in after-hours electronic health record (EHR) work. One consistently reported benefit is the improvement in the patient-physician interaction, as physicians feel more present during a clinical encounter. However, these benefits are counterbalanced by persisting concerns regarding the accuracy, consistency, language use, and style of AI-generated notes. Studies noting errors, omissions, or hallucinations caution that diligent clinician oversight is necessary. The user experience is also heterogeneous, with benefits varying by specialty and individual workflow. Further, there are concerns about ethical and legal issues, algorithmic bias, the potential for long-term “cognitive debt” from overreliance on AI, and even the potential loss of physician autonomy. Additional pragmatic concerns include security, privacy, integration, interoperability, user acceptance and training, and the cost-effectiveness of adoption at scale. Finally, limited studies describe adoption or evaluation of these technologies by nonphysician clinicians and health professionals. Although ambient AI scribes and AI-driven documentation technologies are promising as potentially practice-changing tools, there are many questions remaining. Key issues persist, including responsible deployment with the goal of ensuring that ambient AI scribes produce clinical documentation that supports more efficient, equitable, and patient-centered care. To advance our collective understanding and address key issues, JMIR Medical Informatics is launching a call for papers for a new section on “Ambient AI Scribes and AI-Driven Documentation Technologies.” As editors, we look forward to the opportunity to advance the science and understanding of these fields through publishing high-quality and rigorous scholarly work in this new section of JMIR Medical Informatics.