Background: Artificial intelligence (AI) has the potential to revolutionize health care by enhancing both clinical outcomes and operational efficiency. However, its clinical adoption has been slower than anticipated, largely due to the absence of comprehensive evaluation frameworks. Existing frameworks remain insufficient and tend to emphasize technical metrics such as accuracy and validation, while overlooking critical real-world factors such as clinical impact, integration, and economic sustainability. This narrow focus prevents AI tools from being effectively implemented, limiting their broader impact and long-term viability in clinical practice. Objective: This study aimed to create a framework for assessing AI in health care, extending beyond technical metrics to incorporate social and organizational dimensions. The framework was developed by systematically reviewing, analyzing, and synthesizing the evaluation criteria necessary for successful implementation, focusing on the long-term real-world impact of AI in clinical practice. Methods: A search was performed in July 2024 across the PubMed, Cochrane, Scopus, and IEEE Xplore databases to identify relevant studies published in English between January 2019 and mid-July 2024, yielding 3528 results, among which 44 studies met the inclusion criteria. The systematic review followed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews. Data were analyzed using NVivo through thematic analysis and narrative synthesis to identify key emergent themes in the studies. Results: By synthesizing the included studies, we developed a framework that goes beyond the traditional focus on technical metrics or study-level methodologies. It integrates clinical context and real-world implementation factors, offering a more comprehensive approach to evaluating AI tools. With our focus on assessing the long-term real-world impact of AI technologies in health care, we named the framework AI for IMPACTS. The criteria are organized into seven key clusters, each corresponding to a letter in the acronym: (1) I—integration, interoperability, and workflow; (2) M—monitoring, governance, and accountability; (3) P—performance and quality metrics; (4) A—acceptability, trust, and training; (5) C—cost and economic evaluation; (6) T—technological safety and transparency; and (7) S—scalability and impact. These are further broken down into 28 specific subcriteria. Conclusions: The AI for IMPACTS framework offers a holistic approach to evaluate the long-term real-world impact of AI tools in the heterogeneous and challenging health care context and lays the groundwork for further validation through expert consensus and testing of the framework in real-world health care settings. It is important to emphasize that multidisciplinary expertise is essential for assessment, yet many assessors lack the necessary training. In addition, traditional evaluation methods struggle to keep pace with AI’s rapid development. To ensure successful AI integration, flexible, fast-tracked assessment processes and proper assessor training are needed to maintain rigorous standards while adapting to AI’s dynamic evolution. Trial Registration: reviewregistry1859; https://tinyurl.com/ysn2d7sh

Background: Research has shown that patient engagement is most commonly done at the beginning of research or to test readily available prototypes and less commonly done in other phases such as the execution phases. Previous studies have reported that patients are usually assigned a consultative rather than a decision-making role in health service planning and evaluation. Objective: This study had 2 objectives: to better understand the challenges and opportunities in the inclusion of patients in the development of eHealth technologies and ideas on how to overcome the identified gaps and to create a research-based end-to-end practical blueprint that can guide the relevant stakeholders to successfully engage patients as cocreators in all human-centered design phases rather than mere testers of preplanned prototypes. Methods: Key informant interviews were conducted using in-depth semistructured interviews with 20 participants from 6 countries across Europe. This was followed by a focus group to validate the initial findings. Participants encompassed all the relevant stakeholder groups including patient experts, eHealth experts, health technology providers, clinicians, pharma executives, and health insurance experts. Results: This study shows that engaging patients in eHealth development can help provide different types of value; namely, identifying unmet needs, better usability and desirability, better fit into the patient journey, better adoption and stickiness, better health outcomes, advocacy and trust, a sense of purpose, and better health equity and access. However, the participants agreed that patients are usually engaged too late in the development process, mostly assuming a sounding role in testing a ready-made prototype. The justification for these gaps in engagement is driven by some prominent barriers, notably compliance risks, patient-related factors, power dynamics, patient engagement as lip service, poor value perception, lack of resources, mistrust, and inflexibility. On the positive side, the participants also reflected on facilitators for better patient engagement; for instance, engaging through engagement partners, novel approaches such as the rise of professional patient experts, embedding patients in development teams, expectation management, and professional moderation services. Conclusions: Overcoming the current gaps in patient engagement in eHealth development requires consolidated efforts from all stakeholders in a complex health care ecosystem. The shift toward more patient-driven eHealth development requires education and awareness; frameworks to monitor and evaluate the value of patient engagement; regulatory clarity and simplification; platforms to facilitate patient access and identification; patient incentivization, transparency, and trust; and a mindset shift toward value-based health care.

Background: Technological advancements have opened the path for many technology providers to easily develop and introduce eHealth tools to the public. The use of these tools is increasingly recognized as a critical quality driver in health care; however, choosing a quality tool from the myriad of tools available for a specific health need does not come without challenges. Objective: This review aimed to systematically investigate the literature to understand the different approaches and criteria used to assess the quality and impact of eHealth tools by considering sociotechnical factors (from technical, social, and organizational perspectives). Methods: A structured search was completed following the participants, intervention, comparators, and outcomes framework. We searched the PubMed, Cochrane, Web of Science, Scopus, and ProQuest databases for studies published between January 2012 and January 2022 in English, which yielded 675 results, of which 40 (5.9%) studies met the inclusion criteria. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews of Interventions were followed to ensure a systematic process. Extracted data were analyzed using NVivo (QSR International), with a thematic analysis and narrative synthesis of emergent themes. Results: Similar measures from the different papers, frameworks, and initiatives were aggregated into 36 unique criteria grouped into 13 clusters. Using the sociotechnical approach, we classified the relevant criteria into technical, social, and organizational assessment criteria. Technical assessment criteria were grouped into 5 clusters: technical aspects, functionality, content, data management, and design. Social assessment criteria were grouped into 4 clusters: human centricity, health outcomes, visible popularity metrics, and social aspects. Organizational assessment criteria were grouped into 4 clusters: sustainability and scalability, health care organization, health care context, and developer. Conclusions: This review builds on the growing body of research that investigates the criteria used to assess the quality and impact of eHealth tools and highlights the complexity and challenges facing these initiatives. It demonstrates that there is no single framework that is used uniformly to assess the quality and impact of eHealth tools. It also highlights the need for a more comprehensive approach that balances the social, organizational, and technical assessment criteria in a way that reflects the complexity and interdependence of the health care ecosystem and is aligned with the factors affecting users’ adoption to ensure uptake and adherence in the long term.

Among the thousands of eHealth tools available, the vast majority do not get past pilot phases because they cannot prove value, and only a few have been systematically assessed. Although multiple eHealth assessment frameworks have been developed, these efforts face multiple challenges. This study aimed to address some of these challenges by validating and refining an initial list of 55 assessment criteria based on previous frameworks through a two-round modified Delphi process with in-between rounds of interviews. The expert panel (n = 57) included participants from 18 countries and 9 concerned parties. A consensus was reached on 46 criteria that were classified into foundational and contextual criteria. The 36 foundational criteria focus on evaluating the eHealth tool itself and were grouped into nine clusters: technical aspects, clinical utility and safety, usability and human centricity, functionality, content, data management, endorsement, maintenance, and developer. The 10 contextual criteria focus on evaluating the factors that vary depending on the context the tool is being evaluated for and were grouped into seven clusters: data-protection compliance, safety regulatory compliance, interoperability and data integration, cultural requirements, affordability, cost-benefit, and implementability. The classification of criteria into foundational and contextual helps us assess not only the quality of an isolated tool, but also its potential fit in a specific setting. Criteria subscales may be particularly relevant when determining the strengths and weaknesses of the tool being evaluated. This granularity enables different concerned parties to make informed decisions about which tools to consider according to their specific needs and priorities.

International studies measuring wellbeing/multidimensional mental health before/ during the COVID-19 pandemic, including representative samples for >2 years, identifying risk groups and coping strategies are lacking. COH-FIT is an online, international, anonymous survey measuring changes in well-being (WHO-5) and a composite psychopathology P-score, and their associations with COVID-19 deaths/restrictions, 12 a-priori defined risk individual/cumulative factors, and coping strategies during COVID-19 pandemic (26/04/2020-26/06/2022) in 30 languages (representative, weighted non-representative, adults). T-test, χ², penalized cubic splines, linear regression, correlation analyses were conducted. Analyzing 121,066/142,364 initiated surveys, WHO-5/P-score worsened intra-pandemic by 11.1±21.1/13.2±17.9 points (effect size d=0.50/0.60) (comparable results in representative/weighted non-probability samples). Persons with WHO-5 scores indicative of depression screening (<50, 13% to 32%) and major depression (2 years. The most subjectively effective coping strategies were exercise and walking, internet use, social contacts. Identified risk groups, coping strategies and outcome trajectories can inform global public health strategies.