Objective: To conduct a systematic scoping review of explainable artificial intelligence (XAI) models that use real-world electronic health record data, categorize these techniques according to different biomedical applications, identify gaps of current studies, and suggest future research directions.Materials and Methods: We searched MEDLINE, IEEE Xplore, and the Association for Computing Machinery (ACM) Digital Library to identify relevant papers published between January 1, 2009 and May 1, 2019. We summarized these studies based on the year of publication, prediction tasks, machine learning algorithm, dataset(s) used to build the models, the scope, category, and evaluation of the XAI methods. We further assessed the reproducibility of the studies in terms of the availability of data and code and discussed open issues and challenges.Results: Forty-two articles were included in this review. We reported the research trend and most-studied diseases. We grouped XAI methods into 5 categories: knowledge distillation and rule extraction (N = 13), intrinsically interpretable models (N = 9), data dimensionality reduction (N = 8), attention mechanism (N = 7), and feature interaction and importance (N = 5).Discussion: XAI evaluation is an open issue that requires a deeper focus in the case of medical applications. We also discuss the importance of reproducibility of research work in this field, as well as the challenges and opportunities of XAI from 2 medical professionals’ point of view.Conclusion: Based on our review, we found that XAI evaluation in medicine has not been adequately and formally practiced. Reproducibility remains a critical concern. Ample opportunities exist to advance XAI research in medicine.

Clinical studies, especially randomized, controlled trials, are essential for generating evidence for clinical practice. However, generalizability is a long‐standing concern when applying trial results to real‐world patients. Generalizability assessment is thus important, nevertheless, not consistently practiced. We performed a systematic review to understand the practice of generalizability assessment. We identified 187 relevant articles and systematically organized these studies in a taxonomy with three dimensions: (i) data availability (i.e., before or after trial (a priori vs. a posteriori generalizability)); (ii) result outputs (i.e., score vs. nonscore); and (iii) populations of interest. We further reported disease areas, underrepresented subgroups, and types of data used to profile target populations. We observed an increasing trend of generalizability assessments, but < 30% of studies reported positive generalizability results. As a priori generalizability can be assessed using only study design information (primarily eligibility criteria), it gives investigators a golden opportunity to adjust the study design before the trial starts. Nevertheless, < 40% of the studies in our review assessed a priori generalizability. With the wide adoption of electronic health records systems, rich real‐world patient databases are increasingly available for generalizability assessment; however, informatics tools are lacking to support the adoption of generalizability assessment practice.

Social media analysis has shown tremendous potential to understand public's opinion on a wide variety of topics. In this paper, we have mined Twitter to understand the public's perception of the Internet of Things (IoT). We first generated the discussion trends of the IoT from multiple Twitter data sources and validated these trends with Google Trends. We then performed sentiment analysis to gain insights of the public’s attitude towards the IoT. As anticipated, our analysis indicates that the public's perception of the IoT is predominantly positive. Further, through topic modeling, we learned that public tweets discussing the IoT were often focused on business and technology. However, the public has great concerns about privacy and security issues toward the IoT based on the frequent appearance of related terms. Nevertheless, no unexpected perceptions were identified through our analysis. Our analysis was challenged by the limited fraction of tweets relevant to our study. Also, the user demographics of Twitter users may not be strongly representative of the population of the general public.

Background: Although patients have easy access to their electronic health records and laboratory test result data through patient portals, laboratory test results are often confusing and hard to understand. Many patients turn to web-based forums or question-and-answer (Q&A) sites to seek advice from their peers. The quality of answers from social Q&A sites on health-related questions varies significantly, and not all responses are accurate or reliable. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to have their questions answered. Objective: We aimed to assess the feasibility of using LLMs to generate relevant, accurate, helpful, and unharmful responses to laboratory test–related questions asked by patients and identify potential issues that can be mitigated using augmentation approaches. Methods: We collected laboratory test result–related Q&A data from Yahoo! Answers and selected 53 Q&A pairs for this study. Using the LangChain framework and ChatGPT web portal, we generated responses to the 53 questions from 5 LLMs: GPT-4, GPT-3.5, LLaMA 2, MedAlpaca, and ORCA_mini. We assessed the similarity of their answers using standard Q&A similarity-based evaluation metrics, including Recall-Oriented Understudy for Gisting Evaluation, Bilingual Evaluation Understudy, Metric for Evaluation of Translation With Explicit Ordering, and Bidirectional Encoder Representations from Transformers Score. We used an LLM-based evaluator to judge whether a target model had higher quality in terms of relevance, correctness, helpfulness, and safety than the baseline model. We performed a manual evaluation with medical experts for all the responses to 7 selected questions on the same 4 aspects. Results: Regarding the similarity of the responses from 4 LLMs; the GPT-4 output was used as the reference answer, the responses from GPT-3.5 were the most similar, followed by those from LLaMA 2, ORCA_mini, and MedAlpaca. Human answers from Yahoo data were scored the lowest and, thus, as the least similar to GPT-4–generated answers. The results of the win rate and medical expert evaluation both showed that GPT-4’s responses achieved better scores than all the other LLM responses and human responses on all 4 aspects (relevance, correctness, helpfulness, and safety). LLM responses occasionally also suffered from lack of interpretation in one’s medical context, incorrect statements, and lack of references. Conclusions: By evaluating LLMs in generating responses to patients’ laboratory test result–related questions, we found that, compared to other 4 LLMs and human answers from a Q&A website, GPT-4’s responses were more accurate, helpful, relevant, and safer. There were cases in which GPT-4 responses were inaccurate and not individualized. We identified a number of ways to improve the quality of LLM responses, including prompt engineering, prompt augmentation, retrieval-augmented generation, and response evaluation.

Background: Patients are increasingly able to access their laboratory test results via patient portals. However, merely providing access does not guarantee comprehension. Patients could experience confusion when reviewing their test results. Objective: The aim of this study is to examine the challenges and needs of patients when comprehending laboratory test results. Methods: We conducted a web-based survey with 203 participants and a set of semistructured interviews with 13 participants. We assessed patients’ perceived challenges and needs (both informational and technological needs) when they attempted to comprehend test results, factors associated with patients’ perceptions, and strategies for improving the design of patient portals to communicate laboratory test results more effectively. Descriptive and correlation analysis and thematic analysis were used to analyze the survey and interview data, respectively. Results: Patients face a variety of challenges and confusion when reviewing laboratory test results. To better comprehend laboratory results, patients need different types of information, which are grouped into 2 categories—generic information (eg, reference range) and personalized or contextual information (eg, treatment options, prognosis, what to do or ask next). We also found that several intrinsic factors (eg, laboratory result normality, health literacy, and technology proficiency) significantly impact people’s perceptions of using portals to view and interpret laboratory results. The desired enhancements of patient portals include providing timely explanations and educational resources (eg, a health encyclopedia), increasing usability and accessibility, and incorporating artificial intelligence–based technology to provide personalized recommendations. Conclusions: Patients face significant challenges in interpreting the meaning of laboratory test results. Designers and developers of patient portals should employ user-centered approaches to improve the design of patient portals to present information in a more meaningful way.