Objective: Accurate and rapid phenotyping is a prerequisite to leveraging electronic health records for biomedical research. While early phenotyping relied on rule-based algorithms curated by experts, machine learning (ML) approaches have emerged as an alternative to improve scalability across phenotypes and healthcare settings. This study evaluates ML-based phenotyping with respect to (1) the data sources used, (2) the phenotypes considered, (3) the methods applied, and (4) the reporting and evaluation methods used.Materials and methods: We searched PubMed and Web of Science for articles published between 2018 and 2022. After screening 850 articles, we recorded 37 variables on 100 studies.Results: Most studies utilized data from a single institution and included information in clinical notes. Although chronic conditions were most commonly considered, ML also enabled the characterization of nuanced phenotypes such as social determinants of health. Supervised deep learning was the most popular ML paradigm, while semi-supervised and weakly supervised learning were applied to expedite algorithm development and unsupervised learning to facilitate phenotype discovery. ML approaches did not uniformly outperform rule-based algorithms, but deep learning offered a marginal improvement over traditional ML for many conditions.Discussion: Despite the progress in ML-based phenotyping, most articles focused on binary phenotypes and few articles evaluated external validity or used multi-institution data. Study settings were infrequently reported and analytic code was rarely released.Conclusion: Continued research in ML-based phenotyping is warranted, with emphasis on characterizing nuanced phenotypes, establishing reporting and evaluation standards, and developing methods to accommodate misclassified phenotypes due to algorithm errors in downstream applications.

Introduction: Through the INT ernational Conso R tium of P rimary Care B I g D ata Researchers ( INTRePID ), we compared the pandemic impact on the volume of primary care visits and uptake of virtual care in Australia, Canada, China, Norway, Singapore, South Korea, Sweden, the UK and the USA. Methods: Visit definitions were agreed on centrally, implemented locally across the various settings in INTRePID countries, and weekly visit counts were shared centrally for analysis. We evaluated the weekly rate of primary care physician visits during 2019 and 2020. Rate ratios (RRs) of total weekly visit volume and the proportion of weekly visits that were virtual in the pandemic period in 2020 compared with the same prepandemic period in 2019 were calculated. Results: In 2019 and 2020, there were 80 889 386 primary care physician visits across INTRePID. During the pandemic, average weekly visit volume dropped in China, Singapore, South Korea, and the USA but was stable overall in Australia (RR 0.98 (95% CI 0.92 to 1.05, p=0.59)), Canada (RR 0.96 (95% CI 0.89 to 1.03, p=0.24)), Norway (RR 1.01 (95% CI 0.88 to 1.17, p=0.85)), Sweden (RR 0.91 (95% CI 0.79 to 1.06, p=0.22)) and the UK (RR 0.86 (95% CI 0.72 to 1.03, p=0.11)). In countries that had negligible virtual care prepandemic, the proportion of visits that were virtual were highest in Canada (77.0%) and Australia (41.8%). In Norway (RR 8.23 (95% CI 5.30 to 12.78, p<0.001), the UK (RR 2.36 (95% CI 2.24 to 2.50, p<0.001)) and Sweden (RR 1.33 (95% CI 1.17 to 1.50, p<0.001)) where virtual visits existed prepandemic, it increased significantly during the pandemic. Conclusions: The drop in primary care in-person visits during the pandemic was a global phenomenon across INTRePID countries. In several countries, primary care shifted to virtual visits mitigating the drop in in-person visits.

In many contemporary applications, large amounts of unlabelled data are readily available while labelled examples are limited. There has been substantial interest in semi-supervised learning (SSL) which aims to leverage unlabelled data to improve estimation or prediction. However, current SSL literature focuses primarily on settings where labelled data are selected uniformly at random from the population of interest. Stratified sampling, while posing additional analytical challenges, is highly applicable to many real-world problems. Moreover, no SSL methods currently exist for estimating the prediction performance of a fitted model when the labelled data are not selected uniformly at random. In this paper, we propose a two-step SSL procedure for evaluating a prediction rule derived from a working binary regression model based on the Brier score and overall misclassification rate under stratified sampling. In step I, we impute the missing labels via weighted regression with nonlinear basis functions to account for stratified sampling and to improve efficiency. In step II, we augment the initial imputations to ensure the consistency of the resulting estimators regardless of the specification of the prediction model or the imputation model. The final estimator is then obtained with the augmented imputations. We provide asymptotic theory and numerical studies illustrating that our proposals outperform their supervised counterparts in terms of efficiency gain. Our methods are motivated by electronic health record (EHR) research and validated with a real data analysis of an EHR-based study of diabetic neuropathy.

Although randomized controlled trials (RCTs) are the gold standard for establishing the efficacy and safety of a medical treatment, real-world evidence (RWE) generated from real-world data has been vital in postapproval monitoring and is being promoted for the regulatory process of experimental therapies. An emerging source of real-world data is electronic health records (EHRs), which contain detailed information on patient care in both structured (eg, diagnosis codes) and unstructured (eg, clinical notes and images) forms. Despite the granularity of the data available in EHRs, the critical variables required to reliably assess the relationship between a treatment and clinical outcome are challenging to extract. To address this fundamental challenge and accelerate the reliable use of EHRs for RWE, we introduce an integrated data curation and modeling pipeline consisting of 4 modules that leverage recent advances in natural language processing, computational phenotyping, and causal modeling techniques with noisy data. Module 1 consists of techniques for data harmonization. We use natural language processing to recognize clinical variables from RCT design documents and map the extracted variables to EHR features with description matching and knowledge networks. Module 2 then develops techniques for cohort construction using advanced phenotyping algorithms to both identify patients with diseases of interest and define the treatment arms. Module 3 introduces methods for variable curation, including a list of existing tools to extract baseline variables from different sources (eg, codified, free text, and medical imaging) and end points of various types (eg, death, binary, temporal, and numerical). Finally, module 4 presents validation and robust modeling methods, and we propose a strategy to create gold-standard labels for EHR variables of interest to validate data curation quality and perform subsequent causal modeling for RWE. In addition to the workflow proposed in our pipeline, we also develop a reporting guideline for RWE that covers the necessary information to facilitate transparent reporting and reproducibility of results. Moreover, our pipeline is highly data driven, enhancing study data with a rich variety of publicly available information and knowledge sources. We also showcase our pipeline and provide guidance on the deployment of relevant tools by revisiting the emulation of the Clinical Outcomes of Surgical Therapy Study Group Trial on laparoscopy-assisted colectomy versus open colectomy in patients with early-stage colon cancer. We also draw on existing literature on EHR emulation of RCTs together with our own studies with the Mass General Brigham EHR.