Deep learning (DL)-based predictive models from electronic health records (EHRs) deliver impressive performance in many clinical tasks. Large training cohorts, however, are often required by these models to achieve high accuracy, hindering the adoption of DL-based models in scenarios with limited training data. Recently, bidirectional encoder representations from transformers (BERT) and related models have achieved tremendous successes in the natural language processing domain. The pretraining of BERT on a very large training corpus generates contextualized embeddings that can boost the performance of models trained on smaller datasets. Inspired by BERT, we propose Med-BERT, which adapts the BERT framework originally developed for the text domain to the structured EHR domain. Med-BERT is a contextualized embedding model pretrained on a structured EHR dataset of 28,490,650 patients. Fine-tuning experiments showed that Med-BERT substantially improves the prediction accuracy, boosting the area under the receiver operating characteristics curve (AUC) by 1.21–6.14% in two disease prediction tasks from two clinical databases. In particular, pretrained Med-BERT obtains promising performances on tasks with small fine-tuning training sets and can boost the AUC by more than 20% or obtain an AUC as high as a model trained on a training set ten times larger, compared with deep learning models without Med-BERT. We believe that Med-BERT will benefit disease prediction studies with small local training datasets, reduce data collection expenses, and accelerate the pace of artificial intelligence aided healthcare.

Existing functional description of genes are categorical, discrete, and mostly through manual process. In this work, we explore the idea of gene embedding, distributed representation of genes, in the spirit of word embedding. From a pure data-driven fashion, we trained a 200-dimension vector representation of all human genes, using gene co-expression patterns in 984 data sets from the GEO databases. These vectors capture functional relatedness of genes in terms of recovering known pathways - the average inner product (similarity) of genes within a pathway is 1.52X greater than that of random genes. Using t-SNE, we produced a gene co-expression map that shows local concentrations of tissue specific genes. We also illustrated the usefulness of the embedded gene vectors, laden with rich information on gene co-expression patterns, in tasks such as gene-gene interaction prediction. We proposed a machine learning method that utilizes transcriptome-wide gene co-expression to generate a distributed representation of genes. We further demonstrated the utility of our distribution by predicting gene-gene interaction based solely on gene names. The distributed representation of genes could be useful for more bioinformatics applications.

DNA methylation is an important molecular-level phenotype that links genotypes and complex disease traits. Previous studies have found local correlation between genetic variants and DNA methylation levels (cis-meQTLs). However, general mechanisms underlying cis-meQTLs are unclear. We conducted a cis-meQTL analysis of the Genetics of Lipid Lowering Drugs and Diet Network data (n = 593). We found that over 80% of genetic variants at CpG sites (meSNPs) are meQTL loci (P-value < 10−9), and meSNPs account for over two thirds of the strongest meQTL signals (P-value < 10−200). Beyond direct effects on the methylation of the meSNP site, the CpG-disrupting allele of meSNPs were associated with lowered methylation of CpG sites located within 45 bp. The effect of meSNPs extends to as far as 10 kb and can contribute to the observed meQTL signals in the surrounding region, likely through correlated methylation patterns and linkage disequilibrium. Therefore, meSNPs are behind a large portion of observed meQTL signals and play a crucial role in the biological process linking genetic variation to epigenetic changes.

Columnaris causes severe mortalities among many different wild and cultured freshwater fish species, but understanding of host resistance is lacking. Catfish, the primary aquaculture species in the United States, serves as a great model for the analysis of host resistance against columnaris disease. Channel catfish in general is highly resistant to the disease while blue catfish is highly susceptible. F2 generation of hybrids can be produced where phenotypes and genotypes are segregating, providing a useful system for QTL analysis. To identify genes associated with columnaris resistance, we performed a genome-wide association study (GWAS) using the catfish 250 K SNP array with 340 backcross progenies derived from crossing female channel catfish (Ictalurus punctatus) with male F1 hybrid catfish (female channel catfish I. punctatus × male blue catfish I. furcatus). A genomic region on linkage group 7 was found to be significantly associated with columnaris resistance. Within this region, five have known functions in immunity, including pik3r3b, cyld-like, adcyap1r1, adcyap1r1-like, and mast2. In addition, 3 additional suggestively associated QTL regions were identified on linkage groups 7, 12, and 14. The resistant genotypes on the QTLs of linkage groups 7 and 12 were found to be homozygous with both alleles being derived from channel catfish. The paralogs of the candidate genes in the suggestively associated QTL of linkage group 12 were found on the QTLs of linkage group 7. Many candidate genes on the four associated regions are involved in PI3K pathway that is known to be required by many bacteria for efficient entry into the host. The GWAS revealed four QTLs associated with columnaris resistance in catfish. Strikingly, the candidate genes may be arranged as functional hubs; the candidate genes within the associated QTLs on linkage groups 7 and 12 are not only co-localized, but also functionally related, with many of them being involved in the PI3K signal transduction pathway, suggesting its importance for columnaris resistance.

Recently, epigenetic age acceleration—or older epigenetic age in comparison to chronological age—has been robustly associated with mortality and various morbidities. However, accelerated epigenetic aging has not been widely investigated in relation to inflammatory or metabolic markers, including postprandial lipids. We estimated measures of epigenetic age acceleration in 830 Caucasian participants from the Genetics Of Lipid Lowering Drugs and diet Network (GOLDN) considering two epigenetic age calculations based on differing sets of 5′-Cytosine-phosphate-guanine-3′ genomic site, derived from the Horvath and Hannum DNA methylation age calculators, respectively. GOLDN participants underwent a standardized high-fat meal challenge after fasting for at least 8 h followed by timed blood draws, the last being 6 h postmeal. We used adjusted linear mixed models to examine the association of the epigenetic age acceleration estimate with fasting and postprandial (0- and 6-h time points) low-density lipoprotein (LDL), high-density lipoprotein (HDL), and triglyceride (TG) levels as well as five fasting inflammatory markers plus adiponectin. Both DNA methylation age estimates were highly correlated with chronological age (r > 0.90). We found that the Horvath and Hannum measures of epigenetic age acceleration were moderately correlated (r = 0.50). The regression models revealed that the Horvath age acceleration measure exhibited marginal associations with increased postprandial HDL (p = 0.05), increased postprandial total cholesterol (p = 0.06), and decreased soluble interleukin 2 receptor subunit alpha (IL2sRα, p = 0.02). The Hannum measure of epigenetic age acceleration was inversely associated with fasting HDL (p = 0.02) and positively associated with postprandial TG (p = 0.02), interleukin-6 (IL6, p = 0.007), C-reactive protein (C-reactive protein, p = 0.0001), and tumor necrosis factor alpha (TNFα, p = 0.0001). Overall, the observed effect sizes were small and the association of the Hannum residual with inflammatory markers was attenuated by adjustment for estimated T cell type percentages. Our study demonstrates that epigenetic age acceleration in blood relates to inflammatory biomarkers and certain lipid classes in Caucasian individuals of the GOLDN study. Future studies should consider epigenetic age acceleration in other tissues and extend the analysis to other ethnic groups.