7 July 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models (LMs) taken from Natural Language Processing (NLP). These LMs reach for new prediction frontiers at low inference costs. Here, we trained two auto-regressive models (Transformer-XL, XLNet) and four auto-encoder models (BERT, Albert, Electra, T5) on data from UniRef and BFD containing up to 393 billion amino acids. The protein LMs (pLMs) were trained on the Summit supercomputer using 5616 GPUs and TPU Pod up-to 1024 cores. Dimensionality reduction revealed that the raw pLM- embeddings from unlabeled data captured some biophysical features of protein sequences. We validated the advantage of using the embeddings as exclusive input for several subsequent tasks: (1) a per-residue (per-token) prediction of protein secondary structure (3-state accuracy Q3=81%-87%); (2) per-protein (pooling) predictions of protein sub-cellular location (ten-state accuracy: Q10=81%) and membrane versus water-soluble (2-state accuracy Q2=91%). For secondary structure, the most informative embeddings (ProtT5) for the first time outperformed the state-of-the-art without multiple sequence alignments (MSAs) or evolutionary information thereby bypassing expensive database searches. Taken together, the results implied that pLMs learned some of the grammar of the language of life . All our models are available through https://github.com/agemagician/ProtTrans .
14 March 2025 in Nature Communications
Early life stress (ELS) can increase vulnerability to psychiatric disorders, but also trigger resilience. FKBP51 has been associated with an increased risk for developing psychiatric disorders, specifically in interaction with ELS exposure. Here, the contribution of FKBP51 in glutamatergic forebrain neurons to the long-term consequences of ELS was investigated in both sexes. In female wild-type Fkbp5
lox/lox
mice, ELS exposure led to an anxiolytic phenotype and improved memory performance in a stressful context, however this ELS effect was absent in Fkbp5
Nex
mice. These interactive FKBP51 x ELS effects in female mice were also reflected in reduced brain region volumes, and on structural and electrophysiological properties of CA1 pyramidal neurons of the dorsal hippocampus. In contrast, the behavioral, structural and functional effects in male ELS mice were less pronounced and independent of FKBP51. RNA sequencing of the hippocampus revealed the transcription factor 4 (TCF4) as a potential regulator of the female interactive effects. Cre-dependent viral overexpression of TCF4 in female Nex-Cre mice led to similar beneficial effects on behavior as the ELS exposure. This study demonstrates a sex-specific role for FKBP51 in mediating the adaptive effects of ELS on emotional regulation, cognition, and neuronal function, implicating TCF4 as a downstream effector.
13 August 2025 in iScience
Isoform-specific expression patterns have been linked to stress-related psychiatric disorders such as major depressive disorder (MDD). To further explore their involvement, we constructed co-expression networks using total gene expression (TE) and isoform ratio (IR) data from affected (n = 210, 81% with depressive symptoms) and unaffected (n = 95) individuals. Networks were validated using advanced graph generation methods. Our analysis revealed distinct differences in network topology and structure. Shared hubs exhibited unique co-regulatory patterns in each network, with key master hubs in the affected network showing association with psychiatric disorders. Gene Ontology enrichment highlighted condition-specific biological processes linked to each network’s master hubs. Notably, isoform-level data uncovered unique co-regulatory interactions and enrichments not observed at the gene level. This is the first study to show network-level differences of gene and isoform co-expression between affected and unaffected individuals of stress-related psychiatric disorders, emphasizing the importance of isoforms in understanding the molecular mechanisms of these conditions.