8 December 2023 in bioRxiv
Predicting the effects of mutations in proteins is critical to many applications, from understanding genetic disease to designing novel proteins that can address our most pressing challenges in climate, agriculture and healthcare. Despite a surge in machine learning-based protein models to tackle these questions, an assessment of their respective benefits is challenging due to the use of distinct, often contrived, experimental datasets, and the variable performance of models across different protein families. Addressing these challenges requires scale. To that end we introduce ProteinGym, a large-scale and holistic set of benchmarks specifically designed for protein fitness prediction and design. It encompasses both a broad collection of over 250 standardized deep mutational scanning assays, spanning millions of mutated sequences, as well as curated clinical datasets providing high-quality expert annotations about mutation effects. We devise a robust evaluation framework that combines metrics for both fitness prediction and design, factors in known limitations of the underlying experimental methods, and covers both zero-shot and supervised settings. We report the performance of a diverse set of over 70 high-performing models from various subfields (eg., alignment-based, inverse folding) into a unified benchmark suite. We open source the corresponding codebase, datasets, MSAs, structures, model predictions and develop a user-friendly website that facilitates data access and analysis.
13 May 2021 in PLOS Computational Biology
Deep neural networks have demonstrated improved performance at predicting the sequence specificities of DNA- and RNA-binding proteins compared to previous methods that rely on k-mers and position weight matrices. To gain insights into why a DNN makes a given prediction, model interpretability methods, such as attribution methods, can be employed to identify motif-like representations along a given sequence. Because explanations are given on an individual sequence basis and can vary substantially across sequences, deducing generalizable trends across the dataset and quantifying their effect size remains a challenge. Here we introduce global importance analysis (GIA), a model interpretability method that quantifies the population-level effect size that putative patterns have on model predictions. GIA provides an avenue to quantitatively test hypotheses of putative patterns and their interactions with other patterns, as well as map out specific functions the network has learned. As a case study, we demonstrate the utility of GIA on the computational task of predicting RNA-protein interactions from sequence. We first introduce a convolutional network, we call ResidualBind, and benchmark its performance against previous methods on RNAcompete data. Using GIA, we then demonstrate that in addition to sequence motifs, ResidualBind learns a model that considers the number of motifs, their spacing, and sequence context, such as RNA secondary structure and GC-bias.
12 January 2026 in bioRxiv
Antibodies can bind their targets with exquisite potency and selectivity due in part to large antibody-target protein-protein interaction surface areas. Despite the very large size and diversity of synthetic libraries, in vitro sorting alone tends to yield binders with modest affinities. By analogy to the in vivo affinity maturation in the natural immune system, these initial hits are typically affinity matured in vitro to achieve high affinity binding. However, affinity maturation campaigns can be laborious, often requiring multiple selection rounds and strategies for each clone to be optimized. Here, we investigated whether one could accelerate the discovery of optimized binders using machine learning on sequencing data from single selection sorts of affinity maturation yeast-display campaigns. Our results show that sparse sequencing data from a single sorting round can predict sequences that are enriched after multiple rounds. We also find that linear models outperform deep neural networks and semi-supervised approaches in ranking validated affinity-enhancing substitutions. Linear models are also more interpretable, offering insights into residue preferences that can be leveraged for further engineering. We use our models to design and select optimized nanobody binders to relaxin family peptide receptor 1 (RXFP1), yielding multiple improved binders including 3 sub nanomolar binders with the best exhibiting a ∼2500-fold improvement over WT.