Protein engineering increasingly relies on machine learning models to computationally pre-screen promising novel candidates. Although machine learning approaches have proven effective, their performance on prospective screening data leaves room for improvement; prediction accuracy can vary greatly from one protein variant to the next. So far, it is unclear what characterizes variants that are associated with large prediction error. In order to establish whether structural characteristics influence predictability, we created a novel high-order combinatorial dataset for an enzyme spanning 3,706 variants, that can be partitioned into subsets of variants with mutations at positions exclusively belonging to a particular structural class. By training four different supervised variant effect prediction (VEP) models on structurally partitioned subsets of our data, we found that predictability strongly depended on all four structural characteristics we tested; buriedness, number of contact residues, proximity to the active site and presence of secondary structure elements. These dependencies were also found in several single mutation enzyme variant datasets, albeit with dataset specific directions. Most importantly, we found that these dependencies were similar for all four models we tested, indicating that there are specific structure and function determinants that are insufficiently accounted for by current machine learning algorithms. Overall, our findings suggest that improvements can be made to VEP models by exploring new inductive biases and by leveraging different data modalities of protein variants, and that stratified dataset design can highlight areas of improvement for machine learning guided protein engineering.
The grand challenge of protein engineering is the development of computational models to characterize and generate protein sequences for arbitrary functions. Progress is limited by lack of 1) benchmarking opportunities, 2) large protein function datasets, and 3) access to experimental protein characterization. We introduce the Protein Engineering Tournament—a fully-remote competition designed to foster the development and evaluation of computational approaches in protein engineering. The tournament consists of an in silico round, predicting biophysical properties from protein sequences, followed by an in vitro round where novel protein sequences are designed, expressed and characterized using automated methods. Upon completion, all datasets, experimental protocols, and methods are made publicly available. We detail the structure and outcomes of a pilot Tournament involving seven protein design teams, powered by six multi-objective datasets, with experimental characterization by our partner, International Flavors and Fragrances. Forthcoming Protein Engineering Tournaments aim to mobilize the scientific community towards transparent evaluation of progress in the field. Graphical
Porcine reproductive and respiratory syndrome virus (PRRSV) remains a major challenge to swine production worldwide. Current vaccines have limited efficacy against genetically diverse PRRSV strains. Therefore, strategies with alternative modes of action—such as antiviral approaches that target conserved virus–host interactions, including viral attachment and entry, rather than relying solely on adaptive immune responses—are needed. We first evaluated the in vitro effect of griffithsin (GRFT), a high-mannose-binding lectin, in the monkey kidney cell line MARC-145. Cells were pre-treated with GRFT (50–200 µg/mL) prior to PRRSV infection, after which cell morphology and viral RNA replication (measured by RT-qPCR) were assessed. Pre-treatment with 100–200 µg/mL GRFT, followed by PRRSV inoculation at a multiplicity of infection of 1 or 10, reduced viral replication in MARC145 cells in a dose-dependent manner, achieving almost 100% inhibition of ORF5 and ORF7 RNA compared with untreated controls (p < 0.0001). We next investigated the in vivo effects of intranasal GRFT administration (7.5 or 15 mg/day) in pigs (n = 56). Pigs treated with 15 mg/day GRFT exhibited significantly reduced (p < 0.05) viremia 2, 4 and 7 days post-challenge, compared with untreated, challenged, and controls (log10 8.1 ± 0.2 vs. 9.0 ± 0.25, 8.2 ± 0.1 vs. 9.1 ± 0.2, and 8.9 ± 0.2 vs. 9.3 ± 0.2, respectively), along with earlier resolution of fever and a trend toward increased average daily gain over 42 days (p < 0.1). These findings are the first report of GRFT efficacy in pigs and support its potential as an antiviral strategy against PRRSV, alongside existing interventions.