1 February 2022 in Knowledge-Based Systems
4 November 2026 in bioRxiv
Searching large databases of predicted protein structures can reveal remote homologs, but reliable discovery requires accurate significance estimates. We show that native E-values reported by protein structure search tools such as Foldseek and Reseek can be unreliable, leading to false discoveries or missed homologs. To address this problem, we introduce Struct-Caliper, which provides reliable query-specific p-values for protein structure search. The method uses representation-aware query randomization to generate background queries and compares observed alignment scores with the highest scores from randomized searches of the same database. In simulations, SG p-values generally kept reported false positive at or below the chosen cutoff under the evaluated randomization procedures. In searches of 15,177 ASTRAL40 structures, SG p-values increased Foldseek homolog recovery by 18.2% with slightly fewer false discoveries compared with native E-values at a cutoff of 0.01. For Reseek, false discoveries even decreased by 99.5%. Across 73 queries searched against more than two million AlphaFoldDB structures, SG p-values increased the mean recovery of DALI-supported matches by 23.5% at the same cutoff. These results show that Struct-Caliper can improve the reliability of structure searches and help prioritize discoveries for further biological analysis.
4 November 2026 in bioRxiv
Protein language models (PLMs) are powerful predictors of protein interactions, but strong benchmark performance does not ensure reliable discoveries. We introduce PPI-Caliper, a model-agnostic framework that adapts conformal calibration to select interactions at user-specified false discovery rate (FDR) levels using held-out labels without retraining the predictor. Across five species, two PLM-based predictors, and two human training-data versions for each predictor, PPI-Caliper keeps empirical FDR at or below the target level under in-distribution calibration. Under distribution shift, transferred thresholds tend to become conservative when predictive performance improves, but can fail to control FDR when performance deteriorates. PPI-Caliper connects pre-trained interaction scores to explicit error targets and demonstrates why reliable biological discovery requires evaluating calibration alongside predictive accuracy.