Machine learning is becoming a gatekeeper for metal–organic framework (MOF) discovery, but its credibility depends on whether validation matches the chemistry of deployment. Here, we show that the train/test split is not a technical choice, but the statement of what a model can legitimately support. In a controlled, database-scale adsorption benchmark, we hold targets, descriptors, models, hyperparameters, and metrics fixed while comparing random, geometry-, metal-cluster-, functional-group-, linker-, and topology-held-out splits for CO2 and CH4 uptake. Random, geometry-, functional-group-, and linker-held-out evaluations support broadly similar conclusions, whereas metal-cluster and especially topology holdouts reveal much harsher transfer regimes. For CO2 uptake at 0.15 bar, random-forest performance drops from R2 = 0.758 under random splitting to 0.267 under topology grouping, and top-5% recovery falls from 62.2% to 37.6%. We introduce split-aware, structurally auditable validation as a foundation for trustworthy AI-guided porous-material discovery. Significance statement. High-throughput MOF screening increasingly uses machine learning to decide which candidates should be simulated, synthesized, or inspected experimentally. This paper shows that such decisions cannot be judged by random-split accuracy alone. The same data, descriptors, models, and metrics can support a strong interpolation claim while weakening under a harder reticular-transfer claim. By treating the split as a chemistry-facing variable, we show that validation design changes not only regression scores, but also the identity of candidate shortlists. The resulting validation framework turns model evaluation into a traceable materials-screening argument: define the transfer problem, disclose elite-list survival, and route validation-fragile candidates to structural inspection, uncertainty analysis, additional simulation, or targeted data acquisition.