Distributed optical fiber sensing (DOFS) based on phase-sensitive optical time-domain reflectometry (ϕ-OTDR) is increasingly used for perimeter and pipeline monitoring, yet most recognizers are evaluated as closed-set classifiers, whereas a deployed fiber also records nuisance sources and event types absent from training. In our experiments, a closed-set recognizer with 0.999 accuracy still gives a false-alarm rate (FAR) of 0.42–0.64 on simulated unknown nuisance classes, and a conformal threshold calibrated only on known negatives does not remove this failure. We therefore propose Physically Consistent Risk Calibration (PCRC), which combines a label-free physical-consistency gate computed from spatial compactness, common-mode ratio, and signal-to-noise ratio (SNR) with a conformal threshold for gate-passing negatives and a three-level alarm/review/discard decision. The guarantee is conditional: conformal calibration controls gate-passing negatives under exchangeability, and the gate removes only physically inadmissible nuisance windows. On three public distributed acoustic sensing (DAS) field datasets, the known-negative FAR follows the target level when calibration and test negatives are exchangeable, held-out threat coverage reaches 0.87 and 0.68 on the two multichannel corpora, and every physically admissible held-out nuisance passes the gate and requires recalibration from verified field negatives. In a controlled simulation whose unknown nuisance classes are constructed to be inadmissible, PCRC produces no observed automatic false alarms while retaining 0.998 known-threat recall.