UAV (Unmanned Aerial Vehicle)-to-satellite self-localization is commonly treated as satellite tile retrieval. This makes large-area search tractable, but it also forces a continuous localization problem into a discrete ranking form. Once the task is defined this way, training naturally relies on hard positive–negative tile labels, and inference tends to read coordinates from the top-ranked tile center. The model, therefore, learns image identity more than geographic continuity, while the final estimate remains vulnerable to tile-center quantization and top-1 retrieval errors. We propose PFIRNet (Probability Field Inference Network), a continuous geographic posterior inference framework that reformulates retrieval outputs as evidence for coordinate estimation rather than discrete tile selection. It uses distance-aware geographic supervision to shape candidate responses according to metric proximity, lifts top-k candidates into a coordinate-space probability field, and applies risk-calibrated multi-peak verification to update the estimate only when an alternative posterior peak is sufficiently supported. On DenseUAV, PFIRNet reduces the median localization error to 9.84 m and outperforms both one-stage retrieval methods and two-stage matching baselines. It also remains more robust under sparse and non-aligned satellite galleries.