Mission-critical applications that rely on deep learning (DL) for automation suffer because DL models struggle to provide reliable indicators of failure. Reliable failure prediction can greatly improve the efficiency of a system, because it becomes easier to predict when human intervention is required. DL-based systems thus stand to benefit greatly from robust measures of uncertainty over model predictions. Monte Carlo dropout (MCD), a Bayesian method, and deep ensembles (DE) have emerged as two of the most popular and competitive ways to perform uncertainty estimation. Although literature exploring the usefulness of these approaches exists in medical imaging, robotics and autonomous driving domains, it is scarce to non-existent for remote sensing, and in particular, synthetic aperture radar (SAR) applications. To close this gap, we have created a deep learning model for road extraction (hereafter referred to as segmentation) in SAR and use it to compare standard model outputs against the aforementioned most popular methods for uncertainty estimation, MCD and DE. We demonstrate that these methods are not effective as an indicator of segmentation quality when measuring uncertainty (as indicated by model softmax outputs) across an entire image but are effective when uncertainty is measured from the set of road predictions only. Furthermore, we show a marked improvement in the correlation between prediction uncertainty and segmentation quality when we increase the set of road predictions by including predictions with lower softmax scores. We demonstrate the efficacy of our application of MCD and DE methods with an experimental design that measures performance in real-world quality assessment using in-distribution (ID) and out-of-distribution (OOD) data. These results inform the development of mission-critical deep learning systems in remote sensing. Tasks in medical image analysis that have a similar morphology to road structures, such as blood vessel segmentation, can also benefit from our findings.

We propose a simple modification to standard ResNet architectures--L2 normalization over feature space--that substantially improves out-of-distribution (OoD) performance on the previously proposed Deep Deterministic Uncertainty (DDU) benchmark. We show that this change also induces early Neural Collapse (NC), an effect linked to better OoD performance. Our method achieves comparable or superior OoD detection scores and classification accuracy in a small fraction of the training time of the benchmark. Additionally, it substantially improves worst case OoD performance over multiple, randomly initialized models. Though we do not suggest that NC is the sole mechanism or a comprehensive explanation for OoD behaviour in deep neural networks (DNN), we believe NC's simple mathematical and geometric structure can provide a framework for analysis of this complex phenomenon in future work.

We demonstrate that L2 normalization over feature space can produce capable performance for Out-of-Distribution (OoD) detection for some models and datasets. Although it does not demonstrate outright state-of-the-art performance, this method is notable for its extreme simplicity: it requires only two addition lines of code, and does not need specialized loss functions, image augmentations, outlier exposure or extra parameter tuning. We also observe that training may be more efficient for some datasets and architectures. Notably, only 60 epochs with ResNet18 on CIFAR10 (or 100 epochs with ResNet50) can produce performance within two percentage points (AUROC) of several state-of-the-art methods for some near and far OoD datasets. We provide theoretical and empirical support for this method, and demonstrate viability across five architectures and three In-Distribution (ID) datasets.