In this paper, a simulation concept is presented for creating synthetic aperture radar (SAR) reflectivity maps based on ray tracing. Three-dimensional models of man-made objects are illuminated by a virtual SAR sensor whose signal is approximated by rays sent through the model space. To this end, open-source software tools are adapted and extended to derive output data in SAR geometry followed by creating the reflectivity map. Rays can be followed for multiple reflections within the object scene. Signals having different multiple reflection levels are stored in separate image layers. For evaluating the potentials and limits of the simulation approach, simulated reflectivity maps and distribution maps are compared with real TerraSAR-X images for various complex man-made objects like a skyscraper in Tokyo, the Wynn Hotel in Las Vegas, and the Eiffel Tower in Paris. The results show that the simulation can provide very valuable information to interpret complex SAR images or to predict the reflectivity of planned SAR image acquisitions.

Due to its all time capability, synthetic aperture radar (SAR) remote sensing plays an important role in Earth observation. The ability to interpret the data is limited, even for experts, as the human eye is not familiar to the impact of distance-dependent imaging, signal intensities detected in the radar spectrum as well as image characteristics related to speckle or steps of post-processing. This paper is concerned with machine learning for SAR-to-optical image-to-image translation in order to support the interpretation and analysis of original data. A conditional adversarial network is adopted and optimized in order to generate alternative SAR image representations based on the combination of SAR images (starting point) and optical images (reference) for training. Following this strategy, the focus is set on the value of empirical knowledge for initialization, the impact of results on follow-up applications, and the discussion of opportunities/drawbacks related to this application of deep learning. Case study results are shown for high resolution (SAR: TerraSAR-X, optical: ALOS PRISM) and low resolution (Sentinel-1 and -2) data. The properties of the alternative image representation are evaluated based on feedback from experts in SAR remote sensing and the impact on road extraction as an example for follow-up applications. The results provide the basis to explain fundamental limitations affecting the SAR-to-optical image translation idea but also indicate benefits from alternative SAR image representations.

Tasks such as the monitoring of natural disasters or the detection of change highly benefit from complementary information about an area or a specific object of interest. The required information is provided by fusing high accurate coregistered and georeferenced datasets. Aligned high-resolution optical and synthetic aperture radar (SAR) data additionally enable an absolute geolocation accuracy improvement of the optical images by extracting accurate and reliable ground control points (GCPs) from the SAR images. In this paper, we investigate the applicability of a deep learning based matching concept for the generation of precise and accurate GCPs from SAR satellite images by matching optical and SAR images. To this end, conditional generative adversarial networks (cGANs) are trained to generate SAR-like image patches from optical images. For training and testing, optical and SAR image patches are extracted from TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe. The artificially generated patches are then used to improve the conditions for three known matching approaches based on normalized cross-correlation (NCC), scale-invariant feature transform (SIFT), and binary robust invariant scalable key (BRISK), which are normally not usable for the matching of optical and SAR images. The results validate that a NCC-, SIFT-, and BRISK-based matching greatly benefit, in terms of matching accuracy and precision, from the use of the artificial templates. The comparison with two state-of-the-art optical and SAR matching approaches shows the potential of the proposed method but also revealed some challenges and the necessity for further developments.

Improving the geo-localization of optical satellite images is an important pre-processing step for many remote sensing tasks like monitoring by image time series or scene analysis after sudden events. These tasks require geo-referenced and precisely co-registered multi-sensor data. Images captured by the high resolution synthetic aperture radar (SAR) satellite TerraSAR-X exhibit an absolute geo-location accuracy within a few decimeters. These images represent therefore a reliable source to improve the geo-location accuracy of optical images, which is in the order of tens of meters. In this paper, a deep learning-based approach for the geo-localization accuracy improvement of optical satellite images through SAR reference data is investigated. Image registration between SAR and optical images requires few, but accurate and reliable matching points. These are derived from a Siamese neural network. The network is trained using TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe, in order to learn the two-dimensional spatial shifts between optical and SAR image patches. Results confirm that accurate and reliable matching points can be generated with higher matching accuracy and precision with respect to state-of-the-art approaches.

RaySAR, a 3D SAR simulator, has been made accessible to the SAR community in January 2016. It helps to understand and analyze reflection effects of radar signals at 3D object models. For instance, the nature of persistent scatterers or “ghost” scatterers at man-made structures or basic reflection effects at canonical shapes can be interpreted in more detail. The decision to provide RaySAR to the community was based on the persisting lack of a freely available SAR simulation package and the repetitive interpretation task related to the analysis of SAR imagery, especially in the context of high resolution data. Many signal reflection effects at objects remain to be discovered and explained. New interfaces between simulation techniques and SAR applications may be helpful. In this context, the paper provides a summary of the status-quo of RaySAR and addresses future directions that have not been followed so far.

The 2012 Data Fusion Contest organized by the Data Fusion Technical Committee (DFTC) of the IEEE Geoscience and Remote Sensing Society (GRSS) aimed at investigating the potential use of very high spatial resolution (VHR) multi-modal/multi-temporal image fusion. Three different types of data sets, including spaceborne multi-spectral, spaceborne synthetic aperture radar (SAR), and airborne light detection and ranging (LiDAR) data collected over the downtown San Francisco area were distributed during the Contest. This paper highlights the three awarded research contributions which investigate (i) a new metric to assess urban density (UD) from multi-spectral and LiDAR data, (ii) simulation-based techniques to jointly use SAR and LiDAR data for image interpretation and change detection, and (iii) radiosity methods to improve surface reflectance retrievals of optical data in complex illumination environments. In particular, they demonstrate the usefulness of LiDAR data when fused with optical or SAR data. We believe these interesting investigations will stimulate further research in the related areas.