The production of high-resolution Digital Terrain Models (DTMs) based on images is often hampered by the lack of appropriate stereo observations. Here, we propose a deep learning-based reconstruction of pixel-resolution DTMs from Lunar Reconnaissance Orbiter (LRO) single-view Narrow Angle Camera (NAC) images, constrained by Selenological and Engineering Explorer and LRO LOLA Elevation Models (SLDEM). The procedure is carried out for a set of adjacent images, and the mosaicking of a contiguous large-area DTM is demonstrated. The approach is applied to the CE-3 and CE-4 landing sites, involving six multiple coverage and eight adjacent NAC L/R image pairs, respectively. For the DTM reconstruction, we use an improved Convolutional Neural Network (CNN) architecture with a weighted sum loss function involving three loss terms. We demonstrate that our method is robust and can deal with images acquired under varying illumination conditions. The DTM mosaic (1.5 m pixel size) covering the CE-4 landing area (72.8 km x 30.3 km) is without apparent seams between the individual image boundaries and consistent with the SLDEM (60 m pixel size) in terms of overall elevation, trend, and scale, but is showing considerably more morphologic detail.
12 January 2024 in IEEE Transactions on Geoscience and Remote Sensing
With the increasing number of high-resolution (HR) images captured by various platforms, integrating spectral and spatial properties of data across different HR image types, such as multispectral (MS), hyperspectral (HS), and multitemporal (MT) images, remains a challenging task for object classification. This article proposes a novel hybrid framework named hybrid FusionNet (HFN) that jointly exploits 2-D–3-D convolutional neural networks (CNNs) and a transformer encoder to address a complex classification problem. By incorporating 2-D and 3-D convolutional layers, the proposed HFN generates rich multidimensional hybrid features, including spectral, spatial, and temporal features. These features are then fed into a transformer encoder to learn global saliency and discriminative information, enabling the identification of spatially irregular and spectrally similar objects. The hybrid architecture efficiently captures local intricate spectral-spatial-temporal contextual features through convolutional layers. Then, it learns global long-range dependencies and the spectral dimension through the transformer encoder, thus effectively reducing spectral-spatial mutations, distortions, and variations of ground objects. Experimental results from an high-resolution multispectral (HR-MS) dataset, an high-resolution hyperspectral (HR-HS) dataset, and an high-resolution multitemporal (HR-MT) dataset covering complex urban scenarios confirm the effectiveness of the proposed approach compared to the main state-of-the-art methods. Notably, the proposed HFN can achieve satisfactory classification performance even with limited training samples. The source code will be made available at https://github.com/MissYongjie/Hybrid-FusionNet .
5 May 2024 in ISPRS Journal of Photogrammetry and Remote Sensing
High-resolution Digital Terrain Models (DTMs) are critical for supporting planetary exploration missions and advancing scientific research. Recently, Deep Learning (DL) techniques have been applied to reconstruct high-resolution DTMs from single-view orbiter optical images, particularly for the Moon. However, DL-based methods face challenges in retrieving high-quality multi-scale topographic features, especially in regions with irregular terrains or significant relief. Additionally, their generalization capability across diverse datasets is rarely evaluated. In this paper, we propose an efficient DL-based single-view method with a coarse-resolution DTM as a constraint for high-quality lunar DTM reconstruction, named ELunarDTMNet. This approach introduces a hierarchical transformer-based backbone with a residual-connected mechanism, specifically designed to capture and integrate multi-scale features from single-view lunar images, thereby enhancing prediction accuracy. Meanwhile, given the diverse and complex surface relief, new elevation normalization strategies are proposed to preserve terrain feature contrast while accommodating different elevation distributions. Our method performs well on diverse lunar landscapes with various topographic features and elevation changes. It outperforms existing DL-based methods in accuracy and detail, effectively addressing their encountered challenges. Moreover, the proposed method achieves effective resolutions similar to those of the Shape-From-Shading technique for subtle-scale terrain retrieval, but with enhanced elevation accuracy, illumination robustness, and approximately 850 times faster processing speed. Trained with the Lunar Reconnaissance Orbiter (LRO) Narrow Angle Camera (NAC) images, our model shows superior performance on other high-resolution lunar orbiter images, such as Chang’E-2 imagery.
13 May 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Mismatch removal is a crucial step in multi-view matching of lunar orbiter images. This process involves complex challenges like terrain-induced distortion and non-unique geometric structures due to repetitive textures. Traditional methods, whether global constraints or local constraints, fall short in adequately addressing these issues in orbiter imagery. Therefore, this paper proposes an effective method for mismatch removal of orbiter images based on global constraint and local geometry preservation combined with the imaging model. In this method, a clean neighborhood of each matching point based on the characteristic of centralized distribution of the back-projection residuals globally is constructed. In the local region, we define a local minimum geometric polygon consisting of the center feature point and its three neighbors, and combine the similarity of the back-projection difference vectors with the affine invariance of the polygon to distinguish the correct matches and mismatches by measuring the degree of local geometry preservation. A series of experiments encompassing parameter sensitivity analysis, comparison studies and ablation experiments were conducted on the Lunar Reconnaissance Orbiter image datasets to demonstrate the effectiveness and reliability of the proposed method. The results indicate that our method exhibits a notable insensitivity to parameter variations, and outperforms other advanced methods in both qualitative and quantitative evaluations. Moreover, a large-scale orbiter images multi-view matching tie points extraction framework is extended based on the proposed mismatch removal method, which can achieve better results than commonly used photogrammetric software in terms of the number and accuracy of tie points.
The Mars Orbiter Laser Altimeter (MOLA) Precision Experiment Data Records (PEDR) serve as the geodetic reference of Mars. However, these MOLA footprints were geolocated using outdated auxiliary information that dates back to 2003. In this study, we recompute the MOLA PEDR footprint locations and investigate the impact of the updated spacecraft orbit model and Mars rotational model on MOLA’s geolocation. We observe quasi-exponential increases near the poles of up to 30 m in the recomputation residuals for the nadir profiles. Meanwhile, we demonstrate that limitations exist in the stored MOLA PEDR attitude records, which can shift the footprint up to hundreds of meters laterally and several meters radially. The usage of the Navigation and Ancillary Information Facility (NAIF)-archived attitude information instead can circumvent this issue and avoid the approximation errors due to discrete samplings of the attitude information used in geolocation by the PEDR dataset. These approximation errors can be up to 60 m laterally and 1 m radially amid controlled spacecraft maneuvers. Furthermore, the incorporation of the updated spacecraft orbit and Mars rotational model can shift the MOLA profiles up to 200 m laterally and 0.5 m radially, which are much larger in magnitude than the aforementioned dramatic increases near the poles. However, the shifted locations of the reprocessed profiles are significantly inconsistent with the PEDR profiles after the global cross-over analysis.