Earth observation using remote sensing data is a trending research field across the globe. In this perspective, image registration is a mandatory data processing step for any kind of time series data analytics. Feature-based image registration is one of the prominent categories for superimposing multi-temporal and multi-modal images over each other. The paper provides a comprehensive and comparative survey of feature detection/description techniques for remote sensing images. In addition, outlier removal algorithms are explored in detail to generate putative keypoint correspondences for accurate transformation parameter estimation. The experiments are conducted on multiple remote sensing image pairs comprising visible, Synthetic Aperture Radar (SAR) and infrared images that provide diverse characteristics of the feature target. The co-registration accuracy is quantified for all possible combinations of feature detection/description with outlier removal, and best amalgamation is visually verified to check sub-pixel spatial alignment by swiping multi-temporal or multi-modal images over each other. It has been found that SIFT performs better in optical image registration, whereas A-KAZE feature detection has an upper edge at SAR image registration task. Marginalizing Sample Consensus (MAGSAC) and Mode Guided (MG) based outlier removal techniques achieves better pruning performance for multi-temporal optical and SAR images respectively. The comparative evaluation indicates that Motion Smoothness Constraint (MSC) optimization shows optimal performance for multi-modal remote sensing images. The best possible Root Mean Square Error (RMSE) achieved for multi-temporal optical images is 0.44 pixel whereas for multi-temporal SAR images, it is 0.46 pixels. The RMSE achieved for multi-modal images is 0.48 pixel using combination of SIFT with MSC.
27 May 2021 in IEEE Transactions on Geoscience and Remote Sensing
Remote-sensing image registration is a pivotal preprocessing step for earth observation data analytics. In this regard, georeferencing corrects the systematic geometric degradation in the image. However, it is difficult to achieve subpixel geometric accuracy across multitemporal scenes. This article focuses on hindmost part of geometric correction that uses reference layer and feature detection in hierarchical stages to improve the geometric fidelity of images at subpixel level. The methodology developed is based on patch affine-oriented fast and brief with mode-guided tiled scale invariant feature transform (MT-SIFT) techniques in a coordinate manner at a multistage processing architecture, which we refer to as FIRM. Motion smoothness constraint (MSC) keypoint correspondence optimization is used in FIRM to remove the outliers at gross stage and estimate segmented affine transformation parameters at finer stage. The automatic coregistration pipeline is evaluated in Indian Resourcesat multispectral camera images covering diverse landscapes. The capability of the designed framework is demonstrated to handle relatively large geometrical error. With more than a decade difference in acquisitions, multitemporal images are superimposed over each other and compared with state-of-the-art feature-based methods. The potential of the proposed approach FIRM is assessed on multisatellite imagery acquired from Resourcesat-2 and Landsat -8. It is observed that the root mean square error (RMSE) between coregistered images is 0.12 pixel at a spatial resolution of 5 m.
Band-to-Band Registration (BBR) is a pre-requisite image processing operation essential for specific remote sensing multispectral sensors. BBR aims to align spectral wavelength channels at sub-pixel level accuracy over each other. The paper presents a novel BBR technique utilizing Co-occurrence Scale Space (CSS) for feature point detection and Spatial Confined RANSAC (SC-RANSAC) for removing outlier matched control points. Additionally, the Segmented Affine Transformation (SAT) model reduces distortion and ensures consistent BBR. The methodology developed is evaluated with Nano-MX multispectral images onboard the Indian Nano Satellite (INS-2B) covering diverse landscapes. BBR performance using the proposed method is also verified visually at a 4X zoom level on satellite scenes dominated by cloud pixels. The band misregistration effect on the Normalized Difference Vegetation Index (NDVI) from INS-2B is analyzed and cross-validated with the closest acquisition Landsat-9 OLI NDVI map before and after BBR correction. The experimental evaluation shows that the proposed BBR approach outperforms the state-of-the-art image registration techniques.
1 May 2024 in IEEE Transactions on Geoscience and Remote Sensing
Clouds in optical satellite images are a major concern since their presence hinders the ability to carry out accurate analysis as well as processing. Therefore, deriving accurate pixel-wise cloud masks is a key task in optical remote sensing. Several traditional as well as deep-learning algorithms have emerged to address this problem. However, the task of deriving accurate cloud masks from a variety of satellite images remains elusive due to the presence of confusing spectral signatures and changes in the properties of imaging sensors. In this article, we introduce a deep-learning model based on a hybrid transformer architecture for effective cloud mask generation named C-LiSA - cloud segmentation via the Lipschitz stable attention network. We propose two key attention mechanisms - dual orthogonal self-attention (DOSA) for handling confusing spectral signatures, and the hierarchical cross-channel attention (HC2A) model for effectively highlighting cloud-specific features during the cloud segmentation process. To validate the effectiveness of these mechanisms, we carry out theoretical and empirical Lipschitz stability analysis. We design the whole setup under an adversarial setting in the presence of Lovász-Softmax loss. We demonstrate both qualitative and quantitative outcomes for multiple satellite image datasets including Landsat-8, Sentinel-2, and Cartosat-2S. Our comparative study shows that the proposed model performs better compared to other state-of-the-art methods while providing better generalization across different datasets. We also showcase ablation studies to endorse our choices corresponding to different architectural elements and objective functions.