Due to increasingly complex factors of image degradation, inferring high-frequency details of remote sensing imagery is more difficult compared to ordinary digital photos. This paper proposes an adaptive multi-scale feature fusion network (AMFFN) for remote sensing image super-resolution. Firstly, the features are extracted from the original low-resolution image. Then several adaptive multi-scale feature extraction (AMFE) modules, the squeeze-and-excited and adaptive gating mechanisms are adopted for feature extraction and fusion. Finally, the sub-pixel convolution method is used to reconstruct the high-resolution image. Experiments are performed on three datasets, the key characteristics, such as the number of AMFEs and the gating connection way are studied, and super-resolution of remote sensing imagery of different scale factors are qualitatively and quantitatively analyzed. The results show that our method outperforms the classic methods, such as Super-Resolution Convolutional Neural Network(SRCNN), Efficient Sub-Pixel Convolutional Network (ESPCN), and multi-scale residual CNN(MSRN).
30 March 2020 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Accurate estimation of GDP at small geographies is of great significance to evaluate the distribution and dynamics of socio-economic development. Nighttime light(NTL) data is becoming increasingly important in socio-economic data estimation. However, previous research has found that using NTL alone is insufficient to accurately measure the GDP at small geographies and the contribution of NTL for time-series GDP estimation is unreliable. This paper proposed a Deep Learning method for the CONUS time-series (2012-2015) GDP estimation at the county-level. The model is developed by combining the NTL data from the Visible Infrared Imaging Radiometer Suite Day/Night Band and the MODIS land cover data. The proposed method can improve the existing methods mainly in two ways. First, by taking advantage of the great computing power of the GEE, a histogram-based feature regulation method was employed, which not only keeps more information over regions but also provides dimension-reduced tensors from mass remote sensing data. Second, a Multi-Inputs Convolutional Neural Network-based model was proposed instead of the traditional linear regression model for multi-source feature exploration and learning. The proposed method was evaluated by leave-one-year-out cross-validation with the time-series(2012-2015) data. The results show that the $R^{2}$ between the actual and estimated GDP are 0.81, 0.83, 0.83, and 0.83 for years from 2012 to 2015, indicating a good predictive power of the proposed model. Given that the data employed are globally and publicly available, the proposed method would also be applicable in other countries or regions where socio-economic survey data is hard to obtain.
High-resolution optical remote sensing image registration is still a challenging task due to non-linearity in the intensity differences and geometric distortion. In this paper, an efficient method utilizing a hyper-graph matching algorithm is proposed, which can simultaneously use the high-order structure information and radiometric information, to obtain thousands of feature point pairs for accurate image registration. The method mainly consists of the following steps: firstly, initial matching by Uniform Robust Scale-Invariant Feature Transform (UR-SIFT) is carried out in the highest pyramid image level to derive the approximate geometric relationship between the images; secondly, two-stage point matching is performed to find the matches, that is, a rotation and scale invariant area-based matching method is used to derive matching candidates for each feature point and an efficient hyper-graph matching algorithm is applied to find the best match for each feature point; thirdly, a local quadratic polynomial constraint framework is used to eliminate match outliers; finally, the above process is iterated until finishing the matching in the original image. Then, the obtained correspondences are used to perform the image registration. The effectiveness of the proposed method is tested with six pairs of high-resolution optical images, covering different landscape types—such as mountain area, urban, suburb, and flat land—and registration accuracy of sub-pixel level is obtained. The experiments show that the proposed method outperforms the conventional matching algorithms such as SURF, AKAZE, ORB, BRISK, and FAST in terms of total number of correct matches and matching precision.
To address the issues of loss of important detailed features, insufficient contrast enhancement, and high computational complexity in existing low-light image enhancing methodologies, this paper presents a low-light image enhancement network (MSF-ACA), which uses multi-scale feature fusion and adaptive contrast adjustment. Focus is placed on designing the local–global image feature fusion module (LG-IFFB) and the adaptive image contrast enhancement module (AICEB), in which the LG-IFFB adopts the local–global dual-branching structure to extract multi-scale image features, and utilizes the element-by-element multiplication method to fuse the local details with the global illumination distribution to alleviate the problem of serious loss of image details, while the AICEB incorporates linear contrast enhancement and confidence adaptive stopping mechanism, which dynamically adjusts the computational depth according to the confidence of the feature map, balancing the contrast enhancement and computational efficiency. According to the results of the experiment, the parameter count of MSF-ACA is 0.02 M, and compared with today’s mainstream algorithms, the suggested model attains 21.53 dB in PSNR when evaluated on the LOL-v2-real evaluation dataset, and the BRI is as low as 16.04 on the unpaired dataset DICM, which provides a better detail clarity and color fidelity in visual enhancement, and it is a highly efficient and robust low-light image model.
To address the issues of loss of important detailed features, insufficient contrast enhancement and high computational complexity in existing low-light image enhancing methodologies, this paper presents a low-light image enhancement network(MSF-ACA), which uses multi-scale feature fusion and adaptive contrast adjustment. Focus is placed on designing the local-global image feature fusion module (LG-IFFB) and the adaptive image contrast enhancement module (AICEB), in which the LG-IFFB adopts the local-global dual-branching structure to extract multi-scale image features, and utilizes the element-by-element multiplication method to fuse the local details with the global illumination distribution to alleviate the problem of serious loss of image details, while the AICEB fuses the linear contrast The AICEB incorporates linear contrast enhancement and confidence adaptive stopping mechanism, which dynamically adjusts the computational depth according to the confidence of the feature map, balancing the contrast enhancement and computational efficiency. According to the results of the experiment, the parameter count of MSF-ACA is 0.02M, and compared with today's mainstream algorithms, the suggested model attains 21.53dB in PSNR when evaluated on the LOL-v2-real evaluation dataset, and the BRI is as low as 16.04 on the unpaired dataset DICM, which provides a better detail clarity and color fidelity in visual enhancement, and it is a highly efficient and robust low-light image model.
Attention mechanisms have achieved remarkable success in the field of computer vision. However, in the context of image matching with large deformations, attention often suffers from divergence issues. To address this, we propose a geometry-aware attention-based matching method called GeoAT. This method leverages low-resolution image features to obtain global geometric constraint information between images and uses an affine transformation matrix to guide the subsequent attention computation on high-resolution features, achieving efficient and accurate matching in a coarse-to-fine manner. One of the key innovations of GeoAT is leveraging the affine transformation matrix derived from the coarse matching stage to guide the cross-attention process in the intermediate matching stage. This enables the neural network to focus primarily on regions with potential correspondences while minimizing attention to irrelevant areas, thereby improving the precision and efficiency of the correspondence identification process. GeoAT employs a flexible window-based attention mechanism, integrating self-attention and cross-attention structures, while fusing multi-scale features and positional information. It demonstrates excellent performance in scenarios with large angle variations and achieves highly accurate matching results even under high matching thresholds. We evaluated GeoAT on two well-known datasets, HPatches and MegaDepth, where it outperformed the current state-of-the-art algorithm, LoFTR, with improvements of 3-5% across various metrics. In the future, GeoAT will focus on further optimizing computational efficiency and enhancing robustness.