Cloud contamination greatly limits the potential utilization of optical images for geoscience applications. An effective alternative is to extract data from synthetic aperture radar (SAR) images to remove clouds due to the strong penetration ability of microwaves. In this article, we propose a novel unified spatial–spectral residual network that utilizes SAR images as auxiliary data to remove clouds from optical images. The method can better establish the relationship between SAR and optical images and be divided into two modules: feature extraction and fusion module and reconstruction module. In the feature extraction and fusion module, a gated convolutional layer is introduced to discriminate cloud pixels from clean pixels, which makes up for the lack of distinguishing ability of vanilla convolutional layers and avoids the error of cloud areas in feature extraction. In the reconstruction module, spatial and channel attention mechanisms are introduced to obtain global spatial and spectral information. The network is tested on three datasets with different spatial resolutions and compositions of land covers to verify the effectiveness and applicability of the method. The results show that the method outperforms other mainstream algorithms that simultaneously use SAR images as auxiliary data with a gain of about 2.3 dB in terms of peak signal-to-noise ratio PSNR on the SEN12MS-CR dataset.

Existing infrared (IR) small target detection methods are divided into local priors-based and nonlocal priors-based ones. However, due to heterogeneous structures in IR images, using either local or nonlocal information is always suboptimal, which causes detection performance to be unstable and unrobust. To solve the issue, a comprehensive method, exploiting both local and nonlocal priors, is proposed. The proposed method framework includes dual-window local contrast method (DW-LCM) and multiscale-window IR patch-image (MW-IPI). In the first stage, DW-LCM designs dual-window to compensate for the shortcomings of local priors-based methods, which easily mistake some isolated weak-signal targets. In the second stage, MW-IPI utilizes several small windows with various sizes, which can not only decrease the redundant information generated by sliding windows, but also extract more discriminative information to prevent some pixels in the strong-border edge from being falsely detected. Then, multiplication pooling operation is employed to enhance the target separation and suppress the background clutter simultaneously. Experimental results using five real IR datasets with various scenes reveal the effectiveness and robustness of the proposed method.

The trade-off between spatial and temporal resolution limits the acquisition of dense time series of Landsat images, and limits the ability to properly monitor land surface dynamics in time. Spatiotemporal image fusion methods provide a cost-efficient alternative to generate dense time series of Landsat-like images for applications that require both high spatial and temporal resolution images. The Spatial and Temporal Reflectance Unmixing Model (STRUM) is a kind of spatial-unmixing-based spatiotemporal image fusion method. The temporal change image derived by STRUM lacks spectral variability and spatial details. This study proposed an improved STRUM (ISTRUM) architecture to tackle the problem by taking spatial heterogeneity of land surface into consideration and integrating the spectral mixture analysis of Landsat images. Sensor difference and applicability with multiple Landsat and coarse-resolution image pairs (L-C pairs) are also considered in ISTRUM. Experimental results indicate the image derived by ISTRUM contains more spectral variability and spatial details when compared with the one derived by STRUM, and the accuracy of fused Landsat-like image is improved. Endmember variability and sliding-window size are factors that influence the accuracy of ISTRUM. The factors were assessed by setting them to different values. Results indicate ISTRUM is robust to endmember variability and the publicly published endmembers (Global SVD) for Landsat images could be applied. Only sliding-window size has strong influence on the accuracy of ISTRUM. In addition, ISTRUM was compared with the Spatial Temporal Data Fusion Approach (STDFA), the Enhanced Spatial and Temporal Adaptive Reflectance Fusion Model (ESTARFM), the Hybrid Color Mapping (HCM) and the Flexible Spatiotemporal DAta Fusion (FSDAF) methods. ISTRUM is superior to STDFA, slightly superior to HCM in cases when the temporal change is significant, comparable with ESTARFM and a little inferior to FSDAF. However, the computational efficiency of ISTRUM is much higher than ESTARFM and FSDAF. ISTRUM can to synthesize Landsat-like images on a global scale.

In observations of Earth, the existence of clouds affects the quality and usability of optical remote sensing images in practical applications. Many cloud removal methods have been proposed to solve this issue. Among these methods, synthetic aperture radar (SAR)-based methods have more potential than others because SAR imaging is hardly affected by clouds, and can reflect ground information differences and changes. While SAR images used as auxiliary information for cloud removal may be blurred and noisy, the similar non-local information of spectral and electromagnetic features cannot be effectively utilized by traditional cloud removal methods. To overcome these weaknesses, we propose a novel cloud removal method using SAR-optical data fusion and a graph-based feature aggregation network (G-FAN). First, cloudy optical images and contemporary SAR images are concatenated and transformed into hyper-feature maps by pre-convolution. Second, the hyper-feature maps are inputted into the G-FAN to reconstruct the missing data of the cloud-covered area by aggregating the electromagnetic backscattering information of the SAR image, and the spectral information of neighborhood and non-neighborhood pixels in the optical image. Finally, post-convolution and a long skip connection are adopted to reconstruct the final predicted cloud-free images. Both the qualitative and quantitative experimental results from the simulated data and real data experiments show that our proposed method outperforms traditional deep learning methods for cloud removal.

Optical remote sensing images, as a significant data source for Earth observation, are often impacted by cloud cover, which severely limits their widespread application in Earth sciences. Synthetic aperture radar (SAR), with its all-weather, all-day observation capabilities, serves as a valuable auxiliary data source for cloud removal (CR) tasks. Despite substantial progress in deep learning (DL)-based CR methods utilizing SAR data in recent years, challenges remain in preserving fine texture details and maintaining image visual authenticity. To address these limitations, this study proposes a novel diffusion-based CR method called the Dual-branch Multimodal Conditional Guided Diffusion Model (DMDiff). Considering the intrinsic differences in data characteristics between SAR and optical images, we design a dual-branch feature extraction architecture to enable adaptive feature extraction based on the characteristics of the data. Then, a cross-attention mechanism is employed to achieve deep fusion of the multimodal feature extracted above, effectively guiding the progressive diffusion process to restore cloud-covered regions in optical images. Furthermore, we propose an image adaptive prediction (IAP) strategy within the diffusion model, specifically tailored to the characteristics of remote sensing data, which achieves a nearly 20 dB improvement in PSNR compared to the traditional noise prediction (NP) strategy. Extensive experiments on the airborne, WHU-OPT-SAR, and LuojiaSET-OSFCR datasets demonstrate that DMDiff outperforms SOTA methods in terms of both signal fidelity and visual perceptual quality. Specifically, on the LuojiaSET-OSFCR dataset, our method achieves a remarkable 17% reduction in the FID metric over the second-best method, while also yielding significant enhancements in quality assessment metrics such as PSNR and SSIM.

The timely and accurate prediction of winter wheat yields is of importance in maintaining food security. However, existing deep-learning methods used for crop yield prediction are limited. While most methods utilize recurrent neural networks (RNNs) to interpret crop time series data, they struggle to learn geographical spatial information from input data and often prove challenging to interpret with prior knowledge. In this study, hyperspectral images were used as the input to a BS-Nets network for band selection, and a graph-based RNN framework GT-long-short-term memory (LSTM) [two channels based LSTM-graph neural network (GNN)], was proposed for predicting the winter wheat yield at the county level. Using the BS-Nets for hyperspectral bands selection at the field scale, we obtained the top 4 bands in selection results for all six stages, the results were band 46 (791 nm), 50 (825 nm), 66 (954 nm), and 161 (2484 nm). Based on the results of hyperspectral bands selection, at the county scale, the similar wavelength bands of Sentinel-2 red-edge 3 (783 nm), NIR (834 nm), red-edge 4 (865 nm), SWIR2 (2190 nm) were chosen as inputs for the GT-LSTM county-level estimates of winter wheat yield. When only remote sensing data were used, the highest prediction accuracy ( $R^{2}= 0.688$ , RMSE = 0.54 t/ha) was obtained for DOY135 (30 days before harvest). The incorporation of the meteorological data improved the accuracy by 7% ( $R^{2}= 0.714$ , RMSE = 0.50 t/ha), and the optimal time for predicting wheat yield was at DOY115 (50 days before harvest). Further addition of GNN layers to the model improved the accuracy of the results by an additional 14% ( $R^{2}= 0.757$ , RMSE = 0.43 t/ha), and the best prediction results were then obtained at DOY105 (60 days before harvest).