23 April 2025 in IEEE Transactions on Geoscience and Remote Sensing
Semantic 3D reconstruction from multi-view images is essential for applications such as 3D city modeling and robot navigation. However, existing methods treat semantic segmentation and height estimation as separate tasks, leading to suboptimal reconstruction results. To bridge this gap, we introduce MVSR3D, the first end-to-end framework for semantic 3D reconstruction using multi-view satellite images. MVSR3D employs a dual-stream architecture, consisting of the segmentation branch (MVSAM) based on Segment Anything Model (SAM) and the height estimation branch based on multi-view stereo. To enhance multi-view feature fusion, we propose the Epipolar Cross Attention (ECA) module in the MVSAM branch, which integrates image embeddings primarily along epipolar line to exploit complementary multi-view information. Unlike conventional multi-task learning approaches, we design dedicated interaction modules—the SAM Feature-Guided (SAM-FG) module and the Elevation-Guided Sparse Prompts Generator (EGSPG)—to facilitate multi-task interaction and feature fusion. Extensive evaluations on the DFC19 and SpaceNet4 datasets demonstrate that MVSR3D significantly outperforms the state-of-the-art multi-view multi-task learning method, improving the mIoU3 metric at a 2.5-meter threshold by 37.09%–45.11%.
20 March 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
As increasingly more deep learning models are designed and implemented, the performance of single-view image semantic segmentation is approaching its upper limit. With the increasing availability of multi-view satellite images, using multi-view information is gaining attention as it can address occlusion problems in single-view images and achieve cross-validation to reduce inappropriate segmentation. However, current multi-view semantic segmentation methods often rely on multi-view voting or require complex preprocessing steps, which may not fully leverage the advantages of multi-view images. We analyzed the complementarity and constraints of multi-view information and introduced the Pyramid Spatial Deformable Aggregation (PSDA) module, a plug-and-play module designed to enhance multi-view feature fusion. PSDA is the core component of our Early Multi-view Segmentation (EMVSeg) framework, which facilitates early-stage information fusion by directly extracting features from multi-view images, avoiding the complex and time-consuming production of true orthoimages. In this paper, we first show how we created the Multi-view Segmentation Dataset (MVSeg Dataset) using orthoimages generated from different-view images. Then, the results are shown to prove that our method outperformed the corresponding single-view segmentation method, namely by increasing the Intersection over Union (IoU) metric by approximately 1.23%-3.68% on both datasets. Due to the fusion of multi-view images at an early stage, the computational complexity is 0.29-0.74 times that of the state-of-the-art method, and the IoU metric improved by approximately 2.20%-7.52% on both datasets. Our MVSeg dataset is available at https://github.com/NanCheng2001/MVSeg-Dataset.
14 February 2026 in International Journal of Applied Earth Observation and Geoinformation
23 April 2026 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Reconstructing 3D scenes from a single satellite image remains challenging because reliable metric depth is unavailable and many regions are occluded, which limits the geometric observability and appearance evidence required for faithful reconstruction. To address these issues, we propose SSRecon, a diffusion-driven framework for single-image satellite 3D reconstruction via iterative RGB-D completion. SSRecon fits the RPC model to a pinhole approximation, estimates depth with a language-conditioned diffusion model, and performs RGB and depth inpainting using Stable Diffusion and a step-segmented gradient-guidance scheme, respectively. The input image together with the estimated depth forms the first RGB-D frame to initialize a 3D Gaussian Splatting representation, which is then refined by repeatedly rendering novel views, completing missing regions, and updating the scene representation. Experiments on DFC2019 and SpaceNet4 show that SSRecon improves perceptual realism and cross-view consistency across diverse target poses without auxiliary semantics or height inputs. In the pure single-image setting, SSRecon increases CLIP-Score by 0.65 on DFC2019 and 0.76 on SpaceNet4 over the strongest baseline, with consistent gains in CLIP-IQA quality and sharpness.