1 July 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
With advancements in robust stereo matching and optical flow estimation networks, models pre-trained on synthetic data demonstrate strong robustness to unseen domains. However, their robustness can be seriously degraded when fine-tuning them in real-world scenarios. This paper investigates fine-tuning stereo matching and optical flow estimation networks without compromising their robustness to unseen domains. Specifically, we divide the pixels into consistent and inconsistent regions by comparing Ground Truth (GT) with Pseudo Label (PL) and demonstrate that the imbalance learning of consistent and inconsistent regions in GT causes robustness degradation. Based on our analysis, we propose the DKT framework, which utilizes PL to balance the learning of different regions in GT. The core idea is to utilize an exponential moving average (EMA) teacher to measure what the student network has learned and dynamically adjust the learning regions. We further propose the DKT++ framework, which improves target-domain performances and network robustness by applying slow-fast update teachers to generate more accurate PL, introducing the unlabeled data and synthetic data. We integrate our frameworks with state-of-the-art networks and evaluate their effectiveness on several real-world datasets. Extensive experiments show that our method effectively preserves the robustness of stereo matching and optical flow networks during fine-tuning.
20 December 2025 in Pattern Recognition
1 July 2026 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Synthesizing novel views from sparse views has achieved impressive advances with radiance fields, yet prevailing methods suffer from high consumption or insufficient refinement capability. This paper introduces DNGaussian, a depth-regularized framework based on 3D Gaussian Splatting, offering real-time and high-quality few-shot novel view synthesis at low costs. Our motivation stems from the remarkable advancement of recent 3D Gaussian Splatting, despite it will encounter a geometry degradation when input views decrease. In the Gaussian radiance fields, we find this degradation in scene geometry primarily lined to the positioning of Gaussian primitives and can be mitigated by depth constraint. Consequently, we propose a Hard and Soft Depth Regularization to restore accurate scene geometry under coarse monocular depth supervision while maintaining a fine-grained color appearance. To further refine detailed geometry, we introduce Global-Local Depth Normalization, enhancing the focus on small local depth changes. Although DNGaussian shows impressive performance, its patch-wise regularization obscures the inconsistency in cross-patch errors. Additionally, primitives can still be irreversibly trapped in local minima under sparse views, even if depth regularization is applied. In this paper, we propose an extended version, DNGaussian++. First, a Geometry Instance Regularizer is developed to enable depth regularization for continuous consistency by exploiting reliable instance-level depth cues. Leveraging the depth gradient guidance, we then propose a Depth-Guided Geometry Reorganization to address the aforementioned local minima problem with high representation efficiency. Extensive experiments show that DNGaussian++ exhibits state-of-the-art performance in multiple datasets and scenarios with high efficiency, and the broad applicability and effectiveness are verified on various backbones and tasks.
27 February 2026 in IEEE Transactions on Multimedia
Embodied Question Answering (EQA) requires agents to explore the environment actively and gather reasonable visual information to answer the questions raised by users. Despite the significant progress attained, previous works lack a comprehensive understanding of the observed environment. Particularly, in the question answering task, little attention is paid to the temporal context in multiple observations and the semantic understanding of the environment. Meanwhile, the navigator struggles when combining global spatial context and individual observations to comprehend the geometric environment. In this paper, we propose Context-aware Embodied Question Answering (CaEQA) to address the issues from two aspects. Firstly, to integrate temporal context into image embeddings obtained from the local observations at the navigation endpoint, we propose a local temporal context-aware visual question answering module (LTC-VQA). Specifically, we utilize the large language model in the question answering module to enhance the temporal and semantic understanding of complex unseen environments by introducing general knowledge obtained in the pretraining stage. Secondly, to model the complex interdependence within the overall navigation procedure, we propose a global-geometric-guided navigation module (G3-Nav), facilitating the interpretation of global historical information. In G3-Nav, we use a transformer that incorporates RGB images, depth maps and actions from all past positions to provide geometric hints from inter-image and intra-image perspectives. Experimental results on the Matterport3D and House3D datasets demonstrate that our method outperforms the SoTA methods in terms of navigation, question answering and the overall EQA system.