Mingyuan Zhao

Dr. Mingyuan Zhao

orcid

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 November 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
XHXinggang Hu
Xinggang Hu
YWYanmin Wu
Yanmin Wu
MZMingyuan Zhao
Mingyuan Zhao
LYLinghao Yang
Linghao Yang
XZXiangkui Zhang
Xiangkui Zhang
XJXiangyang Ji
Xiangyang Ji
Visual SLAM (Simultaneous Localization and Mapping) systems based on planar features have been widely applied in fields such as environmental structure perception and augmented reality (AR). However, current research still faces challenges in accurate localization and map construction in planar ambiguous scenes, primarily due to the insufficient accuracy of the planar features and data association methods employed. In this paper, we propose a visual SLAM system based on planar features designed for ambiguous planar scenes, including planar analysis and processing, data association, and multi-constraint factor graph optimization. Initially, we introduce a planar analysis and processing strategy that integrates semantic information to analyze the structure of planes and further refine the selection of planes, providing accurate planar information for subsequent association and optimization processes. Then, we integrate various planar data to propose a multimodal fusion data association strategy, achieving accurate and robust planar data association in ambiguous planar scenes. Finally, based on accurate and rich planar information along with related constraints, we design a set of multi-constraint factor graphs for camera pose optimization. Public datasets and real-world experiments demonstrate that, compared to state-of-the-art related research, our proposed system shows significant competitive advantages in terms of accuracy and robustness for both map construction and camera localization. Regarding quantifiable localization accuracy, our system achieves an average improvement in Absolute Trajectory Error (ATE) of approximately 57% in planar ambiguous scenes and about 25% in non-planar ambiguous scenes. Additionally, the system exhibits great application potential in fields such as augmented reality.

19 May 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
XHXinggang Hu
Xinggang Hu
YWYanmin Wu
Yanmin Wu
MZMingyuan Zhao
Mingyuan Zhao
ZCZhenzhong Cao
Zhenzhong Cao
XZXiangkui Zhang
Xiangkui Zhang
XJXiangyang Ji
Xiangyang Ji
Addressing the impact of dynamic factors on localization accuracy and constructing a long-term consistent map containing only static elements are two crucial tasks in visual simultaneous localization and mapping (SLAM) for dynamic scenes. The introduction of dynamic elements can compromise the geometric constraints essential for visual SLAM, leading to a decrease in localization accuracy. Existing related research faces challenges in simultaneously ensuring localization accuracy in both low-dynamic and high-dynamic scenarios, while also maintaining the system’s real-time performance. To address this issue, we propose a two-stage, coarse-to-fine static-probability-based localization scheme. The construction of object-level maps offers strong support for tasks involving higher-level intelligent agent manipulation as well as augmented reality (AR). However, current research is inadequate for dynamic scenes where the objects to be modeled are frequently and irregularly obscured by dynamic objects, and where there are significant challenges such as severe image and point cloud noise, semantic noise, and lack of observational perspectives. To overcome these challenges, we first propose an object parameter estimation algorithm that combines clustering, weighted Principal Component Analysis (PCA) based on an energy function, and a minimum bounding rectangle. Then, we design a multi-modal object data association strategy based on appearance, semantic, and spatial features. The proposed object parameter estimation algorithm and data association strategy demonstrate improved accuracy and robustness in dynamic scenes with the aforementioned challenges. Finally, based on the entire system, we further develop a dynamic object tracking algorithm and construct an AR system to demonstrate the system’s application prospects. A series of public datasets and real-world scene results have been used to evaluate the effectiveness of the proposed system.

4 June 2025 in Remote Sensing
Verified
Journal Article
Open access
PDF file
MZMingyuan Zhao
Mingyuan Zhao
Long XuLong Xu
Long Xu
This paper addresses the challenge of robust pose estimation for spacecraft under rapid inter-frame motion, proposing a two-stage point cloud registration framework. The first stage computes coarse pose estimation by leveraging Fast Point Feature Histogram (FPFH) descriptors with random sample and consensus (RANSAC) for correspondence matching, effectively handling significant positional displacements. The second stage refines the solution through geometry-aware fine registration using raw point cloud data, enhancing precision through a multi-scale iterative ICP-like framework. To validate the approach, we simulate time-of-flight (ToF) sensor measurements by rendering NASA’s public 3D spacecraft models and obtain 3D point clouds by back-projecting the depth measurements to 3D space. Comprehensive experiments demonstrate superior performance over several state-of-the-art methods in both accuracy and robustness under rapid inter-frame motion scenarios. The dual-stage architecture proves effective in maintaining tracking continuity while mitigating error accumulation from fast relative motion, showing promise for autonomous spacecraft proximity operations.

21 August 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
MZMingyuan Zhao
Mingyuan Zhao
XYXuexin Yu
Xuexin Yu
XHXinggang Hu
Xinggang Hu
FWFei Wang
Fei Wang
Long XuLong Xu
Long Xu
XJXiangyang Ji
Xiangyang Ji
The registration of time-varying 3D shapes with high degrees of freedom remains a challenging task. Most existing techniques attempt to address this issue by solving an optimization problem defined on deformation graph with as-rigid-as-possible smoothness prior, which usually struggle to capture large scale displacements. Motivated by the insight that a set of points tends to collectively undergo significant rigid motion accompanied by slight nonrigid deformation, we propose a two-step approach to address nonrigid registration in a coarse-to- fine manner. In the first step, coarse correlations between source and target points are constructed by estimating a set of rigid transformations for local patches which are regional clusters of points. To leverage more contextual information, a bidirectional registration module is introduced that estimates both the forward and backward patch-wise rigid transformation fields (PRTFs). Subsequently, in the second step, the source point set is warped by blending both forward and backward PRTFs and fed into a deformation optimization module. Here, unidirectional point-based correspondences are sought to refine the global nonrigid transformation fields (GNTFs) while adhering to local rigidity constraints. To illustrate the efficacy of our method, we conduct tests on challenging scenarios involving human datasets, including large displacements resulting from fast inter-frame motions or pose changes. Both qualitative and quantitative results demonstrate that our approach outperforms several state-of-the-art methods in terms of robustness and registration accuracy.

9 June 2025 in IEEE Access
Verified
Journal Article
Open access
MZMingyuan Zhao
Mingyuan Zhao
LXLong Xu
Long Xu
Accurately estimating the 6D pose of objects or scenes is a fundamental challenge in point cloud registration (PCR), especially when dealing with partially overlapping scenarios. Typically, features are first extracted—either through hand-crafted descriptors or deep learning-based methods—to establish a putative correspondence set. However, despite advances in deep learning for feature extraction and matching, the presence of a low inlier ratio often complicates the estimation process. To address these challenges, this paper proposes a novel inlier estimation approach in PCR that leverages edge-triangle combined compatibility for progressively filtering out outliers. First, geometric compatibility is exploited by randomly selecting two points to form an edge. If the compatibility test is passed, both endpoints receive a vote, and the top-ranked correspondences are retained for the next stage, enabling rapid removal of a large portion of outliers. Next, a stricter constraint is introduced based on triangles, requiring all three edges to satisfy compatibility conditions, which further refines the candidate set. The refined correspondences are then processed through a lightweight RANSAC module for hypothesis generation and selection. Finally, a geometry-based pose refinement module is presented, which utilizes the original geometric information and formulates registration as a nonlinear least squares problem. To enhance robustness against outliers,Welsch function is selected as a robust kernel, efficiently solving the optimization using the Majorization-Minimization algorithm. Extensive experiments on benchmark datasets, including 3DMatch, 3DLoMatch, and KITTI, validate the effectiveness of the proposed approach. The results demonstrate state-of-the-art performance, particularly in scenarios with extremely low inlier ratios.

27 March 2024 in Preprints.org
Verified
Preprint
Open access
PDF file
MZMingyuan Zhao
Mingyuan Zhao
XYXuexin Yu
Xuexin Yu
Long XuLong Xu
Long Xu
This paper introduces PatchFusion, an innovative approach for nonrigid tracking and reconstruction of deformable objects using a single RGB-D sensor. Existing methods face challenges in accurately capturing the rapid deformations of soft and flexible objects, thereby limiting their utility in diverse scenarios. Our approach overcomes this challenge by employing a dynamic patch-based framework that adapts to rapid inter-frame motions. Firstly, patch-wise rigid transformation fields for non-overlapping patches are solved via Iterative Closest Point (ICP) by incorporating geometric features as additional similarity constraints, thereby enhancing robustness and accuracy. Secondly, deformation optimization based on a nonrigid solver is applied to refine the coarse transformation fields. In order to enable simultaneous tracking and reconstruction of deformable objects, the patch-based rigid solver is designed to run in parallel with the nonrigid solver, serving as a plug-and-play module requiring minimal modifications for integration while enabling real-time performance. Following a comprehensive evaluation, PatchFusion showcases superior performance in effectively dealing with rapid inter-frame deformations when compared to existing techniques, rendering it a promising solution with broad applicability across domains such as robotics, computer vision, and human-computer interaction.