Ke Wang

Dr. Ke Wang

Hong Kong University of Science and Technology, Hong Kong, China
+1
orcid

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 January 2024 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
CLChuhao Liu
Chuhao Liu
KWKe Wang
Ke Wang
JSJieqi Shi
Jieqi Shi
ZQZhijian Qiao
Zhijian Qiao
SSShaojie Shen
Shaojie Shen
Semantic mapping based on the supervised object detectors is sensitive to image distribution. In real-world environments, the object detection and segmentation performance can lead to a major drop, preventing the use of semantic mapping in a wider domain. On the other hand, the development of vision-language foundation models demonstrates a strong zero-shot transferability across data distribution. It provides an opportunity to construct generalizable instance-aware semantic maps. Hence, this work explores how to boost instance-aware semantic mapping from object detection generated from foundation models. We propose a probabilistic label fusion method to predict close-set semantic classes from open-set label measurements. An instance refinement module merges the over-segmented instances caused by inconsistent segmentation. We integrate all the modules into a unified semantic mapping system. Reading a sequence of RGB-D input, our work incrementally reconstructs an instance-aware semantic map. We evaluate the zero-shot performance of our method in ScanNet and SceneNN datasets. Our method achieves 40.3 mean average precision (mAP) on the ScanNet semantic instance segmentation task. It outperforms the traditional semantic mapping method significantly.

2 April 2022 in Sensors
Verified
Journal Article
Open access
PDF file
KWKe Wang
Ke Wang
CLChuhao Liu
Chuhao Liu
SSShaojie Shen
Shaojie Shen
Traditional calibration methods rely on the accurate localization of the chessboard points in images and their maximum likelihood estimation (MLE)-based optimization models implicitly require all detected points to have an identical uncertainty. The uncertainties of the detected control points are mainly determined by camera pose, the slant of the chessboard and the inconsistent imaging capabilities of the camera. The negative influence of the uncertainties that are induced by the two former factors can be eliminated by adequate data sampling. However, the last factor leads to the detected control points from some sensor areas having larger uncertainties than those from other sensor areas. This causes the final calibrated parameters to overfit the control points that are located at the poorer sensor areas. In this paper, we present a method for measuring the uncertainties of the detected control points and incorporating these measured uncertainties into the optimization model of the geometric calibration. The new model suppresses the influence from the control points with large uncertainties while amplifying the contributions from points with small uncertainties for the final convergence. We demonstrate the usability of the proposed method by first using eight cameras to collect a calibration dataset and then comparing our method to other recent works and the calibration module in OpenCV using that dataset.

17 October 2022 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
KWKe Wang
Ke Wang
SSShaojie Shen
Shaojie Shen
With the development of 3D sensors, estimating the point movements between two consecutive point clouds is becoming increasingly attractive. Using correlation-based deep neural networks, existing solutions achieve promising performance. However, these methods couple the non-occluded points and occluded points into the same cost volume and attempt to regress the scene flows of the occluded points from those invalid matching costs directly, which severely suppress the performance of the flow predictor. As an alternative to previous scene flow estimation methods, our method adopts a subnet to predict the occlusion mask and explicitly masks those occluded points, which ensures our flow predictor to focus on estimating the motion flows of non-occluded points from those valid matching costs. Moreover, we further improve the flow estimation performance by employing a local-adaptive cost volume, which can deal with the local structure dissimilarity induced by the sparse sampling of depth sensors (LiDAR). For occluded points, we design an uncertainties-truncated propagation network to propagate the flows of non-occluded to those occluded points. We demonstrate the effectiveness of the proposed method by quantitative and qualitative comparisons with recent baseline works on the FlyingThings3D dataset and the KITTI 2015 dataset, and our results surpass all competing methods by a large margin.

5 June 2025 in IEEE Transactions on Robotics
Verified
Journal Article
Closed access
CLChuhao Liu
Chuhao Liu
ZQZhijian Qiao
Zhijian Qiao
JSJieqi Shi
Jieqi Shi
KWKe Wang
Ke Wang
PLPeize Liu
Peize Liu
SSShaojie Shen
Shaojie Shen
This paper addresses the challenges of registering two rigid semantic scene graphs, an essential capability when an autonomous agent needs to register its map against a remote agent, or against a prior map. The hand-crafted descriptors in classical semantic-aided registration, or the ground-truth annotation reliance in learning-based scene graph registration, impede their application in practical real-world environments. To address the challenges, we design a scene graph network to encode multiple modalities of semantic nodes: open-set semantic feature, local topology with spatial awareness, and shape feature. These modalities are fused to create compact semantic node features. The matching layers then search for correspondences in a coarse-to-fine manner. In the back-end, we employ a robust pose estimator to decide transformation according to the correspondences. We manage to maintain a sparse and hierarchical scene representation. Our approach demands fewer GPU resources and fewer communication bandwidth in multi-agent tasks. Moreover, we design a new data generation approach using vision foundation models and a semantic mapping module to reconstruct semantic scene graphs. It differs significantly from previous works, which rely on ground-truth semantic annotations to generate data. We validate our method in a two-agent SLAM benchmark. It significantly outperforms the hand-crafted baseline in terms of registration success rate. Compared to visual loop closure networks, our method achieves a slightly higher registration recall while requiring only 52 KB of communication bandwidth for each query frame. Code available at: http://github.com/HKUST-Aerial-Robotics/SG-Reg

17 December 2021 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
KWKe Wang
Ke Wang
SSShaojie Shen
Shaojie Shen
While unsupervised approaches have been proposed to estimate reflectance and shading layers for images, the decomposition process is a challenging, under-determined inverse problem. Previous unsupervised approaches for intrinsic decomposition of images are strictly reliant on special image sequences, for example, image sequences for a fixed scene with changing illumination or multi-view images containing rich illumination variation. As an alternative to these unsupervised intrinsic decomposition methods, we propose a semi-supervised method. Our method adopts a simplified scene representation to greatly reduce the complexity of spatially varying lighting, which allows us to partially restore the lighting from a night image pair. Moreover, we design several novel unsupervised guiding losses, and the training data used in our method are easily collected by setting different exposure times for a standard stereo setup. We further demonstrate the effectiveness of the proposed method by quantitatively and qualitatively comparing our method with recent works on the modified MPI dataset and a collected highly dynamic dataset named NightCampus. The NightCampus dataset has been released at https://github.com/kewangtt/srlefnip .

3 March 2021 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
KWKe Wang
Ke Wang
CLChuhao Liu
Chuhao Liu
KWKaixuan Wang
Kaixuan Wang
SSShaojie Shen
Shaojie Shen
Many methods have been proposed to process stereo matching for rolling shutter image pairs, they treat all pixels from an image pair in an identical way and require additional estimation approaches to estimate motion states of the camera. However, pixels from a rolling shutter image pair naturally have diverse baseline lengths, and motion estimation methods are unstable for one instantaneous image pair input. In this letter, we present a rolling shutter stereo depth estimation pipeline, which can robustly estimate motion states and depth maps by alternating the estimation of the depth maps and refining the motion states from coarse image levels to fine image levels. What is more, we design a novel cost volume building method for rolling shutter image pairs, which adapts depth candidates to the change of baseline lengths for all pixels. We further demonstrate the usability of the proposed method by constructing a new platform, building an outdoor evaluation dataset, and comparing it with baseline methods.