Hongtao Wu

Hongtao Wu

Auto-generated profile, based on publicly available information

Hongtao Wu

Laboratory for Computational Sensing and Robotics (LCSR), Johns Hopkins University, Baltimore, MD, USA

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 August 2020 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
AHAndrew Hundt
Andrew Hundt
BKBenjamin Killeen
Benjamin Killeen
NGNicholas Greene
Nicholas Greene
HWHongtao Wu
Hongtao Wu
HKHeeyeon Kwon
Heeyeon Kwon
GHGregory D. Hager
Gregory D. Hager
Current Reinforcement Learning (RL) algorithms struggle with long-horizon tasks where time can be wasted exploring dead ends and task progress may be easily reversed. We develop the SPOT framework, which explores within action safety zones, learns about unsafe regions without exploring them, and prioritizes experiences that reverse earlier progress to learn with remarkable efficiency. The SPOT framework successfully completes simulated trials of a variety of tasks, improving a baseline trial success rate from 13% to 100% when stacking 4 cubes, from 13% to 99% when creating rows of 4 cubes, and from 84% to 95% when clearing toys arranged in adversarial patterns. Efficiency with respect to actions per trial typically improves by 30% or more, while training takes just 1-20k actions, depending on the task. Furthermore, we demonstrate direct sim to real transfer. We are able to create real stacks in 100% of trials with 61% efficiency and real rows in 100% of trials with 59% efficiency by directly loading the simulation-trained model on the real robot with no additional real-world fine-tuning. To our knowledge, this is the first instance of reinforcement learning with successful sim to real transfer applied to long term multi-step tasks such as block-stacking and row-making with consideration of progress reversal. Code is available at https://github.com/jhu-lcsr/good_robot.

27 July 2022 in IEEE Transactions on Robotics
Verified
Journal Article
Open access
SRSipu Ruan
Sipu Ruan
KPKaren L. Poblete
Karen L. Poblete
HWHongtao Wu
Hongtao Wu
QMQianli Ma
Qianli Ma
GCGregory S. Chirikjian
Gregory S. Chirikjian
Path planning has long been one of the major research areas in robotics, with probabilistic roadmap (PRM) and rapidly-exploring random trees (RRT) being two of the most effective classes of planners. Though generally very efficient, these sampling-based planners can become computationally expensive in the important case of “narrow passages.” This article develops a path planning paradigm specifically formulated for narrow passage problems. The core is based on planning for rigid-body robots encapsulated by unions of ellipsoids. Each environmental feature is represented geometrically using a strictly convex body with a $\mathcal {C}^{1}$ boundary (e.g., superquadric). The main benefit of doing this is that configuration-space obstacles can be parameterized explicitly in closed form, thereby allowing prior knowledge to be used to avoid sampling infeasible configurations. Then, by characterizing a tight volume bound for multiple ellipsoids, robot transitions involving rotations are guaranteed to be collision free without needing to perform traditional collision detection. Furthermore, by combining with a stochastic sampling strategy, the proposed planning framework can be extended to solving higher dimensional problems, in which the robot has a moving base and articulated appendages. Benchmark results show that the proposed framework often outperforms the sampling-based planners in terms of computational time and success rate in finding a path through narrow corridors for both single-body robots and those with higher dimensional configuration spaces. Physical experiments using the proposed framework are further demonstrated on a humanoid robot that walks in several cluttered environments with narrow passages.

6 January 2025 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
PLPeiyan Li
Peiyan Li
HWHongtao Wu
Hongtao Wu
YHYan Huang
Yan Huang
CCChilam Cheang
Chilam Cheang
LWLiang Wang
Liang Wang
TKTao Kong
Tao Kong
The robotics community has consistently aimed to achieve generalizable robot manipulation with flexible natural language instructions. One primary challenge is that obtaining robot trajectories fully annotated with both actions and texts is time-consuming and labor-intensive. However, partially-annotated data, such as human activity videos without action labels and robot trajectories without text labels, are much easier to collect. Can we leverage these data to enhance the generalization capabilities of robots? In this letter, we propose GR-MG, a novel method which supports conditioning on a text instruction and a goal image. During training, GR-MG samples goal images from trajectories and conditions on both the text and the goal image or solely on the image when text is not available. During inference, where only the text is provided, GR-MG generates the goal image via a diffusion-based image-editing model and conditions on both the text and the generated image. This approach enables GR-MG to leverage large amounts of partially-annotated data while still using languages to flexibly specify tasks. To generate accurate goal images, we propose a novel progress-guided goal image generation model which injects task progress information into the generation process. In simulation experiments, GR-MG improves the average number of tasks completed in a row of 5 from 3.35 to 4.04. In real-robot experiments, GR-MG is able to perform 58 different tasks and improves the success rate from 68.7% to 78.1% and 44.4% to 60.6% in simple and generalization settings, respectively. It also outperforms comparing baseline methods in few-shot learning of novel skills.

1 October 2021
Verified
Conference Paper
Open access
WLWeixiao Liu
Weixiao Liu
HWHongtao Wu
Hongtao Wu
GCGregory S. Chirikjian
Gregory S. Chirikjian
Probabilistic point cloud registration methods are becoming more popular because of their robustness. However, unlike point-to-plane variants of iterative closest point (ICP) which incorporate local surface geometric information such as surface normals, most probabilistic methods (e.g., coherent point drift (CPD)) ignore such information and build Gaussian mixture models (GMMs) with isotropic Gaussian covariances. This results in sphere-like GMM components which only penalize the point-to-point distance between the two point clouds. In this paper, we propose a novel method called CPD with Local Surface Geometry (LSG-CPD) for rigid point cloud registration. Our method adaptively adds different levels of point-to-plane penalization on top of the point-to-point penalization based on the flatness of the local surface. This results in GMM components with anisotropic covariances. We formulate point cloud registration as a maximum likelihood estimation (MLE) problem and solve it with the Expectation-Maximization (EM) algorithm. In the E step, we demonstrate that the computation can be recast into simple matrix manipulations and efficiently computed on a GPU. In the M step, we perform an unconstrained optimization on a matrix Lie group to efficiently update the rigid transformation of the registration. The proposed method outperforms state-of-the-art algorithms in terms of accuracy and robustness on various datasets captured with range scanners, RGBD cameras, and LiDARs. Also, it is significantly faster than modern implementations of CPD. The source code is available at https://github.com/ChirikjianLab/LSG-CPD.git.

24 November 2020 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
HWHongtao Wu
Hongtao Wu
GCGregory S. Chirikjian
Gregory S. Chirikjian
Open containers, i.e., containers without covers, are an important and ubiquitous class of objects in human life. In this letter, we propose a novel method for robots to "imagine" the open containability affordance of a previously unseen object via physical simulations. We implement our imagination method on a UR5 manipulator. The robot autonomously scans the object with an RGB-D camera. The scanned 3D model is used for open containability imagination which quantifies the open containability affordance by physically simulating dropping particles onto the object and counting how many particles are retained in it. This quantification is used for open-container vs. non-open-container binary classification (hereafter referred to as open container classification). If the object is classified as an open container, the robot further imagines pouring into the object, again using physical simulations, to obtain the pouring position and orientation for real robot autonomous pouring. We evaluate our method on open container classification and autonomous pouring of granular material on a dataset containing 130 previously unseen objects with 57 object categories. Although our proposed method uses only 11 objects for simulation calibration (training), its open container classification aligns well with human judgements. In addition, our method endows the robot with the capability to autonomously pour into the 55 containers in the dataset with a very high success rate. We also compare to a deep learning method. Results show that our method achieves the same performance as the deep learning method on open container classification and outperforms it on autonomous pouring. Moreover, our method is fully explainable.

1 August 2019
Verified
Conference Paper
Closed access
HWHongtao Wu
Hongtao Wu
JMJiteng Mu
Jiteng Mu
TDTing Da
Ting Da
MXMengdi Xu
Mengdi Xu
RTRussell H. Taylor
Russell H. Taylor
GCGregory S. Chirikjian
Gregory S. Chirikjian
Multi-mosquito object detection and 2D pose estimation are essential steps towards fully automated extracting PfSPZ-infected mosquito salivary glands for manufacture of PfSPZ Vaccine, which has been shown to protect against malaria in multiple clinical trials in the US, Europe, and Africa. This paper presents a deep learning approach to perform cluster condition classification and bounding box detection of multiple mosquitoes in an image. It also estimates the 2D pose of each non-clustered mosquito by body part detection. This approach is based on two popular convolutional neural network (CNN) architectures, Mask R-CNN and DeeperCut. In addition, we propose a cascaded image processing approach to achieve the multi-mosquito detection, cluster condition classification, and body parts detection in a multi-step manner. We compare the two approaches in terms of their functionality, robustness, accuracy, and speed. We hope our effective approaches would push forward the automation of PfSPZ Vaccine production to facilitate the prevention and elimination of this disease worldwide.