Jieyi Zhang

Jieyi Zhang

Auto-generated profile, based on publicly available information

Jieyi Zhang

School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 February 2024 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
TTTutian Tang
Tutian Tang
JLJiyu Liu
Jiyu Liu
JZJieyi Zhang
Jieyi Zhang
HFHaoyuan Fu
Haoyuan Fu
WXWenqiang Xu
Wenqiang Xu
CLCewu Lu
Cewu Lu
Transparent objects are widely used in our daily lives, making it important to teach robots to interact with them. However, it's not easy because the reflective and refractive effects can make depth cameras fail to give accurate geometry measurements. To solve this problem, this paper introduces RFTrans, an RGB-D-based method for surface normal estimation and manipulation of transparent objects. By leveraging refractive flow as an intermediate representation, the proposed method circumvents the drawbacks of directly predicting the geometry ( e . g . surface normal) from images and helps bridge the sim-to-real gap. It integrates the RFNet, which predicts refractive flow, object mask, and boundaries, followed by the F2Net, which estimates surface normal from the refractive flow. To make manipulation possible, a global optimization module will take in the predictions, refine the raw depth, and construct the point cloud with normal. An off-the-shelf analytical grasp planning algorithm is followed to generate the grasp poses. We build a synthetic dataset with physically plausible ray-tracing rendering techniques to train the networks. Results show that the proposed method trained on the synthetic dataset can consistently outperform the baseline method in both synthetic and real-world benchmarks by a large margin. Finally, a real-world robot grasping task witnesses an 83% success rate, proving that refractive flow can help enable direct sim-to-real transfer.

16 December 2024 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
JZJieyi Zhang
Jieyi Zhang
WXWenqiang Xu
Wenqiang Xu
ZYZhenjun Yu
Zhenjun Yu
PXPengfei Xie
Pengfei Xie
TTTutian Tang
Tutian Tang
CLCewu Lu
Cewu Lu
This study introduces a novel language-guided diffusion-based learning framework, DexTOG, aimed at advancing the field of task-oriented grasping (TOG) with dexterous hands. Unlike existing methods that mainly focus on 2-finger grippers, this research addresses the complexities of dexterous manipulation, where the system must identify non-unique optimal grasp poses under specific task constraints, cater to multiple valid grasps, and search in a high degree-of-freedom configuration space in grasp planning. The proposed DexTOG includes a diffusion-based grasp pose generation model, DexDiffu, and a data engine to support the DexDiffu. By leveraging DexTOG, we also proposed a new dataset, DexTOG-80K, which was developed using a shadow robot hand to perform various tasks on 80 objects from 5 categories, showcasing the dexterity and multitasking capabilities of the robotic hand. This research not only presents a significant leap in dexterous TOG but also provides a comprehensive dataset and simulation validation, setting a new benchmark in robotic manipulation research.

16 August 2024 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
WXWenqiang Xu
Wenqiang Xu
JZJieyi Zhang
Jieyi Zhang
TTTutian Tang
Tutian Tang
ZYZhenjun Yu
Zhenjun Yu
YLYutong Li
Yutong Li
CLCewu Lu
Cewu Lu
Grasp planning is an important task for robotic manipulation. Though it is a richly studied area, a standalone, fast, and differentiable grasp planner that can work with robot grippers of different DOFs has not been reported. In this work, we present DiPGrasp, a grasp planner that satisfies all these goals. DiPGrasp takes a force-closure geometric surface matching grasp quality metric. It adopts a gradient-based optimization scheme on the metric, which also considers parallel sampling and collision handling. This not only drastically accelerates the grasp search process over the object surface but also makes it differentiable. We apply DiPGrasp to three applications, namely grasp dataset construction, mask-conditioned planning, and pose refinement. For dataset generation, as a standalone planner, DiPGrasp has clear advantages over speed and quality compared with several classic planners. For mask-conditioned planning, it can turn a 3D perception model into a 3D grasp detection model instantly. As a pose refiner, it can optimize the coarse grasp prediction from the neural network, as well as the neural network parameters. Finally, we conduct real-world experiments with the Barrett hand and Schunk SVH 5-finger hand.