Jingyi Hu

Jingyi Hu

Auto-generated profile, based on publicly available information

Jingyi Hu

Institute of Automation, Chinese Academy of Sciences, Beijing, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
21 July 2020 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
SCShaowei Cui
Shaowei Cui
RWRui Wang
Rui Wang
JWJunhang Wei
Junhang Wei
JHJingyi Hu
Jingyi Hu
SWShuo Wang
Shuo Wang
Predicting whether a particular grasp will succeed is critical to performing stable grasping and manipulating tasks. Robots need to combine vision and touch as humans do to accomplish this prediction. The primary problem to be solved in this process is how to learn effective visual-tactile fusion features. In this paper, we propose a novel Visual-Tactile Fusion learning method based on Self-Attention mechanism (VTFSA) to address this problem. We compare the proposed method with the traditional methods on two public multimodal grasping datasets, and the experimental results show that the VTFSA model achieves outperforming traditional methods by a margin of 5+% and 7+%. Furthermore, visualization analysis indicate that the VTFSA model can further capture some position-related visual-tactile fusion features that are beneficial to this task and is more robust compared to traditional methods.

19 September 2023 in IEEE Transactions on Industrial Electronics
Verified
Journal Article
Closed access
CZChaofan Zhang
Chaofan Zhang
SCShaowei Cui
Shaowei Cui
SWShuo Wang
Shuo Wang
JHJingyi Hu
Jingyi Hu
YCYinghao Cai
Yinghao Cai
YWYu Wang
Yu Wang
Visuotactile sensors have significant advantages in high-resolution contact shape sensing, which is crucial for robots to locate and manipulate objects. However, high precision and robust 3-D geometry sensing is still a challenging problem. In this article, we propose an improved GelStereo sensor, named GelStereo 2.0, with a more stable and compact structure. A refractive stereo ray tracing modeling method is proposed to eliminate 3-D geometry reconstruction errors caused by multimedium light refraction. In addition, we introduce a novel calibration method named multiple contact depths to obtain refractive parameters, which only needs a few pressings using calibration boards. Extensive experimental results show that the measurement error of contact surface depth is less than 0.15 mm. Furthermore, two well-calibrated sensors mounted on the gripper are successfully applied to a fingertip bolt screwing task, which demonstrates the effectiveness of the GelStereo 2.0 sensor and the proposed tactile 3-D geometry sensing method.

16 December 2021 in IEEE Transactions on Instrumentation and Measurement
Verified
Journal Article
Closed access
SCShaowei Cui
Shaowei Cui
RWRui Wang
Rui Wang
JHJingyi Hu
Jingyi Hu
CZChaofan Zhang
Chaofan Zhang
LCLipeng Chen
Lipeng Chen
SWShuo Wang
Shuo Wang
Vision-based tactile sensors have recently shown promising contact information sensing capabilities in various fields, especially for dexterous robotic manipulation. However, dense contact geometry measurement is still a challenging problem. In this article, we update the design of our previous GelStereo tactile sensor and present a self-supervised contact geometry learning pipeline. Specifically, a self-supervised stereo-based depth estimation neural network (GS-DepthNet) is proposed to achieve real-time disparity estimation, and two specifically designed loss functions are proposed to accelerate the convergence of the network during the training process and improve the inference accuracy. Furthermore, extensive qualitative and quantitative experiments of perceived contact shape were performed on our GelStereo sensor. The experimental results verify the accuracy and robustness of the proposed contact geometry sensing pipeline. This updated GelStereo tactile sensor with dense contact geometric sensing capability has predictable application potential in the field of industrial and service robots.

2 February 2023 in IEEE Transactions on Industrial Informatics
Verified
Journal Article
Closed access
JHJingyi Hu
Jingyi Hu
SCShaowei Cui
Shaowei Cui
SWShuo Wang
Shuo Wang
CZChaofan Zhang
Chaofan Zhang
RWRui Wang
Rui Wang
YLYuhao Li
Yuhao Li
Recently, visuotactile sensors have shown promising potential in robotics due to their high-resolution sensing ability. Unfortunately, the majority of available visuotactile sensors are limited to flat shapes, which severely limits their application possibilities. In this article, we propose a novel curved visuotactile sensor, the GelStereo Palm, which senses the 3-D contact geometry on a curved surface using a binocular vision system. Meanwhile, to solve the light refraction problem in the binocular stereo vision system under a curved medium, a refractive stereo ray tracing model for GelStereo Palm is presented. Moreover, a 3-D tactile point cloud sensing pipeline is introduced to reconstruct the 3-D contact geometry in real-time. Finally, extensive experiments are conducted to verify the accuracy and robustness of the 3-D contact geometry sensing of our GelStereo Palm sensor.

9 November 2021 in IEEE Transactions on Industrial Informatics
Verified
Journal Article
Closed access
JWJunhang Wei
Junhang Wei
SCShaowei Cui
Shaowei Cui
JHJingyi Hu
Jingyi Hu
PHPeng Hao
Peng Hao
SWShuo Wang
Shuo Wang
ZLZheng Lou
Zheng Lou
Unknown surface material classification (SMC) can inform a robot about material properties, enabling it to interact with environments appropriately. Recent research has leveraged multimodal data using deep learning to improve the performance of SMC. In this article, we present a deep learning model, multimodal temporal convolutional neural network (MTCNN), which integrates energy spectrum, dilated convolutions, and sequence poolings into a unified network architecture. The proposed model can learn material representations from auditory and multitactile (i.e., acceleration, normal force, and friction force) data generated by dragging a tool along surfaces, and distinguish unknown object surface materials into categories. For surface material data collection, a tool is also designed to detect different object surfaces. The performance of MTCNN is evaluated on a public dataset and the highest classification accuracy is 87.55%. A robotic curling example is provided to illustrate how the presented model helps the robot in manipulation.

28 February 2023 in Sensors
Verified
Journal Article
Open access
PDF file
CZChaofan Zhang
Chaofan Zhang
SCShaowei Cui
Shaowei Cui
SWShuo Wang
Shuo Wang
JHJingyi Hu
Jingyi Hu
YHYipeng Huangfu
Yipeng Huangfu
BZBoyue Zhang
Boyue Zhang
GelStereo sensing technology is capable of performing three-dimensional (3D) contact shape measurement under various contact structures such as bionic curved surfaces, which has promising advantages in the field of visuotactile sensing. However, due to multi-medium ray refraction in the imaging system, robust and high-precision tactile 3D reconstruction remains a challenging problem for GelStereo-type sensors with different structures. In this paper, we first propose a universal Refractive Stereo Ray Tracing (RSRT) model for GelStereo-type sensing systems to realize 3D reconstruction of the contact surface. Moreover, a relative geometry-based optimization method is presented to calibrate multiple parameters of the proposed RSRT model, such as the refractive indices and structural dimensions. Furthermore, extensive quantitative calibration experiments are performed on four different GelStereo sensing platforms; the experimental results show that the proposed calibration pipeline can achieve less than 0.35 mm in Euclidean distance error, based on which we believe that the proposed refractive calibration method can be further applied in more complex GelStereo-type and other similar visuotactile sensing systems. Such high-precision visuotactile sensors can facilitate the study of robotic dexterous manipulation.