杰仁 邓

Dr. 杰仁 邓

orcid

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
31 July 2023 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
XZXingxuan Zhang
Xingxuan Zhang
HZHaojian Zhang
Haojian Zhang
JHJianhua Hu
Jianhua Hu
杰邓杰仁 邓
杰仁 邓
YWYunkuan Wang
Yunkuan Wang
Accurate recognition of human pose and prediction of human motion intention are essential for exoskeleton robots to provide effective assistance. In recent years, inertial measurement units (IMUs) have been widely used to estimate human pose due to their stability and economy. However, physical sensors are inherently subject to time delay relative to human movement, which makes it challenging to meet the demand for real-time control of power-assisted exoskeletons. To address this issue, we propose a data-driven approach called motion forecasting network (MoFCNet) that leverages historical IMU readings to forecast the 3D pose of human in future frames. Our approach consists of two stages, state variables forecasting and pose regression. In stage I, we forecast the IMU data, joint positions and velocities. In stage II, we regress the future human poses from the forecasting results of stage I. Besides, we design a basic stack that can be applied to both stages. The mean joint rotation error on DIP-IMU and TotalCapture datasets are $\bm {10.34}$ and $\bm {16.11}$ degrees, respectively. When focusing solely on the first frame of the prediction results, these errors decrease to 9.13 and 14.50 degrees. Experimental results indicate that our method not only outperforms the forecasting baseline methods but also achieves comparable performance to the sparse IMU-based human pose estimation algorithm.

2 August 2023 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
杰邓杰仁 邓
杰仁 邓
HZHaojian Zhang
Haojian Zhang
JHJianhua Hu
Jianhua Hu
XZXingxuan Zhang
Xingxuan Zhang
YWYunkuan Wang
Yunkuan Wang
We introduce a new task, called Class Incremental Robotic Pick-and-Place (CIRPAP), which calls for the capacity to learn to pick and place new categories of objects while retaining the skill of dealing with the previously learned ones. CIRPAP faces three challenges: catastrophic forgetting, few-shot learning, and robust picking in cluttered environments. To address the challenges of catastrophic forgetting and few-shot learning, we propose a novel CIRPAP framework that is built on Incremental Few-Shot Object Detection (iFSD). Specifically, with fixed pre-trained Transformer-like object detection models, we only fine-tune the additional adapter modules, which is called adapter-tuning. To address the challenge of robust picking in cluttered environments, we also utilize multiview fusion to integrate object detection and grasp prediction results. As for iFSD evaluation, experiments show that our adapter-tuning-based approach outperforms state-of-the-art methods on COCO and our dataset. As for full CIRPAP system evaluation, experimental results on a real robotic platform demonstrate the effectiveness of our proposed framework.

10 September 2024 in Agriculture
Verified
Journal Article
Open access
PDF file
HWHaotian Wang
Haotian Wang
HZHaojian Zhang
Haojian Zhang
YZYukai Zhang
Yukai Zhang
杰邓杰仁 邓
杰仁 邓
CLChengbao Liu
Chengbao Liu
JTJie Tan
Jie Tan
This study addresses the challenge of efficiently peeling pineapples, which have a distinct elliptical form, thick skin, and small eyes that are difficult to detect with conventional automated methods. This results in significant flesh waste. To improve the process, we developed an integrated system combining an enhanced BlendMask method, termed SAAF-BlendMask, and a Pose Correction Planning (PCP) method. SAAF-BlendMask improves the detection of small pineapple eyes, while PCP ensures accurate posture adjustment for precise path planning. The system uses 3D vision and deep learning technologies, achieving an average precision (AP) of 73.04% and a small object precision (APs) of 62.54% in eye detection, with a path planning success rate reaching 99%. The fully automated electromechanical system was tested on 110 real pineapples, demonstrating a reduction in flesh waste by 11.7% compared to traditional methods. This study highlights the potential of advanced machine vision and robotics in enhancing the efficiency and precision of food processing.

24 June 2024 in IEEE Sensors Journal
Verified
Journal Article
Closed access
ZWZihao Wan
Zihao Wan
HZHaojian Zhang
Haojian Zhang
JHJianhua Hu
Jianhua Hu
杰邓杰仁 邓
杰仁 邓
YWYunkuan Wang
Yunkuan Wang
3D instance segmentation is a fundamental task in sensor data processing for robot manipulation. However, recent works in this field did not pay much attention to 3D instance segmentation under cluttered conditions. Specifically, current widely-used 3D instance segmentation datasets mainly focus on indoor scenes, and they often reconstruct the whole 3D scene by multiple depth sensors. Therefore, there are no occluded or cluttered areas in these datasets. Moreover, we explore prevailing 3D instance segmentation models and find that the traditional heuristic algorithms in these models severely harm their performance in cluttered scenes. To tackle these issues, we first build a monocular depth sensor system to collect a new dataset named Cluttered Objects (CO), which is tailored for robotic manipulation under cluttered conditions. Then, we propose Instance-Guided Net (IGN) to segment targets in these complex areas. In our IGN, the point-wise instance information is fully exploited to make all heuristic algorithms task-oriented. The keys to IGN are Dual-Instance-Guided Block (D-IGB), Instance-Guided Upsampler (IGUS), and Instance-Guided Downsampler (IGDS). In D-IGB, we build the instance-aware receptive field and dilated receptive field to gather intra-instance information and long-range context simultaneously. In IGUS, the instance-aware receptive field is applied to realize task-oriented interpolation. Moreover, we propose IGDS to retain more beneficial foreground features by eliminating background points. Extensive experiments on CO and other public datasets show the effectiveness of our IGN.

9 February 2023 in arXiv
Verified
Preprint
Open access
PDF file
杰邓杰仁 邓
杰仁 邓
JHJianhua Hu
Jianhua Hu
HZHaojian Zhang
Haojian Zhang
YWYunkuan Wang
Yunkuan Wang
Class incremental learning(CIL) has attracted much attention, but most existing related works focus on fine-tuning the entire representation model, which inevitably results in much catastrophic forgetting. In the contrast, with a semantic-rich pre-trained representation model, parameter-additional-tuning (PAT) only changes very few parameters to learn new visual concepts. Recent studies have proved that PAT-based CIL can naturally avoid fighting against forgetting by replaying or distilling like most of the existing methods. However, we find that PAT-based CIL still faces serious semantic drift, the high-level forgetting problem caused by classifier learning bias at different learning phases, which significantly reduces the performance of PAT-based CIL. To address this problem, we propose Incremental Prototype Tuning (IPT), a simple but effective method that tunes category prototypes for classification and learning example prototypes to compensate for semantic drift. Extensive experiments demonstrate that our method can effectively compensate for semantic drift. Combined with well-pre-trained Vit backbones and other PAT methods, IPT surpasses the state-of-the-art baselines on mainstream incremental learning benchmarks.

9 August 2024 in Preprints.org
Verified
Preprint
Open access
PDF file
HWHaotian Wang
Haotian Wang
HZHaojian Zhang
Haojian Zhang
YZYukai Zhang
Yukai Zhang
杰邓杰仁 邓
杰仁 邓
CLChengbao Liu
Chengbao Liu
JTJie Tan
Jie Tan
The study addresses the challenge of efficiently peeling pineapples, which have a distinct elliptical form, thick skin, and small eyes that are difficult to detect with conventional automated methods. This results in significant flesh waste. To improve the process, the researchers developed an integrated system combining an enhanced BlendMask method, termed SAAF-BlendMask, and a PCP method. SAAF-BlendMask improves the detection of small pineapple eyes, while PCP ensures accurate posture adjustment for precise path planning. The system uses 3D vision and deep learning technologies, achieving an average precision (AP) of 73.04% and a small object precision (APs) of 62.54% in eye detection, with a path planning success rate reaching 99%. The fully automated electromechanical system was tested on 110 real pineapples, demonstrating a reduction in flesh waste by 11.7% compared to traditional methods. This study highlights the potential of advanced machine vision and robotics in enhancing the efficiency and precision of food processing.