Jiantao Gao

Dr. Jiantao Gao

Shanghai University, Shanghai, China
orcid

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
24 February 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
WXWeiping Xiao
Weiping Xiao
YPYan Peng
Yan Peng
CLChang Liu
Chang Liu
JGJiantao Gao
Jiantao Gao
YWYiqiang Wu
Yiqiang Wu
XLXiaomao Li
Xiaomao Li
Accurately detecting multi-class objects in a single pass is critical but challenging for real-world autonomous driving scenarios. Several single-class anchor-based methods have recently achieved the state-of-the-art performance in the car category, but when extending to multi-class detection tasks, their performance on small objects ( i.e ., pedestrians and cyclists) is limited. We find that the core problem that causes this phenomenon lies in the unbalanced sample quality and the classification objective. To address this problem, we proposed a single-model multi-class 3D object detector with balanced sample assignment and objective, named BSAODet. Specifically, the quality-balanced sample assignment (QBSA) is introduced to dynamically collect stable high-quality samples for each class according to the predicted sample performance and geometric constraints. In conjunction with the QBSA, the class-balanced classification objective (CBCO) performs instance-wise label normalization and weighting on positive samples, preventing the model from biasing toward objects with more samples. Extensive experiments on the popular KITTI dataset, the latest large-scale ONCE dataset, and the challenging Waymo Open Dataset show that our method steadily improves the performance of current state-of-the-art detectors by 2–7 mAP in pedestrians and cyclists while maintaining competitiveness in cars. Moreover, our best model achieves 66.31 mAP on three classes, outperforming all published LiDAR-only detectors on the KITTI benchmark.

6 January 2023 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
JGJiantao Gao
Jiantao Gao
XYXu Yan
Xu Yan
WZWeibing Zhao
Weibing Zhao
ZLZhen Lyu
Zhen Lyu
YLYinghong Liao
Yinghong Liao
CZChaoda Zheng
Chaoda Zheng
Single object tracking (SOT) is one of the most active research directions in the field of computer vision. Compared with the 2-D image-based SOT which has already been well-studied, SOT on 3-D point clouds is a relatively emerging research field. In this article, a novel approach, namely, the contextual-aware tracker (CAT), is investigated to achieve a superior 3-D SOT through spatially and temporally contextual learning from the LiDAR sequence. More precisely, in contrast to the previous 3-D SOT methods merely exploiting point clouds in the target bounding box as the template, CAT generates templates by adaptively including the surroundings outside the target box to use available ambient cues. This template generation strategy is more effective and rational than the previous area-fixed one, especially when the object has only a small number of points. Moreover, it is deduced that LiDAR point clouds in 3-D scenes are often incomplete and significantly vary from frame to another, which makes the learning process more difficult. To this end, a novel cross-frame aggregation (CFA) module is proposed to enhance the feature representation of the template by aggregating the features from a historical reference frame. Leveraging such schemes enables CAT to achieve a robust performance, even in the case of extremely sparse point clouds. The experiments confirm that the proposed CAT outperforms the state-of-the-art methods on both the KITTI and NuScenes benchmarks, achieving 3.9% and 5.6% improvements in terms of precision.

Verified
Journal Article
Open access
PDF file
JGJiantao Gao
Jiantao Gao
JZJingting Zhang
Jingting Zhang
Chang LiuChang Liu
Chang Liu
XLXiaomao Li
Xiaomao Li
YPYan Peng
Yan Peng
Water segmentation is essential for the autonomous driving system of unmanned surface vehicles (USVs), which provides reliable navigation for making safety decisions. However, existing methods have only used monocular images as input, which often suffer from the changes in illumination and weather. Compared with monocular images, LiDAR point clouds can be collected independently of ambient light and provide sufficient 3D information but lack the color and texture that images own. Thus, in this paper, we propose a novel camera-LiDAR cross-modality fusion water segmentation method, which combines the data characteristics of the 2D image and 3D LiDAR point cloud in water segmentation for the first time. Specifically, the 3D point clouds are first supplemented with 2D color and texture information from the images and then distinguished into water surface points and non-water points by the early 3D cross-modality segmentation module. Subsequently, the 3D segmentation results and features are fed into the late 2D cross-modality segmentation module to perform 2D water segmentation. Finally, the 2D and 3D water segmentation results are fused for the refinement by an uncertainty-aware cross-modality fusion module. We further collect, annotate and present a novel Cross-modality Water Segmentation (CMWS) dataset to validate our proposed method. To the best of our knowledge, this is the first water segmentation dataset for USVs in inland waterways consisting of images and corresponding point clouds. Extensive experiments on the CMWS dataset demonstrate that our proposed method can significantly improve image-only-based methods, achieving improvements in accuracy and MaxF of approximately 2% for all the image-only-based methods.

Verified
Journal Article
Open access
PDF file
JZJingting Zhang
Jingting Zhang
JGJiantao Gao
Jiantao Gao
JLJinshuo Liang
Jinshuo Liang
YWYiqiang Wu
Yiqiang Wu
BLBin Li
Bin Li
XLXiaomao Li
Xiaomao Li
Water segmentation is a critical task for ensuring the safety of unmanned surface vehicles (USVs). Most existing image-based water segmentation methods may be inaccurate due to light reflection on the water. The fusion-based method combines the paired 2D camera images and 3D LiDAR point clouds as inputs, resulting in a high computational load and considerable time consumption, with limits in terms of practical applications. Thus, in this study, we propose a multimodal fusion water segmentation method that uses a transformer and knowledge distillation to leverage 3D LiDAR point clouds in order to assist in the generation of 2D images. A local and non-local cross-modality fusion module based on a transformer is first used to fuse 2D images and 3D point cloud information during the training phase. A multi-to-single-modality knowledge distillation module is then applied to distill the fused information into a pure 2D network for water segmentation. Extensive experiments were conducted with a dataset containing various scenes collected by USVs in the water. The results demonstrate that the proposed method achieves approximately 1.5% improvement both in accuracy and MaxF over classical image-based methods, and it is much faster than the fusion-based method, achieving speeds ranging from 15 fps to 110 fps.

1 August 2021
Verified
Conference Paper
Open access
PDF file
WZWeibing Zhao
Weibing Zhao
Yan XuYan Xu
Yan Xu
JGJiantao Gao
Jiantao Gao
RZRuimao Zhang
Ruimao Zhang
JZJiayan Zhang
Jiayan Zhang
SCShuguang Cui
Shuguang Cui
Point Cloud Sampling and Recovery (PCSR) is critical for massive real-time point cloud collection and processing since raw data usually requires large storage and computation. This paper addresses a fundamental problem in PCSR: How to downsample the dense point cloud with arbitrary scales while preserving the local topology of discarded points in a case-agnostic manner (i.e., without additional storage for point relationships)? We propose a novel Locally Invertible Embedding (PointLIE) framework to unify the point cloud sampling and upsampling into one single framework through bi-directional learning. Specifically, PointLIE decouples the local geometric relationships between discarded points from the sampled points by progressively encoding the neighboring offsets to a latent variable. Once the latent variable is forced to obey a pre-defined distribution in the forward sampling path, the recovery can be achieved effectively through inverse operations. Taking the recover-pleasing sampled points and a latent embedding randomly drawn from the specified distribution as inputs, PointLIE can theoretically guarantee the fidelity of reconstruction and outperform state-of-the-arts quantitatively and qualitatively.

16 April 2024 in IEEE Transactions on Geoscience and Remote Sensing
Verified
Journal Article
Closed access
YWYiqiang Wu
Yiqiang Wu
WXWeiping Xiao
Weiping Xiao
JGJiantao Gao
Jiantao Gao
CLChang Liu
Chang Liu
YQYu Qin
Yu Qin
XLXiaomao Li
Xiaomao Li
One-stage 3-D object detection, known for its simplicity and high-speed inference, is attracting increasing attention in autonomous driving scenarios. However, current one-stage detectors tend to perform suboptimally compared to two-stage competitors. Our experimental findings suggest that one-stage detectors underperform due to the underutilization of spatial information in feature embedding and classification. Concretely, the spatial context is severely lost during feature propagation, inducing distorted spatial awareness. On the other hand, category recognition relies on the full utilization of spatial information, which is neglected by current detectors. This inadequate spatial awareness of the classification branch can exacerbate misclassification. To address these issues, we propose spatial-aware learning in feature embedding and classification for one-stage 3-D object detection (SLDet). Specifically, to restore the distorted spatial awareness, categorywise spatial augmentation (CSA) is proposed to adaptively bring the network with pre-encoding multiscale spatial contexts. As for misclassification, spatial guiding classification (SGC) is introduced to guide the classification using explicit scale information. It employs the natural scale divergences among categories to rectify misclassification. Comprehensive experiments demonstrate that SLDet efficiently utilizes spatial information and achieves newly state-of-the-art performance on both the Waymo open dataset and the ONCE dataset. Furthermore, additional experiments demonstrate the excellent generalization capacity of SLDet.