Dongsheng Jiang

Dongsheng Jiang

Auto-generated profile, based on publicly available information

Dongsheng Jiang

HuaWei Inc., Shenzhen, China

Research Focus & Metrics

Academic Career Length

17Years (2010-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 February 2023
Verified
Book Chapter
Closed access
PDF file
HCHu Cao
Hu Cao
YWYueyue Wang
Yueyue Wang
JCJoy Chen
Joy Chen
DJDongsheng Jiang
Dongsheng Jiang
XZXiaopeng Zhang
Xiaopeng Zhang
Manning WangManning Wang
Manning Wang

3 November 2022
Verified
Book Chapter
Closed access
PDF file
YCYabo Chen
Yabo Chen
YLYuchen Liu
Yuchen Liu
DJDongsheng Jiang
Dongsheng Jiang
XZXiaopeng Zhang
Xiaopeng Zhang
WDWenrui Dai
Wenrui Dai
QTQi Tian
Qi Tian

3 July 2024 in IEEE Transactions on Intelligent Transportation Systems
Verified
Journal Article
Open access
HCHu Cao
Hu Cao
GCGuang Chen
Guang Chen
HZHengshuang Zhao
Hengshuang Zhao
DJDongsheng Jiang
Dongsheng Jiang
XZXiaopeng Zhang
Xiaopeng Zhang
AKAlois Christian Knoll
Alois Christian Knoll
Image segmentation plays a critical role in autonomous driving by providing vehicles with a detailed and accurate understanding of their surroundings. Transformers have recently shown encouraging results in image segmentation. However, transformer-based models are challenging to strike a better balance between performance and efficiency. The computational complexity of the transformer-based models is quadratic with the number of inputs, which severely hinders their application in dense prediction tasks. In this paper, we present the semantic-aware dimension-pooling transformer (SDPT) to mitigate the conflict between accuracy and efficiency. The proposed model comprises an efficient transformer encoder for generating hierarchical features and a semantic-balanced decoder for predicting semantic masks. In the encoder, a dimension-pooling mechanism is used in the multi-head self-attention (MHSA) to reduce the computational cost, and a parallel depth-wise convolution is used to capture local semantics. Simultaneously, we further apply this dimension-pooling attention (DPA) to the decoder as a refinement module to integrate multi-level features. With such a simple yet powerful encoder-decoder framework, we empirically demonstrate that the proposed SDPT achieves excellent performance and efficiency on various popular benchmarks, including ADE20K, Cityscapes, and COCO-Stuff. For example, our SDPT achieves 48.6 $\%$ mIOU on the ADE20K dataset, which outperforms the current methods with fewer computational costs. The codes can be found at https://github.com/HuCaoFighting/SDPT.

28 June 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
JCJiazhong Cen
Jiazhong Cen
ZJZekun Jiang
Zekun Jiang
LXLingxi Xie
Lingxi Xie
DJDongsheng Jiang
Dongsheng Jiang
WSWei Shen
Wei Shen
QTQi Tian
Qi Tian
Anomaly segmentation is a critical task for safety-critical applications, such as autonomous driving in urban environments. Its objective is to detect out-of-distribution (OOD) samples with unseen categories, given a pre-trained segmentation model. The core challenge of this task is how to distinguish hard in-distribution samples from OOD samples, which has not been explicitly discussed in previous research. In this paper, we propose a simple yet effective approach named CosMe (Consensus Synergizes with Memory) to address this challenge. CosMe consists of two key components: 1) building a memory bank comprising seen prototypes extracted from multiple layers of the given segmentation model, and 2) training an auxiliary model that mimics the behavior of the given model and using the consensus of their mid-level features as complementary cues that synergize with the memory bank. The former serves as a baseline that can detect all potential outliers, including both OOD and hard in-distribution samples; the latter assists in distinguishing between these two types of outliers. Experimental results on several urban scene anomaly segmentation datasets demonstrate that CosMe outperforms previous approaches by a significant margin.

8 March 2024 in arXiv
Verified
Preprint
Open access
PDF file
DJDongsheng Jiang
Dongsheng Jiang
YLYuchen Liu
Yuchen Liu
SLSonglin Liu
Songlin Liu
JZJin'E Zhao
Jin'E Zhao
HZHao Zhang
Hao Zhang
HXHongkai Xiong
Hongkai Xiong
Multi-modal Large Language Models (MLLMs) have made significant strides in expanding the capabilities of Large Language Models (LLMs) through the incorporation of visual perception interfaces. Despite the emergence of exciting applications and the availability of diverse instruction tuning data, existing approaches often rely on CLIP or its variants as the visual branch, and merely extract features from the deep layers. However, these methods lack a comprehensive analysis of the visual encoders in MLLMs. In this paper, we conduct an extensive investigation into the effectiveness of different vision encoders within MLLMs. Our findings reveal that the shallow layer features of CLIP offer particular advantages for fine-grained tasks such as grounding and region understanding. Surprisingly, the vision-only model DINO, which is not pretrained with text-image alignment, demonstrates promising performance as a visual branch within MLLMs. By simply equipping it with an MLP layer for alignment, DINO surpasses CLIP in fine-grained related perception tasks. Building upon these observations, we propose a simple yet effective feature merging strategy, named COMM, that integrates CLIP and DINO with Multi-level features Merging, to enhance the visual capabilities of MLLMs. We evaluate COMM through comprehensive experiments on a wide range of benchmarks, including image captioning, visual question answering, visual grounding, and object hallucination. Experimental results demonstrate the superior performance of COMM compared to existing methods, showcasing its enhanced visual capabilities within MLLMs.

20 May 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
JLJin Li
Jin Li
YWYaoming Wang
Yaoming Wang
XZXiaopeng Zhang
Xiaopeng Zhang
DJDongsheng Jiang
Dongsheng Jiang
WDWenrui Dai
Wenrui Dai
QTQi Tian
Qi Tian
Recent advances in self-supervised learning have witnessed great achievements, especially with the introduction of contrastive learning, where the goal is to maximize the mutual information between different augmentations of the same image, i.e., positive pairs. However, such optimization does not necessarily correspond to optimal representation due to noisy samples, thus inevitably being over-confident in the relevance between views. As a result, the learned model would capture spurious correlation and retain superfluous information that deteriorates representations. In this paper, we facilitate contrastive learning by reducing superfluous relevance between positive views. To this end, we introduce the representation entropy minimization regularization over the objective of vanilla contrastive learning, which forces representations to retain possibly the least information, thus alleviating superfluous relevance from irrelevant views. Then, we derive the analytical expression of the proposed objective by converting it to an information bottleneck problem and solving via variation approximation, which leads to a novel contrastive learning framework, termed as CLIMB, short for Contrastive Learning via variational InforMation Bottleneck. Experiments over multiple benchmarks demonstrate that CLIMB brings consistent improvement. Notably, using DINO as an instantiation, CLIMB achieves 4.5% and 3.5% gain under the k-NN classification metric with EfficientNet-B0 and ResNet-50 as backbones, respectively.