shengchuan zhang

Dr. shengchuan zhang


Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 October 2022
Verified
Book Chapter
Closed access
PDF file
JJJiazhen Ji
Jiazhen Ji
HWHuan Wang
Huan Wang
YHYuge Huang
Yuge Huang
JWJiaxiang Wu
Jiaxiang Wu
XXXingkun Xu
Xingkun Xu
RJRongrong Ji
Rongrong Ji

28 October 2024
Verified
Conference Paper
Open access
YQYansong Qu
Yansong Qu
SDShaohui Dai
Shaohui Dai
XLXinyang Li
Xinyang Li
JLJianghang Lin
Jianghang Lin
LCLiujuan Cao
Liujuan Cao
RJRongrong Ji
Rongrong Ji
3D open-vocabulary scene understanding, crucial for advancing augmented reality and robotic applications, involves interpreting and locating specific regions within a 3D space as directed by natural language instructions. To this end, we introduce GOI, a framework that integrates semantic features from 2D vision-language foundation models into 3D Gaussian Splatting (3DGS) and identifies 3D Gaussians of Interest using an Optimizable Semantic-space Hyperplane. Our approach includes an efficient compression method that utilizes scene priors to condense noisy high-dimensional semantic features into compact low-dimensional vectors, which are subsequently embedded in 3DGS. During the open-vocabulary querying process, we adopt a distinct approach compared to existing methods, which depend on a manually set fixed empirical threshold to select regions based on their semantic feature distance to the query text embedding. This traditional approach often lacks universal accuracy, leading to challenges in precisely identifying specific target areas. Instead, our method treats the feature selection process as a hyperplane division within the feature space, retaining only those features that are highly relevant to the query. We leverage off-the-shelf 2D Referring Expression Segmentation (RES) models to fine-tune the semantic-space hyperplane, enabling a more precise distinction between target regions and others. This fine-tuning substantially improves the accuracy of open-vocabulary queries, ensuring the precise localization of pertinent 3D Gaussians. Extensive experiments demonstrate GOI's superiority over previous state-of-the-art methods. The dataset, model, and code are available at https://quyans.github.io/GOI-Hyperplane/.

28 October 2024
Verified
Conference Paper
Open access
TGTimin Gao
Timin Gao
PCPeixian Chen
Peixian Chen
MZMengdan Zhang
Mengdan Zhang
CFChaoyou Fu
Chaoyou Fu
Yunhang ShenYunhang Shen
Yunhang Shen
RJRongrong Ji
Rongrong Ji
With the advent of large language models(LLMs) enhanced by the chain-of-thought(CoT) methodology, the visual reasoning problem is usually decomposed into manageable sub-tasks and tackled sequentially with various external tools. However, such a paradigm faces the challenge of the potential "determining hallucinations" in decision generation due to insufficient visual information and the limitation of low-level perception tools that fail to provide abstract summaries necessary for comprehensive reasoning. We argue that converging visual context acquisition and logical reasoning is pivotal for tackling visual reasoning tasks. This paper delves into the realm of multimodal CoT to solve intricate visual reasoning tasks with multimodal large language models(MLLMs) and their cognitive capability. To this end, we propose an innovative multimodal CoT framework, termed Cantor, characterized by a perception-decision architecture. Cantor first acts as a decision generator and integrates visual inputs to analyze the image and problem, ensuring a closer alignment with the actual context. Furthermore, Cantor leverages the advanced cognitive functions of MLLMs to perform as multifaceted experts for deriving higher-level information, enhancing the CoT generation process. Our extensive experiments demonstrate the efficacy of the proposed framework, showing significant improvements in multimodal CoT performance across two complex visual reasoning datasets, without necessitating fine-tuning or ground-truth rationales. Project Page: https://ggg0919.github.io/cantor/.


28 October 2024
Verified
Conference Paper
Open access
PYPengfei Yue
Pengfei Yue
JLJianghang Lin
Jianghang Lin
SZshengchuan zhang
shengchuan zhang
JHJie Hu
Jie Hu
YLYilin Lu
Yilin Lu
RJRongrong Ji
Rongrong Ji
Referring image segmentation (RIS) aims to segment a particular region based on a specific expression. Existing one-stage methods have explored various fusion strategies, yet they encounter two significant issues. Primarily, most methods rely on manually selected visual features from the visual encoder layers. Moreover, the direct fusion of word-level features into coarse aligned features disrupts the established vision-language alignment. In this paper, we introduce an innovative framework for RIS that seeks to overcome these challenges with adaptive alignment of vision and language features, termed the Adaptive Selection with Dual Alignment (ASDA). ASDA innovates in two aspects. Firstly, we design an Adaptive Feature Selection and Fusion (AFSF) module to dynamically select visual features focusing on different regions related to various descriptions. AFSF is equipped with scale-wise feature aggregator to provide hierarchically coarse features that preserve crucial low-level details. Secondly, a Word Guided Dual-Branch Aligner (WGDA) is leveraged to integrate coarse features with linguistic cues by word-guided attention, which effectively addresses the common issue of vision-language misalignment. Extensive experimental results demonstrate that our ASDA framework surpasses state-of-the-art methods on RefCOCO, RefCOCO+ and G-Ref benchmark.

24 November 2024
Verified
Book Chapter
Closed access
PDF file
XLXunfa Lai
Xunfa Lai
ZYZhiyu Yang
Zhiyu Yang
JHJie Hu
Jie Hu
SZshengchuan zhang
shengchuan zhang
LCLiujuan Cao
Liujuan Cao
RJRongrong Ji
Rongrong Ji