Shichen Lu

Shichen Lu

Auto-generated profile, based on publicly available information

Shichen Lu

Beihang University

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 October 2024
Verified
Conference Paper
Open access
SLShichen Lu
Shichen Lu
LGLongteng Guo
Longteng Guo
WWWenxuan Wang
Wenxuan Wang
ZZZijia Zhao
Zijia Zhao
TYTongtian Yue
Tongtian Yue
SLSi Liu
Si Liu
Recently, large vision language models (LVLMs) have advanced AI by integrating visual and linguistic data for tasks like visual conversation, image captioning, and visual question answering. Current LVLM research either scales up model size for performance or reduces parameters for limited computational resources. We believe both large and tiny models have unique strengths and that collaborative training yields better results than independent training. We propose Collaborative Training of Tiny-Large Vision Language Models (CTVLMs), a framework connecting large and tiny models via a projection layer and leveraging a synergistic training strategy. Our framework improves training efficiency by strengthening the interconnection between large and tiny models. Using the parameter efficiency of tiny models, we effectively align image-text features, then apply knowledge distillation to help large models better align cross-modal information. During fine-tuning, the large model's extensive knowledge enhances tiny model's performance. This collaborative approach allows models to adapt to various computational resources and outperforms existing methods in vision-language tasks.



26 January 2023 in arXiv
Verified
Preprint
Open access
PDF file
KBKeenan Burnett
Keenan Burnett
DYDavid J. Yoon
David J. Yoon
YWYuchen Wu
Yuchen Wu
ALAndrew Zou Li
Andrew Zou Li
HZHaowei Zhang
Haowei Zhang
TBTimothy D. Barfoot
Timothy D. Barfoot
The Boreas dataset was collected by driving a repeated route over the course of one year, resulting in stark seasonal variations and adverse weather conditions such as rain and falling snow. In total, the Boreas dataset includes over 350km of driving data featuring a 128-channel Velodyne Alpha Prime lidar, a 360$^\circ$ Navtech CIR304-H scanning radar, a 5MP FLIR Blackfly S camera, and centimetre-accurate post-processed ground truth poses. Our dataset will support live leaderboards for odometry, metric localization, and 3D object detection. The dataset and development kit are available at https://www.boreas.utias.utoronto.ca

18 May 2024 in arXiv
Verified
Preprint
Open access
PDF file
YYYichen Yan
Yichen Yan
XHXingjian He
Xingjian He
SCSihan Chen
Sihan Chen
SLShichen Lu
Shichen Lu
JLJing Liu
Jing Liu
Referring Image Segmentation (RIS) aims to segment an object described in natural language from an image, with the main challenge being a text-to-pixel correlation. Previous methods typically rely on single-modality features, such as vision or language features, to guide the multi-modal fusion process. However, this approach limits the interaction between vision and language, leading to a lack of fine-grained correlation between the language description and pixel-level details during the decoding process. In this paper, we introduce FCNet, a framework that employs a bi-directional guided fusion approach where both vision and language play guiding roles. Specifically, we use a vision-guided approach to conduct initial multi-modal fusion, obtaining multi-modal features that focus on key vision information. We then propose a language-guided calibration module to further calibrate these multi-modal features, ensuring they understand the context of the input sentence. This bi-directional vision-language guided approach produces higher-quality multi-modal features sent to the decoder, facilitating adaptive propagation of fine-grained semantic information from textual features to visual features. Experiments on RefCOCO, RefCOCO+, and G-Ref datasets with various backbones consistently show our approach outperforming state-of-the-art methods.