Cunxiao Du

Cunxiao Du

Auto-generated profile, based on publicly available information

Cunxiao Du

Sea AI Lab

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2025
Verified
Conference Paper
Open access
PDF file
XZXuan Zhang
Xuan Zhang
CDCunxiao Du
Cunxiao Du
SYSicheng Yu
Sicheng Yu
JWJiawei Wu
Jiawei Wu
FZFengzhuo Zhang
Fengzhuo Zhang
QLQian Liu
Qian Liu
Xuan Zhang, Cunxiao Du, Sicheng Yu, Jiawei Wu, Fengzhuo Zhang, Wei Gao, Qian Liu. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2025.

20 November 2024 in Qeios Preprints
Verified
Preprint
Open access
PDF file
XZXuan Zhang
Xuan Zhang
CDCunxiao Du
Cunxiao Du
CDChao Du
Chao Du
TPTianyu Pang
Tianyu Pang
WGWei Gao
Wei Gao
MLMin Lin
Min Lin
Recent advancements in large language models (LLMs) have extended their capabilities to handle long contexts. However, increasing the number of model layers and the length of input sequences significantly escalates the memory required to store key-value (KV) cache, posing challenges for efficient inference. To mitigate this issue, we present SimLayerKV, a simple yet effective method that reduces inter-layer KV cache redundancies by selectively dropping cache in identified lazy layers. Our approach is based on the observation that certain layers in long-context LLMs exhibit “lazy” behavior, contributing less to modeling long-range dependencies compared to non-lazy layers. By analyzing attention weight patterns, we find that the behavior of these lazy layers is consistent across tokens during generation for a given input. This insight motivates our SimLayerKV, which identifies lazy layers and reduces their KV cache accordingly. SimLayerKV is training-free, generalizable, and can be implemented with only seven lines of code. We conduct extensive experiments on three representative LLMs, e.g., LLaMA2-7B, LLaMA3-8B, and Mistral-7B across 16 tasks from the LongBench benchmark. The results demonstrate that SimLayerKV achieves a KV cache compression ratio of 5\(\times\)with only a 1.2% performance drop when combined with 4-bit quantization. Our code is available at https://github.com/sail-sg/SimLayerKV.

25 May 2025 in arXiv
Verified
Preprint
Open access
PDF file
XZXuan Zhang
Xuan Zhang
CDCunxiao Du
Cunxiao Du
SYSicheng Yu
Sicheng Yu
JWJiawei Wu
Jiawei Wu
FZFengzhuo Zhang
Fengzhuo Zhang
QLQian Liu
Qian Liu
Due to the auto-regressive nature of current video large language models (Video-LLMs), the inference latency increases as the input sequence length grows, posing challenges for the efficient processing of video sequences that are usually very long. We observe that during decoding, the attention scores of most tokens in Video-LLMs tend to be sparse and concentrated, with only certain tokens requiring comprehensive full attention. Based on this insight, we introduce Sparse-to-Dense (StD), a novel decoding strategy that integrates two distinct modules: one leveraging sparse top-K attention and the other employing dense full attention. These modules collaborate to accelerate Video-LLMs without loss. The fast (sparse) model speculatively decodes multiple tokens, while the slow (dense) model verifies them in parallel. StD is a tuning-free, plug-and-play solution that achieves up to a 1.94$\times$ walltime speedup in video processing. It maintains model performance while enabling a seamless transition from a standard Video-LLM to a sparse Video-LLM with minimal code modifications.