Kun Zhu

Dr. Kun Zhu

orcid

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2023
Verified
Conference Paper
Open access
PDF file
KZKun Zhu
Kun Zhu
XFXiaocheng Feng
Xiaocheng Feng
XFXiachong Feng
Xiachong Feng
YWYingsheng Wu
Yingsheng Wu
BQBing Qin
Bing Qin
Kun Zhu, Xiaocheng Feng, Xiachong Feng, Yingsheng Wu, Bing Qin. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.

17 November 2023 in arXiv
Verified
Preprint
Open access
PDF file
KZKun Zhu
Kun Zhu
XFXiaocheng Feng
Xiaocheng Feng
XFXiachong Feng
Xiachong Feng
YWYingsheng Wu
Yingsheng Wu
BQBing Qin
Bing Qin
Scientific literature review generation aims to extract and organize important information from an abundant collection of reference papers and produces corresponding reviews while lacking a clear and logical hierarchy. We observe that a high-quality catalogue-guided generation process can effectively alleviate this problem. Therefore, we present an atomic and challenging task named Hierarchical Catalogue Generation for Literature Review as the first step for review generation, which aims to produce a hierarchical catalogue of a review paper given various references. We construct a novel English Hierarchical Catalogues of Literature Reviews Dataset with 7.6k literature review catalogues and 389k reference papers. To accurately assess the model performance, we design two evaluation metrics for informativeness and similarity to ground truth from semantics and structure.Our extensive analyses verify the high quality of our dataset and the effectiveness of our evaluation metrics. We further benchmark diverse experiments on state-of-the-art summarization models like BART and large language models like ChatGPT to evaluate their capabilities. We further discuss potential directions for this task to motivate future research.

29 October 2024 in arXiv
Verified
Preprint
Open access
PDF file
YGYuxuan Gu
Yuxuan Gu
XFXiaocheng Feng
Xiaocheng Feng
LHLei Huang
Lei Huang
YWYingsheng Wu
Yingsheng Wu
ZZZekun Zhou
Zekun Zhou
BQBing Qin
Bing Qin
We present an novel framework for efficiently and effectively extending the powerful continuous diffusion processes to discrete modeling. Previous approaches have suffered from the discrepancy between discrete data and continuous modeling. Our study reveals that the absence of guidance from discrete boundaries in learning probability contours is one of the main reasons. To address this issue, we propose a two-step forward process that first estimates the boundary as a prior distribution and then rescales the forward trajectory to construct a boundary conditional diffusion model. The reverse process is proportionally adjusted to guarantee that the learned contours yield more precise discrete data. Experimental results indicate that our approach achieves strong performance in both language modeling and discrete image generation tasks. In language modeling, our approach surpasses previous state-of-the-art continuous diffusion language models in three translation tasks and a summarization task, while also demonstrating competitive performance compared to auto-regressive transformers. Moreover, our method achieves comparable results to continuous diffusion models when using discrete ordinal pixels and establishes a new state-of-the-art for categorical image generation on the Cifar-10 dataset.

19 December 2024 in arXiv
Verified
Preprint
Open access
PDF file
YGYuxuan Gu
Yuxuan Gu
WWWenjie Wang
Wenjie Wang
XFXiaocheng Feng
Xiaocheng Feng
WZWeihong Zhong
Weihong Zhong
KZKun Zhu
Kun Zhu
BQBing Qin
Bing Qin
Large language models (LLMs) have demonstrated impressive instruction following capabilities, while still struggling to accurately manage the length of the generated text, which is a fundamental requirement in many real-world applications. Existing length control methods involve fine-tuning the parameters of LLMs, which is inefficient and suboptimal for practical use. In this paper, we propose a novel iterative sampling framework for text length control, integrating the Metropolis-Hastings algorithm with an importance sampling acceleration strategy. This framework efficiently and reliably regulates LLMs to generate length-constrained text without modifying the underlying parameters, thereby preserving the original capabilities of LLMs. Experimental results demonstrate that our framework achieves almost 100\% success rates of length control on Llama3.1 for tasks such as length-controlled abstractive summarization and length-constrained instruction following, with minimal additional computational overhead. This also highlights the significant potential of our method for precise length control across a broader range of applications, without compromising the versatility of LLMs.