Donglin Zhou

Donglin Zhou

Auto-generated profile, based on publicly available information

Donglin Zhou

College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 April 2025 in ACM Transactions on Intelligent Systems and Technology
Verified
Journal Article
Open access
SXShuyi Xie
Shuyi Xie
WYWenlin Yao
Wenlin Yao
YDYong Dai
Yong Dai
SWShaobo Wang
Shaobo Wang
ZXZishan Xu
Zishan Xu
YLYuhong Liu
Yuhong Liu
Large language models (LLMs) have shown impressive capabilities across various natural language tasks. However, evaluating their alignment with human preferences remains a challenge. To this end, we propose a comprehensive human evaluation framework to assess LLMs’ proficiency in following instructions on diverse real-world tasks. We construct a hierarchical task tree encompassing 7 major areas covering over 200 categories and over 800 tasks, which covers diverse capabilities such as question answering, reasoning, multiturn dialogue, and text generation, to evaluate LLMs in a comprehensive and in-depth manner. We also design detailed evaluation standards and processes to facilitate consistent, unbiased judgments from human evaluators. A test set of over 3,000 instances is released, spanning different difficulty levels and knowledge domains. Our work provides a standardized methodology to evaluate human alignment in LLMs for both English and Chinese. We also analyze the feasibility of automating parts of evaluation with a strong LLM (GPT-4). Our framework supports a thorough assessment of LLMs as they are integrated into real-world applications. We have made publicly available the task tree, TencentLLMEval dataset, and evaluation methodology which have been demonstrated as effective in assessing the performance of Tencent Hunyuan LLMs 1 . By doing so, we aim to facilitate the benchmarking of advances in the development of safe and human-aligned LLMs.


1 January 2023
Verified
Conference Paper
Closed access
PDF file
LLLiujun Li
Liujun Li
DZDonglin Zhou
Donglin Zhou
JCJiang Chang
Jiang Chang
DSDev Shrestha
Dev Shrestha