Linyi Yang

Linyi Yang

Auto-generated profile, based on publicly available information

Linyi Yang

Southern University of Science and Technology, Shenzhen, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 March 2024 in ACM Transactions on Intelligent Systems and Technology
Verified
Journal Article
Open access
YCYupeng Chang
Yupeng Chang
XWXu Wang
Xu Wang
JWJindong Wang
Jindong Wang
YWYuan Wu
Yuan Wu
LYLinyi Yang
Linyi Yang
XXXing Xie
Xing Xie
Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level, but also at the society level for better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate , where to evaluate , and how to evaluate . Firstly, we provide an overview from the perspective of evaluation tasks, encompassing general natural language processing tasks, reasoning, medical usage, ethics, education, natural and social sciences, agent applications, and other areas. Secondly, we answer the ‘where’ and ‘how’ questions by diving into the evaluation methods and benchmarks, which serve as crucial components in assessing the performance of LLMs. Then, we summarize the success and failure cases of LLMs in different tasks. Finally, we shed light on several future challenges that lie ahead in LLMs evaluation. Our aim is to offer invaluable insights to researchers in the realm of LLMs evaluation, thereby aiding the development of more proficient LLMs. Our key point is that evaluation should be treated as an essential discipline to better assist the development of LLMs. We consistently maintain the related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey

2 September 2025 in ACM Computing Surveys
Verified
Journal Article
Open access
CWCunxiang Wang
Cunxiang Wang
XLXiaoze Liu
Xiaoze Liu
YYYuanhao Yue
Yuanhao Yue
QGQipeng Guo
Qipeng Guo
XHXiangkun Hu
Xiangkun Hu
YZYue Zhang
Yue Zhang
This survey addresses the crucial issue of factuality in Large Language Models (LLMs). As LLMs find applications across diverse domains, the reliability and accuracy of their outputs become vital. We define the “factuality issue” as the probability of LLMs to produce content inconsistent with established facts. We first delve into the implications of these inaccuracies. Subsequently, we analyze the mechanisms through which LLMs store and process facts, seeking the primary causes of factual errors. Our discussion then transitions to methodologies for evaluating LLM factuality, emphasizing key metrics, benchmarks, and studies. We further explore strategies for enhancing LLM factuality. Our survey offers a structured guide for researchers aiming to fortify the factual reliability of LLMs. We consistently maintain and update the related open-source materials at https://github.com/wangcunxiang/LLM-Factuality-Survey .

18 July 2023
Verified
Conference Paper
Open access
FGFang Guo
Fang Guo
YLYun Luo
Yun Luo
LYLinyi Yang
Linyi Yang
YZYue Zhang
Yue Zhang
Systematic review is a crucial method that has been widely used. by scholars from different research domains. However, screening for relevant scientific literature from paper candidates remains an extremely time-consuming process so the task of screening prioritization has been established to reduce the human workload. Various methods under the human-in-the-loop fashion are proposed to solve this task by using lexical features. These methods, even though achieving better performance than more sophisticated feature-based models such as BERT, omit rich and essential semantic information, therefore suffered from feature bias. In this study, we propose a novel framework SciMine to accelerate this screening process by capturing semantic feature representations from both background and the corpus. In particular, based on contextual representation learned from the pre-trained language models, our approach utilizes an autoencoder-based classifier and a feature-dependent classification module to extract general document-level and phrase-level information. Then a ranking ensemble strategy is used to combine these two complementary pieces of information. Experiments on five real-world datasets demonstrate that SciMine achieves state-of-the-art performance and comprehensive analysis further shows the efficacy of SciMine to solve feature bias.

26 October 2023
Verified
Conference Paper
Open access
CLChenyang Lyu
Chenyang Lyu
WLWenxi Li
Wenxi Li
TJTianbo Ji
Tianbo Ji
LWLongyue Wang
Longyue Wang
LZLiting Zhou
Liting Zhou
JFJennifer Foster
Jennifer Foster
Video-language learning has attracted significant attention in the fields of multimedia, computer vision and natural language processing in recent years. One of the key challenges in this area is how to effectively integrate visual and linguistic information to enable machines to understand video content and query information. In this work, we leverage graph-based representations and multi-grained audio-visual alignment to address this challenge. First, our approach starts by transforming video and query inputs into visual-scene graphs and semantic role graphs using a visual-scene parser and semantic role labeler respectively. These graphs are then encoded using graph neural networks to obtain enriched representations and combined to obtain a video-query joint representation that enhances the semantic expressivity of the inputs. Second, to achieve accurate matching of relevant parts of audio and visual features, we propose a multi-grained alignment module that aligns the audio and visual features at multiple scales. This enables us to effectively fuse the audio and visual information in a way that is consistent with the semantic-level information captured by the graph-based representations. Experiments on five representative datasets collected for Video Retrieval and Video Question Answering tasks show that our approach outperforms the literature on several metrics. Our extensive ablation studies demonstrate the effectiveness of graph-based representation and multi-grained audio-visual alignment.

27 November 2025 in ACM Transactions on Intelligent Systems and Technology
Verified
Journal Article
Open access
JWJindong Wang
Jindong Wang
LYLinyi Yang
Linyi Yang
SSSunayana Sitaram
Sunayana Sitaram
QYQiang Yang
Qiang Yang
BRBhiksha Raj
Bhiksha Raj

15 April 2022 in arXiv
Verified
Preprint
Open access
PDF file
YLYun Luo
Yun Luo
HCHongjie Cai
Hongjie Cai
LYLinyi Yang
Linyi Yang
YQYanxia Qin
Yanxia Qin
RXRui Xia
Rui Xia
YZYue Zhang
Yue Zhang
Since previous studies on open-domain targeted sentiment analysis are limited in dataset domain variety and sentence level, we propose a novel dataset consisting of 6,013 human-labeled data to extend the data domains in topics of interest and document level. Furthermore, we offer a nested target annotation schema to extract the complete sentiment information in documents, boosting the practicality and effectiveness of open-domain targeted sentiment analysis. Moreover, we leverage the pre-trained model BART in a sequence-to-sequence generation method for the task. Benchmark results show that there exists large room for improvement of open-domain targeted sentiment analysis. Meanwhile, experiments have shown that challenges remain in the effective use of open-domain data, long documents, the complexity of target structure, and domain variances.