Xiaoyuan Yi

Xiaoyuan Yi

Auto-generated profile, based on publicly available information

Xiaoyuan Yi

Microsoft Research Asia, Beijing, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 March 2024 in ACM Transactions on Intelligent Systems and Technology
Verified
Journal Article
Open access
YCYupeng Chang
Yupeng Chang
XWXu Wang
Xu Wang
JWJindong Wang
Jindong Wang
YWYuan Wu
Yuan Wu
LYLinyi Yang
Linyi Yang
XXXing Xie
Xing Xie
Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level, but also at the society level for better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate , where to evaluate , and how to evaluate . Firstly, we provide an overview from the perspective of evaluation tasks, encompassing general natural language processing tasks, reasoning, medical usage, ethics, education, natural and social sciences, agent applications, and other areas. Secondly, we answer the ‘where’ and ‘how’ questions by diving into the evaluation methods and benchmarks, which serve as crucial components in assessing the performance of LLMs. Then, we summarize the success and failure cases of LLMs in different tasks. Finally, we shed light on several future challenges that lie ahead in LLMs evaluation. Our aim is to offer invaluable insights to researchers in the realm of LLMs evaluation, thereby aiding the development of more proficient LLMs. Our key point is that evaluation should be treated as an essential discipline to better assist the development of LLMs. We consistently maintain the related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey

20 May 2024 in IEEE Transactions on Big Data
Verified
Journal Article
Closed access
YLYuqing Lan
Yuqing Lan
ZLZhenghao Liu
Zhenghao Liu
YGYu Gu
Yu Gu
XYXiaoyuan Yi
Xiaoyuan Yi
XLXiaohua Li
Xiaohua Li
GYGe Yu
Ge Yu
Fact verification tasks aim to identify the integrity of textual contents according to the truthful corpus. Existing fact verification models usually build a fully connected reasoning graph, which regards claim-evidence pairs as nodes and connects them with edges. They employ the graph to propagate the semantics of the nodes. Nevertheless, the noisy nodes usually propagate their semantics via the edges of the reasoning graph, which misleads the semantic representations of other nodes and amplifies the noise signals. To mitigate the propagation of noisy semantic information, we introduce a Confidential Graph Attention Network (CO-GAT), which proposes a node masking mechanism for modeling the nodes. Specifically, CO-GAT calculates the node confidence score by estimating the relevance between the claim and evidence pieces. Then, the node masking mechanism uses the node confidence scores to control the noise information flow from the vanilla node to the other graph nodes. CO-GAT achieves a 73.59% FEVER score on the FEVER dataset and shows the generalization ability by broadening the effectiveness to the science-specific domain.

27 October 2025
Verified
Conference Paper
Open access
ZLZhenghao Liu
Zhenghao Liu
XZXingsheng Zhu
Xingsheng Zhu
TZTianshuo Zhou
Tianshuo Zhou
XZXinyi Zhang
Xinyi Zhang
XYXiaoyuan Yi
Xiaoyuan Yi
MSMaosong Sun
Maosong Sun

25 June 2025 in ACM Transactions on Information Systems
Verified
Journal Article
Open access
JYJing Yao
Jing Yao
XWXiting Wang
Xiting Wang
JLJianxun Lian
Jianxun Lian
XYXiaoyuan Yi
Xiaoyuan Yi
XXXing Xie
Xing Xie
Explainability is critical for recommender systems to ensure good user experience and facilitate designers to debug. However, generating explanations in recommender systems usually requires large efforts due to the dependency on additional data and case-by-case model design. One possible solution to these challenges is reasoning with logic rules, whose validity or confidence can automatically indicate high-quality explanations and formats are general. However, pioneer methods can be hardly applied in recommendation due to the high sparsity of interaction data, which raises the difficulty in accurately computing the rule validity, and the specific ranking-oriented task. To bridge this gap, we propose a general framework for Reco mmendation with lo gic r ule reasoning ( Recolor ) that satisfies three desirable properties. First, we explicitly estimate the rule validity to ensure well-grounded decisions, where a fuzzy logic validity module is designed for accurate estimation on highly sparse recommendation data. Second, we ensure the generality for both the types of input data and model architectures by designing a neural logic generation module, which decouples the user-item representation learning from the rule construction. Third, we integrate the two above-mentioned modules with a ranking-oriented BPR loss and achieve a unified optimization of explainability and accuracy. For any given neural recommendation model, our proposed logic rule reasoning framework can upgrade it to a self-explainable version. Numerical experiments and user studies on four public recommendation datasets with different levels of sparsity demonstrate that our framework shows high-validity rule explanations, generality in architecture and data, and high recommendation accuracy.

28 October 2024
Verified
Conference Paper
Open access
XWXingqi Wang
Xingqi Wang
XYXiaoyuan Yi
Xiaoyuan Yi
XXXing Xie
Xing Xie
JJJia Jia
Jia Jia
Recent advancements in diffusion models trained on large-scale data have enabled the generation of indistinguishable human-level images, yet they often produce harmful content misaligned with human values, e.g., social bias, and offensive content. Despite extensive research on Large Language Models (LLMs), the challenge of Text-to-Image (T2I) model alignment remains largely unexplored. Addressing this problem, we propose LiVO (Lightweight Value Optimization), a novel lightweight method for aligning T2I models with human values. LiVO only optimizes a plug-and-play value encoder to integrate a specified value principle with the input prompt, allowing the control of generated images over both semantics and values. Specifically, we design a diffusion model-tailored preference optimization loss, which theoretically approximates the Bradley-Terry model used in LLM alignment but provides a more flexible trade-off between image quality and value conformity. To optimize the value encoder, we also develop a framework to automatically construct a text-image preference dataset of 86k (prompt, aligned image, violating image, value principle) samples. Without updating most model parameters and through adaptive value selection from the input prompt, LiVO significantly reduces harmful outputs and achieves faster convergence, surpassing several strong baselines and taking an initial step towards ethically aligned T2I models. Warning: This paper involves descriptions and images depicting discriminatory, pornographic, bloody, and horrific scenes.