Song Wang

Dr. Song Wang


Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 October 2025 in arXiv
Verified
Preprint
Open access
PDF file
THTianyu Hu
Tianyu Hu
ZTZhen Tan
Zhen Tan
SWSong Wang
Song Wang
HQHuaizhi Qu
Huaizhi Qu
Tianlong ChenTianlong Chen
Tianlong Chen
With advancements in reasoning capabilities, Large Language Models (LLMs) are increasingly employed for automated judgment tasks. While LLMs-as-Judges offer promise in automating evaluations, current approaches often rely on simplistic aggregation methods (e.g., majority voting), which can fail even when individual agents provide correct answers. To address this, we propose a multi-agent debate judge framework where agents collaboratively reason and iteratively refine their responses. We formalize the debate process mathematically, analyzing agent interactions and proving that debate amplifies correctness compared to static ensembles. To enhance efficiency, we introduce a stability detection mechanism that models judge consensus dynamics via a time-varying Beta-Binomial mixture, with adaptive stopping based on distributional similarity (Kolmogorov-Smirnov test). This mechanism models the judges' collective correct rate dynamics using a time-varying mixture of Beta-Binomial distributions and employs an adaptive stopping criterion based on distributional similarity (Kolmogorov-Smirnov statistic). Experiments across multiple benchmarks and models demonstrate that our framework improves judgment accuracy over majority voting while maintaining computational efficiency.

14 September 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
ZZzheng zhang
zheng zhang
ZGZhixiang Guo
Zhixiang Guo
YSYuan Si
Yuan Si
SLSiyuan Liang
Siyuan Liang
SLShunyu Liu
Shunyu Liu
DTDacheng Tao
Dacheng Tao
World models are increasingly used to predict and simulate how environments evolve, yet their evaluation remains fragmented across video generation, autonomous driving, and robotics. World-model benchmarks differ in four respects: what capability is evaluated, how the model is evaluated, which metrics are used, and where the data come from. Thus, similar scores may reflect different evidence and do not necessarily support the same capability claims. We present an evaluation-centric survey of 102 representative benchmarks released between 2018 and 2026. We characterize them through four coupled dimensions: Evaluation Target, Evaluation Protocol, Evaluation Metrics, and Evaluation Data. Evaluation Target encompasses visual and temporal quality, spatial and state consistency, long-horizon memory and state persistence, physical plausibility, causal and counterfactual reasoning, control fidelity and interactive dynamics, and functional utility. Our analysis reveals that evaluation targets are expanding toward Control Fidelity and Interactive Dynamics and Functional Utility. However, benchmarks in the former category predominantly use open-loop protocols, leaving reliability under feedback-driven interaction insufficiently tested. These findings highlight the need to align evaluation protocols, metrics, and data with the capability claims they are intended to support. We identify priorities for evaluating functional utility across downstream roles, developing intervention-based and closed-loop protocols, combining prediction-level and downstream outcome metrics, constructing multimodal action-grounded data, and building standardized and reproducible evaluation toolkits. By clarifying what each benchmark measures and which claims its evidence can support, this survey provides a roadmap toward credible and comparable world-model evaluation. The survey website is available at https://world-model-benchmarks.github.io/.

10 February 2026 in arXiv
Verified
Preprint
Open access
PDF file
JLJunhong Lin
Junhong Lin
BZBing Zhang
Bing Zhang
SWSong Wang
Song Wang
ZLZiyan Liu
Ziyan Liu
DGDan Gutfreund
Dan Gutfreund
YZYada Zhu
Yada Zhu
Large language models (LLMs) continue to struggle with knowledge-intensive questions that require up-to-date information and multi-hop reasoning. Augmenting LLMs with hybrid external knowledge, such as unstructured text and structured knowledge graphs, offers a promising alternative to costly continual pretraining. As such, reliable evaluation of their retrieval and reasoning capabilities becomes critical. However, many existing benchmarks increasingly overlap with LLM pretraining data, which means answers or supporting knowledge may already be encoded in model parameters, making it difficult to distinguish genuine retrieval and reasoning from parametric recall. We introduce HybridRAG-Bench, a framework for constructing benchmarks to evaluate retrieval-intensive, multi-hop reasoning over hybrid knowledge. HybridRAG-Bench automatically couples unstructured text and structured knowledge graph representations derived from recent scientific literature on arXiv, and generates knowledge-intensive question-answer pairs grounded in explicit reasoning paths. The framework supports flexible domain and time-frame selection, enabling contamination-aware and customizable evaluation as models and knowledge evolve. Experiments across three domains (artificial intelligence, governance and policy, and bioinformatics) demonstrate that HybridRAG-Bench rewards genuine retrieval and reasoning rather than parametric recall, offering a principled testbed for evaluating hybrid knowledge-augmented reasoning systems. We release our code and data at github.com/junhongmit/HybridRAG-Bench.