Junyuan Tan

Junyuan Tan


Research Focus & Metrics

Academic Career Length

1Year (2026-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
21 July 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
KZKewen Zhu
Kewen Zhu
ZSZe Sheng
Ze Sheng
ZLZixi Liu
Zixi Liu
JTJunyuan Tan
Junyuan Tan
Large Language Models (LLMs) are increasingly deployed as hiring evaluators, yet no systematic reliability audit exists for this high-stakes domain. We benchmark 9 frontier models from 3 families (GPT-5.4, Claude, Gemini) across 10 behavioral questions, 3 companies, and 2 seniority levels (141 cells, 10–50 samples each, 4,600 total ratings), revealing five reliability failures: (1) Scale collapse — a 5-point scale degenerates into 2.03 effective categories (normalized entropy = 0.44); (2) Test-retest unreliability — ICC(2,1) ranges from 0.16 to 0.85 across models; (3) Inter-model disagreement — Krippendorff's α = 0.52, below the 0.67 tentative-conclusion threshold, with same-brand variants (Spearman ρ = 0.25) disagreeing more than cross-vendor pairs (ρ = 0.87); (4) Question-type sensitivity — identical answers receive ratings differing by 2 ordinal grades across question types; (5) Expectation gap — community-endorsed answers score below "Hire" across all models, with flagship models significantly harsher than lightweight (Wilcoxon z = −2.80, p = .005). We propose a minimum reliability checklist and discuss implications for AI hiring policy.