The rapid rise of Large Language Models (LLMs) and Large Reasoning Models (LRMs) has been accompanied by an equally rapid increase of benchmarks used to assess them. However, due to both improved model competence resulting from scaling and novel training advances as well as likely many of these datasets being included in pre or post training data, results become saturated, driving a continuous need for new and more challenging replacements. In this paper, we discuss whether surpassing a benchmark truly demonstrates reasoning ability or are we simply tracking numbers divorced from the capabilities we claim to measure? We present an investigation focused on three model families, OpenAI, Anthropic, and Google, and how their reasoning capabilities across different benchmarks evolve over the years. We also analyze performance trends over the years across different reasoning tasks and discuss the current situation of benchmarking and remaining challenges. By offering a comprehensive overview of benchmarks and reasoning tasks, our work aims to serve as a first reference to ground future research in reasoning evaluation and model development.
As large language models (LLMs) scale to cover more languages, their potential to support low-resource settings becomes increasingly promising. However, the mechanisms underlying cross-lingual transfer and the factors that facilitate it remain insufficiently understood. Prior work has highlighted the role of linguistic similarity—particularly syntactic structure—in enabling transfer across languages. In this study, we present a broad empirical analysis of how multilingual LLMs encode and relate structural information across languages with varying typological properties. We combine multiple complementary methods, including hidden-state similarity analysis, typological correlation, probing for syntactic features, and attention-based structural comparisons, across four multilingual models and thirteen languages. Our findings show consistent correlations between representational similarity and syntactic relatedness, suggesting that structural properties of language influence how information is organized and shared across languages. We further observe that attention-derived structures exhibit partial alignment with gold-standard syntax, though this alignment should be interpreted as heuristic rather than direct evidence of syntactic encoding. Overall, our results provide a comparative empirical perspective on cross-lingual structural bias in multilingual LLMs and highlight the importance of careful methodological interpretation when linking representation geometry to linguistic structure.