Large language models (LLMs) are increasingly adopted across scientific and engineering fields. However, applying general-purpose LLMs to specialized engineering domains imposes stringent requirements for structured knowledge, rigorous reasoning, and technical precision. Thus, the suitability of current general-purpose LLMs for practical applications in engineering domains remains questionable. To understand the mastery level of LLMs in the building science domain as one broad but specific engineering domain, in this paper, we perform a comprehensive benchmark analysis (with benchmark dataset of 1,487 questions) to evaluate abilities of 15 state-of-the-art (SOTA) LLMs across 12 core subject topics in the building science domain. To enable scalable and robust evaluation, we propose and validate an AI-Judger for assessment across five dimensions of abilities, i.e., knowledge and concept, logic and consistency, clarity of expression, and reflection and exploratory. Overall, SOTA general-purposes LLMs achieve only ~50% accuracy on average in answering different types of questions. The capabilities of LLMs decrease progressively from linguistic expression and factual knowledge to logical reasoning, then reflection and exploratory thinking. For different tasks, LLMs exhibit notably low accuracy on calculation (~13%), short-answer (~23%), and cloze tasks (~30%), contrast to stronger performance on single-choice (74%) and multiple-choice questions (63%). Finally, pronounced variance of LLM performance exists across topics, with relatively low accuracy on physics fundamental and HVAC&R-related questions (median of 20%-40%) compared to ~80% for building standards and codes. These identified gaps highlight the limitations of general-purpose LLMs in engineering contexts, clearly pointing to the necessity of developing domain-specific LLMs tailored for engineering applications.

The formulation and implementation of carbon reduction policies are pivotal strategies for attaining the Carbon Peaking and Carbon Neutrality objectives in China, yet there has been limited in-depth research at the policy level. This study systematically compiled 179 central government carbon reduction policy documents and 1183 local government carbon reduction policy documents from China. These policies were classified into command-and-control (CC), market-based (MB), and public participation (PP) categories based on their policy tools. Through detailed content analysis, the intensity of each policy within each category was calculated and the distributions of both quantity and intensity were analyzed. Subsequently, a multiple regression analysis was conducted to evaluate the impact of policy intensity on carbon emissions at the provincial level. The findings highlight a more pronounced policy activity and intensity in the eastern regions relative to the central and western regions, reveal the dominance of CC policies in terms of both their prevalence and intensity, and identify a counterintuitive increase in carbon emissions associated with CC policies. This research elucidates the landscape of China’s carbon reduction policies, offering nuanced insights into their distribution, intensity, and effectiveness in lowering carbon emissions, often a major concern of policymakers, researchers, and industry stakeholders.