27 March 2026 in SSRN Electronic Journal
Indoor air quality (IAQ) is a major public health concern because modern urban residents spend around 90% of their daily lives indoors. Key concerns include controlling pollutant emissions, managing contaminants, and maintaining ventilation or filtration performance. With the increasing prevalence of Large Language Models (LLMs), IAQ researchers and building engineers may rely on them to guide their decisions. However, the reliability of such tools remains unclear. To address this gap, we benchmarked nine recently released LLMs - spanning closed- and open-weight architectures - against 174 expert-verified conceptual IAQ questions manually derived from authoritative textbooks and the peer-reviewed literature. To capture variability in model responses, each question was tested across five independent trials. Outputs were then automatically evaluated against reference answers using a hybrid framework which integrated overlap-based metrics (BLEU, ROUGE-L), semantic similarity (BERT), and AI-as-judge scoring, and has been trained and tested against expert assessments to ensure estimation accuracy. We found that incorporating the AI-based scoring enhanced the reliability of the automatic evaluation system, especially in identifying incorrect LLM answers. Prompting models with domain-specific instructions slightly improved their accuracy and consistency. Importantly, the accuracy of all LLMs dropped below 80% for certain topics, such as particle impaction and sampling efficiency, possibly due to limited domain-specific training data and specialized terminologies inherent to these topics. This study provides systematic benchmarks of commonly-used LLMs in the context of IAQ, offering guidance on their current capabilities, highlighting areas where reliability is insufficient, and pointing out the need for fine-tuned models to ensure trustworthy decision support for certain IAQ topics.