Hanchong Zhang, Ruisheng Cao, Lu Chen, Hongshen Xu, Kai Yu. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.

Recently, there has been growing interest in using Large Language Models (LLMs) for scientific research. Numerous benchmarks have been proposed to evaluate the ability of LLMs for scientific research. However, current benchmarks are mostly based on pre-collected objective questions. This design suffers from data leakage problem and lacks the evaluation of subjective Q/A ability. In this paper, we propose SciEval, a comprehensive and multi-disciplinary evaluation benchmark to address these issues. Based on Bloom's taxonomy, SciEval covers four dimensions to systematically evaluate scientific research ability. In particular, we design a "dynamic" subset based on scientific principles to prevent evaluation from potential data leakage. Both objective and subjective questions are included in SciEval. These characteristics make SciEval a more effective benchmark for scientific research ability evaluation of LLMs. Comprehensive experiments on most advanced LLMs show that, although GPT-4 achieves SOTA performance compared to other LLMs, there is still substantial room for improvement, especially for dynamic questions. The codes and data are publicly available on https://github.com/OpenDFM/SciEval.

Hypoxia-inducible factor 2α (HIF2α) antagonists are effective against clear cell renal cell carcinomas (ccRCCs) that highly express HIF2α. To identify potential drug targets in HIF2α ^low/− ccRCC, we constructed an epigenetic-focused single-guide RNA library and performed an in vivo CRISPR-Cas9 knockout screen in BALB/c nude mice transplanted with 786-O (HIF2α ^high ) or Caki-2 (HIF2α ^low/− ) cells. We found that the m6A demethylase fat mass and obesity-associated ( FTO ) gene was indispensable to the growth of HIF2α ^low/− but not HIF2α ^high ccRCC. Activation of FTO in HIF2α ^low/− ccRCC was caused by an increased intracellular α-ketoglutarate–to-succinate ratio and stabilized bromodomain-containing protein 9 ( BRD9 ) messenger RNA via m6A demethylation. RNA sequencing and chromatin immunoprecipitation sequencing profiling further revealed that SRY-box transcription factor 17 (SOX17) recruited BRD9 to de novo super enhancers associated with genes that feature prominently in ccRCC pathogenesis, including CCND1 , VEGFR2 , CDC20 , SRC , and MAPK6 . BRD9 knockdown or the BRD9-selective antagonist I-BRD9 suppressed the growth of HIF2α ^low/− but not HIF2α ^high ccRCC cells in vitro. In BALB/c nude mice bearing HIF2α ^low/− ccRCC cell line–derived xenografts and patient-derived tumor xenografts, I-BRD9 administration effectively inhibited tumor growth and prolonged the survival of tumor-bearing mice with greater efficacy than sunitinib. Together, these findings indicate that BRD9 is a druggable target for treating HIF2α ^low/− ccRCC.

Senyu Han, Lu Chen, Li-Min Lin, Zhengshan Xu, Kai Yu. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.