Jinghui Chen

Dr. Jinghui Chen


Research Focus & Metrics

Academic Career Length

15Years (2012-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 April 2023
Verified
Conference Paper
Open access
JLJooyoung Lee
Jooyoung Lee
TLThai Le
Thai Le
JCJinghui Chen
Jinghui Chen
DLDongwon Lee
Dongwon Lee
Past literature has illustrated that language models (LMs) often memorize parts of training instances and reproduce them in natural language generation (NLG) processes. However, it is unclear to what extent LMs “reuse” a training corpus. For instance, models can generate paraphrased sentences that are contextually similar to training samples. In this work, therefore, we study three types of plagiarism (i.e., verbatim, paraphrase, and idea) among GPT-2 generated texts, in comparison to its training data, and further analyze the plagiarism patterns of fine-tuned LMs with domain-specific corpora which are extensively used in practice. Our results suggest that (1) three types of plagiarism widely exist in LMs beyond memorization, (2) both size and decoding methods of LMs are strongly associated with the degrees of plagiarism they exhibit, and (3) fine-tuned LMs’ plagiarism patterns vary based on their corpus similarity and homogeneity. Given that a majority of LMs’ training data is scraped from the Web without informing content owners, their reiteration of words, phrases, and even core ideas from training sets into generated texts has ethical implications. Their patterns are likely to exacerbate as both the size of LMs and their training data increase, raising concerns about indiscriminately pursuing larger models with larger training corpora. Plagiarized content can also contain individuals’ personal and sensitive information. These findings overall cast doubt on the practicality of current LMs in mission-critical writing tasks and urge more discussions around the observed phenomena. Data and source code are available at https://github.com/Brit7777/LM-plagiarism.

5 February 2021 in medRxiv
Verified
Preprint
Open access
PDF file
ECEstee Y Cramer
Estee Y Cramer
EREvan L Ray
Evan L Ray
VLVelma K Lopez
Velma K Lopez
JBJohannes Bracher
Johannes Bracher
ABAndrea Brennen
Andrea Brennen
NRNicholas G Reich
Nicholas G Reich
Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multi-model ensemble forecast that combined predictions from dozens of different research groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-week horizon 3-5 times larger than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. Significance Statement This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the US. Results show high variation in accuracy between and within stand-alone models, and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public health action.

25 September 2023 in BMC Anesthesiology
Verified
Journal Article
Open access
PDF file
XMXiaofei Mo
Xiaofei Mo
FHFa Huang
Fa Huang
XWXiaoying Wu
Xiaoying Wu
JFJumian Feng
Jumian Feng
JZJiequn Zeng
Jiequn Zeng
JCJinghui Chen
Jinghui Chen
Background: Intrathecal dexmedetomidine, as an adjuvant to local anesthetics, has been reported to improve the quality of spinal anesthesia and reduce the required local anesthetic dose. However, the optimal dosage regimen for intrathecal dexmedetomidine combined with plain ropivacaine for cesarean section (CS) remains undetermined. The present study aimed to determine the median effective dose (ED50) of intrathecal dexmedetomidine as an adjuvant to plain ropivacaine for spinal anesthesia during CS. Methods: Sixty parturients undergoing CS were randomly assigned to either group: plain ropivacaine 8 mg (Group Rop8) or plain ropivacaine 10 mg (Group Rop10). The initial dosage of intrathecal dexmedetomidine in each group was 5 µg. The effective dose was defined as a bilateral sensory block at the level of T6 or above to pinprick attained within 10 min after intrathecal injection, without the need for supplementary intraoperative epidural anesthesia. Effective or ineffective responses were determined, followed by a 1 µg increment or decrement in the dose of intrathecal dexmedetomidine for the next parturient using up-down sequential allocation. ED50 were calculated using probit regression. Results: The ED50 of intrathecal dexmedetomidine with plain ropivacaine was 5.9 µg (95% confidence interval [CI], 4.9–7.4 µg) in Group Rop8 and 3.1 µg (95% CI, 0.1–4.8 µg) in Group Rop10 (P < 0.05). Hemodynamic stability, side effects, patient satisfaction and neonatal outcomes were comparable between the two groups. Conclusions: The present data suggested that the ED50 of intrathecal dexmedetomidine as an adjuvant to 8 mg and 10 mg plain ropivacaine in spinal anesthesia during cesarean section was approximately 6 µg and 3 µg, respectively. Trial registration: Chinese Clinical Trial Registry, identifier: ChiCTR2200055928.


4 August 2023
Verified
Conference Paper
Open access
MYMuchao Ye
Muchao Ye
JCJinghui Chen
Jinghui Chen
CMChenglin Miao
Chenglin Miao
HLHan Liu
Han Liu
TWTing Wang
Ting Wang
FMFenglong Ma
Fenglong Ma
Despite a plethora of prior explorations, conducting text adversarial attacks in practical settings is still challenging with the following constraints: black box -- the inner structure of the victim model is unknown; hard label -- the attacker only has access to the top-1 prediction results; and semantic preservation - the perturbation needs to preserve the original semantics. In this paper, we present PAT, a novel adversarial attack method employed under all these constraints. Specifically, PAT explicitly models the adversarial and non-adversarial prototypes and incorporates them to measure semantic changes for replacement selection in the hard-label black-box setting to generate high-quality samples. In each iteration, PAT finds original words that can be replaced back and selects better candidate words for perturbed positions in a geometry-aware manner guided by this estimation, which maximally improves the perturbation construction and minimally impacts the original semantics. Extensive evaluation with benchmark datasets and state-of-the-art models shows that PAT outperforms existing text adversarial attacks in terms of both attack effectiveness and semantic preservation. Moreover, we validate the efficacy of PAT against industry-leading natural language processing platforms in real-world settings.

21 October 2023
Verified
Conference Paper
Open access
HGHangzhi Guo
Hangzhi Guo
FJFeiran Jia
Feiran Jia
JCJinghui Chen
Jinghui Chen
ASAnna Squicciarini
Anna Squicciarini
AYAmulya Yadav
Amulya Yadav
Counterfactual (CF) explanations for machine learning (ML) models are preferred by end-users, as they explain the predictions of ML models by providing a recourse (or contrastive) case to individuals who are adversely impacted by predicted outcomes. Existing CF explanation methods generate recourses under the assumption that the underlying target ML model remains stationary over time. However, due to commonly occurring distributional shifts in training data, ML models constantly get updated in practice, which might render previously generated recourses invalid and diminish end-users trust in our algorithmic framework. To address this problem, we propose RoCourseNet, a training framework that jointly optimizes predictions and recourses that are robust to future data shifts. This work contains four key contributions: (1) We formulate the robust recourse generation problem as a tri-level optimization problem which consists of two sub-problems: (i) a bi-level problem that finds the worst-case adversarial shift in the training data, and (ii) an outer minimization problem to generate robust recourses against this worst-case shift. (2) We leverage adversarial training to solve this tri-level optimization problem by: (i) proposing a novel virtual data shift (VDS) algorithm to find worst-case shifted ML models via explicitly considering the worst-case data shift in the training dataset, and (ii) a block-wise coordinate descent procedure to optimize for prediction and corresponding robust recourses. (3) We evaluate RoCourseNet's performance on three real-world datasets, and show that RoCourseNet consistently achieves more than 96% robust validity and outperforms state-of-the-art baselines by at least 10% in generating robust CF explanations. (4) Finally, we generalize the RoCourseNet framework to accommodate any parametric post-hoc methods for improving robust validity.