Introduction: To systematically review studies evaluating the performance of Sequential Organ Failure Assessment (SOFA)-based models for predicting mortality in patients in the intensive care unit (ICU). Methods: Medline, EMBASE and other databases were searched for English-language articles with the major objective of evaluating the prognostic performance of SOFA-based models in predicting mortality in surgical and/or medical ICU admissions. The quality of each study was assessed based on a quality framework for prognostic models. Results: Eighteen articles met all inclusion criteria. The studies differed widely in the SOFA derivatives used and in their methods of evaluation. Ten studies reported about developing a probabilistic prognostic model, only five of which used an independent validation data set. The other studies used the SOFA-based score directly to discriminate between survivors and non-survivors without fitting a probabilistic model. In five of the six studies, admission-based models (Acute Physiology and Chronic Health Evaluation (APACHE) II/III) were reported to have a slightly better discrimination ability than SOFA-based models at admission (the receiver operating characteristic curve (AUC) of SOFA-based models ranged between 0.61 and 0.88), and in one study a SOFA model had higher AUC than the Simplified Acute Physiology Score (SAPS) II model. Four of these studies used the Hosmer-Lemeshow tests for calibration, none of which reported a lack of fit for the SOFA models. Models based on sequential SOFA scores were described in 11 studies including maximum SOFA scores and maximum sum of individual components of the SOFA score (AUC range: 0.69 to 0.92) and delta SOFA (AUC range: 0.51 to 0.83). Studies comparing SOFA with other organ failure scores did not consistently show superiority of one scoring system to another. Four studies combined SOFA-based derivatives with admission severity of illness scores, and they all reported on improved predictions for the combination. Quality of studies ranged from 11.5 to 19.5 points on a 20-point scale. Conclusions: Models based on SOFA scores at admission had only slightly worse performance than APACHE II/III and were competitive with SAPS II models in predicting mortality in patients in the general medical and/or surgical ICU. Models with sequential SOFA scores seem to have a comparable performance with other organ failure scores. The combination of sequential SOFA derivatives with APACHE II/III and SAPS II models clearly improved prognostic performance of either model alone. Due to the heterogeneity of the studies, it is impossible to draw general conclusions on the optimal mathematical model and optimal derivatives of SOFA scores. Future studies should use a standard evaluation methodology with a standard set of outcome measures covering discrimination, calibration and accuracy.

Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet

Introduction: Advanced age is thought to be associated with increased mortality in critically ill patients. This report reviews available data on factors that determine outcome, on the value of prognostic models, and on preferences regarding life-sustaining treatments in (very) elderly intensive care unit (ICU) patients. Methods: We searched the Medline database (January 1966 to January 2005) for English language articles. Selected articles were cross-checked for other relevant publications. Results: Mortality rates are higher in elderly ICU patients than in younger patients. However, it is not age per se but associated factors, such as severity of illness and premorbid functional status, that appear to be responsible for the poorer prognosis. Patients' preferences regarding life-sustaining treatments are importantly influenced by the likelihood of a beneficial outcome. Commonly used prognostic models have not been calibrated for use in the very elderly. Furthermore, they do not address long-term survival and functional outcome. Conclusion: We advocate the development of new prognostic models, validated in elderly ICU patients, that predict not only survival but also functional and cognitive status after discharge. Such a model may support informed decision making with respect to patients' preferences.

Introduction: Arterial concentrations of carbon dioxide (PaCO₂) and oxygen (PaO₂) during admission to the intensive care unit (ICU) may substantially affect organ perfusion and outcome after cardiac arrest. Our aim was to investigate the independent and synergistic effects of both parameters on hospital mortality.Methods: This was a cohort study using data from mechanically ventilated cardiac arrest patients in the Dutch National Intensive Care Evaluation (NICE) registry between 2007 and 2012. PaCO₂and PaO₂levels from arterial blood gas analyses corresponding to the worst oxygenation in the first 24 h of ICU stay were retrieved for analyses. Logistic regression analyses were performed to assess the relationship between hospital mortality and both categorized groups and a spline-based transformation of the continuous values of PaCO₂and PaO₂.Results: In total, 5,258 cardiac arrest patients admitted to 82 ICUs in the Netherlands were included. In the first 24 h of ICU admission, hypocapnia was encountered in 22 %, and hypercapnia in 35 % of included cases. Hypoxia and hyperoxia were observed in 8 % and 3 % of the patients, respectively. Both PaCO₂and PaO₂had an independent U-shaped relationship with hospital mortality and after adjustment for confounders, hypocapnia and hypoxia were significant predictors of hospital mortality: OR 1.37 (95 % CI 1.17–1.61) and OR 1.34 (95 % CI 1.08–1.66). A synergistic effect of concurrent derangements of PaCO₂and PaO₂was not observed (P = 0.75).Conclusions: The effects of aberrant arterial carbon dioxide and arterial oxygen concentrations were independently but not synergistically associated with hospital mortality after cardiac arrest.

Introduction: The Risk, Injury, Failure, Loss, and End-Stage Renal Disease (RIFLE) is a consensus-based classification system for diagnosing acute kidney insufficiency (AKI), based on serum creatinine (SCr) and urine output criteria (RIFLE_SCr+UO). The urine output criteria, however, are frequently discarded and many studies in the literature applied only the SCr criteria (RIFLE_SCr). We diagnosed AKI using both RIFLE methods and compared the effects on time to AKI diagnosis, AKI incidence and AKI severity. Methods: This was a prospective observational cohort study during four months in adult critically ill patients admitted to the ICU for at least 48 hours. During the first week patients were scored daily for AKI according to RIFLE_SCr+UO and RIFLE_SCr. We assessed urine output hourly and fluid balance daily. The baseline SCr was estimated if a recent pre-ICU admission SCr was unknown. Based on the two RIFLE methods for each patient we determined time to AKI diagnosis (AKI-0) and maximum RIFLE grade. Results: We studied 260 patients. A pre-ICU admission SCr was available in 101 (39%) patients. The two RIFLE methods resulted in statistically significantly different outcomes for incidence of AKI, diagnosis of AKI for individual patients, distribution of AKI-0 and distribution of the maximum RIFLE grade. Discarding the RIFLE urine criteria for AKI diagnosis significantly underestimated the presence and grade of AKI on admission and during the first ICU week (P < 0,001) and significantly delayed the diagnosis of AKI (P < 0.001). Based on RIFLE_SCr 45 patients had no AKI on admission but subsequently developed AKI. In 24 of these patients (53%) AKI would have been diagnosed at least one day earlier if the RIFLE urine criteria had been applied. Mortality rate in the AKI population was 38% based on RIFLE_SCr and 24% based on RIFLE_SCr+UO (P = 0.02). Conclusions: The use of RIFLE without the urine criteria significantly underscores the incidence and grade of AKI, significantly delays the diagnosis of AKI and is associated with higher mortality.

Background: Free-text descriptions in electronic health records (EHRs) can be of interest for clinical research and care optimization. However, free text cannot be readily interpreted by a computer and, therefore, has limited value. Natural Language Processing (NLP) algorithms can make free text machine-interpretable by attaching ontology concepts to it. However, implementations of NLP algorithms are not evaluated consistently. Therefore, the objective of this study was to review the current methods used for developing and evaluating NLP algorithms that map clinical text fragments onto ontology concepts. To standardize the evaluation of algorithms and reduce heterogeneity between studies, we propose a list of recommendations. Methods: Two reviewers examined publications indexed by Scopus, IEEE, MEDLINE, EMBASE, the ACM Digital Library, and the ACL Anthology. Publications reporting on NLP for mapping clinical text from EHRs to ontology concepts were included. Year, country, setting, objective, evaluation and validation methods, NLP algorithms, terminology systems, dataset size and language, performance measures, reference standard, generalizability, operational use, and source code availability were extracted. The studies’ objectives were categorized by way of induction. These results were used to define recommendations. Results: Two thousand three hundred fifty five unique studies were identified. Two hundred fifty six studies reported on the development of NLP algorithms for mapping free text to ontology concepts. Seventy-seven described development and evaluation. Twenty-two studies did not perform a validation on unseen data and 68 studies did not perform external validation. Of 23 studies that claimed that their algorithm was generalizable, 5 tested this by external validation. A list of sixteen recommendations regarding the usage of NLP systems and algorithms, usage of data, evaluation and validation, presentation of results, and generalizability of results was developed. Conclusion: We found many heterogeneous approaches to the reporting on the development and evaluation of NLP algorithms that map clinical text to ontology concepts. Over one-fourth of the identified publications did not perform an evaluation. In addition, over one-fourth of the included studies did not perform a validation, and 88% did not perform external validation. We believe that our recommendations, alongside an existing reporting standard, will increase the reproducibility and reusability of future studies and NLP algorithms in medicine.