Benchmarking Inpatient Mortality Using Electronic Medical Record Data: A Retrospective, Multicenter Analytical Observational Study

Author Department

Pulmonary/Critical Care Medicine

Document Type

Article, Peer-reviewed

Publication Date

9-2021

Abstract

Objectives: To develop a model to benchmark mortality in hospitalized patients using accessible electronic medical record data.

Design: Univariate analysis and multivariable logistic regression were used to identify variables collected during the first 24 hours following admission to test for risk factors associated with the end point of hospital mortality. Models were built using specific diagnosis (International Classification of Diseases, 9th Edition or International Classification of Diseases, 10th Edition) captured at discharge, rather than admission diagnosis, which may be discordant. Variables were selected based, in part, on prior the Acute Physiology and Chronic Health Evaluation methodology and included primary diagnosis information plus three aggregated indices (physiology, comorbidity, and support). A Physiology Index was created using parsimonious nonlinear modeling of heart rate, mean arterial pressure, temperature, respiratory rate, hematocrit, platelet counts, and serum sodium. A Comorbidity Index incorporates new or ongoing diagnoses captured by the electronic medical record during the preceding year. A Support Index considered 10 interventions such as mechanical ventilation, selected IV drugs, and hemodialysis. Accuracy was determined using area under the receiver operating curve for discrimination, calibration curves, and modified Brier score for calibration.

Setting and patients: We used deidentified electronic medical record data from 74,434 adult inpatients (ICU and ward) at 15 hospitals from 2010 to 2013 to develop the mortality model and validated using data for additional 49,752 patients from the same 15 hospitals. A second revalidation was accomplished using data on 83,684 patients receiving care at six hospitals between 2014 and 2016. The model was also validated on a subset of patients with an ICU stay on day 1.

Interventions: None.

Measurements and main results: This model uses physiology, comorbidity, and support indices, primary diagnosis, age, lowest Glasgow Coma Score, and elapsed time since hospital admission to predict hospital mortality. In the initial validation cohort, observed mortality was 4.04% versus predicted mortality 4.12% (Student t test, p = 0.37). In the revalidation using a different set of hospitals, predicted and observed mortality were 2.66% and 2.99%, respectively. Area under the receiver operating curve were 0.902 (0.895-0.909) and 0.884 (0.877-0.891), respectively, and calibration curves show a close relationship of observed and predicted mortalities. In the evaluation of the subset of ICU patients on day1, the area under the receiver operating curve was 0.87, with an observed mortality of 8.78% versus predicted mortality of 8.93% (Student t test, p = 0.52) and a standardized mortality ratio of 0.98 (0.932-1.034).

Conclusions: Variables considered by traditional ICU prognostic models accurately benchmark patient mortality for patients receiving care in multiple hospital locations, not only the ICU. Unlike Acute Physiology and Chronic Health Evaluation, this model relies on electronic medical record data alone and does not require personnel to collect the independent predictor variables. Assessing the model's utility for benchmarking hospital performance will require prospective testing in a larger representative sample of hospitals.

PMID

34582424

Link to Full Text

Share

COinS