Publications

Importance

Accurate prediction of outcomes among patients in intensive care units (ICUs) is important for clinical research and monitoring care quality. Most existing prediction models do not take full advantage of the electronic health record, using only the single worst value of laboratory tests and vital signs and largely ignoring information present in free-text notes. Whether capturing more of the available data and applying machine learning and natural language processing (NLP) can improve and automate the prediction of outcomes among patients in the ICU remains unknown.

Objectives

To evaluate the change in power for a mortality prediction model among patients in the ICU achieved by incorporating measures of clinical trajectory together with NLP of clinical text and to assess the generalizability of this approach.

Design, Setting, and Participants

This retrospective cohort study included 101 196 patients with a first-time admission to the ICU and a length of stay of at least 4 hours. Twenty ICUs at 2 academic medical centers (University of California, San Francisco [UCSF], and Beth Israel Deaconess Medical Center [BIDMC], Boston, Massachusetts) and 1 community hospital (Mills-Peninsula Medical Center [MPMC], Burlingame, California) contributed data from January 1, 2001, through June 1, 2017. Data were analyzed from July 1, 2017, through August 1, 2018.

Main Outcomes and Measures

In-hospital mortality and model discrimination as assessed by the area under the receiver operating characteristic curve (AUC) and model calibration as assessed by the modified Hosmer-Lemeshow statistic.

Results

Among 101 196 patients included in the analysis, 51.3% (n = 51 899) were male, with a mean (SD) age of 61.3 (17.1) years; their in-hospital mortality rate was 10.4% (n = 10 505). A baseline model using only the highest and lowest observed values for each laboratory test result or vital sign achieved a cross-validated AUC of 0.831 (95% CI, 0.830-0.832). In contrast, that model augmented with measures of clinical trajectory achieved an AUC of 0.899 (95% CI, 0.896-0.902; P < .001 for AUC difference). Further augmenting this model with NLP-derived terms associated with mortality further increased the AUC to 0.922 (95% CI, 0.916-0.924; P < .001). These NLP-derived terms were associated with improved model performance even when applied across sites (AUC difference for UCSF: 0.077 to 0.021; AUC difference for MPMC: 0.071 to 0.051; AUC difference for BIDMC: 0.035 to 0.043; P < .001) when augmenting with NLP at each site.

Conclusions and Relevance

Intensive care unit mortality prediction models incorporating measures of clinical trajectory and NLP-derived terms yielded excellent predictive performance and generalized well in this sample of hospitals. The role of these automated algorithms, particularly those using unstructured data from notes and other sources, in clinical research and quality improvement seems to merit additional investigation.

STUDY OBJECTIVE

Emergency department (ED) visits for syncope are common and routine diagnostic testing is frequently low yield. Our objective is to determine whether recent guidelines emphasizing limiting hospitalization and advanced diagnostic testing to high-risk patients have changed patterns of syncope care.

METHODS

This was a retrospective population epidemiology study of syncope-related ED visits and hospitalizations using the National Emergency Department Sample from 2006 to 2014 and the State Inpatient Databases and Emergency Department Databases from 2009 and 2013. Primary outcomes were annual incidence rates of syncope ED visits and subsequent hospitalizations, and rates of hospitalization, observation, 30-day revisits, and diagnostic testing comparing 2009 with 2013. Differences were estimated with multivariable logistic regression modeling adjusted for patient clinical and demographic characteristics.

RESULTS

From 2006 to 2014, we identified 15,154,920 survey-weighted ED visits for syncope. Annual rates of ED visits increased from 643 to 771 per 100,000 adults, whereas hospitalizations declined from 36.3% to 24.7% (-11.6% absolute difference; 95% confidence interval [CI] -13.0% to -10.2%). In multistate adjusted analyses, the proportion of ED visits resulting in hospital admission decreased 11.7% (95% CI -11.9% to -11.6%) between 2009 and 2013, whereas the proportion of ED visits resulting in observation care increased by 7.9% (95% CI 7.8% to 8.0%), with no significant change in 30-day ED revisit rates (absolute difference 0.1%; 95% CI -0.1% to 0.3%). The frequency of advanced cardiac testing increased from 13.8% to 17.0%, and neuroimaging increased from 40.6% to 44.3%, driven by increased testing of patients receiving observation and inpatient care.

CONCLUSION

Although the incidence of ED visits for syncope has increased, hospitalization rates have declined, without an adverse effect on ED revisits, possibly because of increased use of observation care. Use of advanced cardiac testing and neuroimaging has increased, driven by growth in testing of patients receiving observation and inpatient care.