News: Roughly 10% discrepancy in software sepsis prediction model, JAMA says

Epic claims its sepsis prediction model sorts patients’ sepsis risk correctly 76% of the time, however, a study published in the Journal of the American Medical Association (JAMA) Internal Medicine, found that it only sorts patients correctly 63% of the time.

The retrospective study was conducted among 27,697 patients 18-years and older with 38,455 hospitalizations between December 6, 2018, and October 20, 2019. The definition of sepsis used for the study was a composite of the CDC surveillance criteria, and the ICD-10-CM diagnostic codes accompanied by two systemic inflammatory response syndrome criteria and one organ dysfunction criterion within six hours of another.

Of the patients, the study found that sepsis occurred in 2,552 patients, or 7%. The Epic Sepsis Model missed identifying 1,709 patients with sepsis, despite generating alerts for an Epic Sepsis Model score of six or higher for 6,971 of all 38,455 hospitalized patients, thus creating a large burden of alert fatigue.

The JAMA Internal Medicine study suggests that the Epic Sepsis Model has “poor discrimination and calibration in predicting the onset of sepsis. The widespread adoption of the [Epic Sepsis Model] despite its poor performance raises fundamental concerns about sepsis management on a national level.”

Editor’s note: The JAMA Internal Medicine study can be found here.