Tim Buckman. My opinions expressed in this talk are personal and may not reflect the position of the journals or of the Society of Critical Care Medicine. My opinions do not reflect positions of the United States government or agencies thereof. My assigned question, did data science say they are septic? We'll approach the question by breaking it down word by word. So let's start here. Do data science say they were septic? Unfortunately, contemporary definitions of sepsis are state-centric, subjective, and circular. Now, it started with sepsis one, Roger Bone's definition that saw sepsis as the combination of inflammation, the presence of the inflammatory response syndrome, with infection. Sepsis two was a refinement of Bone's definition. It was collective opinion and it enumerated organ failures. And you've just heard about sepsis three derived by an international expert panel in the prior talk from Dr. Arrington. But in all of these, we're talking about states, not diagnoses, and it's very subjective. It's a suspected, not proven infectious state. And in fact, epidemiologic studies always require adjudication by a panel of experts to decide the patient called septic really was. And then there's the issue of ordering the treatment that is prescribing antibiotics. That's necessary, but it's also sufficient to assert that the patient is septic and apply the diagnosis. And yet half of all sepsis cases are pathogen unidentified. In a paper published in Critical Care Medicine in February of 2020, Dr. Sarian Henry pointed out that there were too many definitions of sepsis and expressed the hope that machine learning might leverage the electronic health record to increase accuracy and bring consensus. They enumerated and then annotated the many definitions of sepsis, pointing out that relying on Billy codes or the SEP1 metric, sepsis three, or the electronic adult sepsis event, or any EHR-based sepsis phenotyping, these are all retrospective definitions. Real-time definitions are far fewer and they are not terribly precise. Well, let's move on. Did data science say they were septic? In fact, the current definition depends on the capacity to mount a response. That life thread embedded in sepsis three means that a uniform inoculum of a known pathogen into a defined space may or may not cause sepsis, depending on the host. This further means that while Koch's postulates can support causal inference between a pathogen and an infection, there's always going to be a probabilistic aspect to inferring causal inference. Between a pathogen and sepsis, and thus applying the sepsis label depends at least in part on who they are and our clinical assessment of the patient's capacity to defend against the threat to life. As Walt Kelly's Pogo put it, we've met the enemy and he is us. This patient has erysipelas. You can see the redness and the swelling in the patient's left leg. Is it sepsis? A glance at the data shows that it is. Fever, hypotension, leukocytosis, creatinine more than double baseline, bilirubin more than double baseline, lactic acidosis, SOFA score of four. Yes, this meets sepsis three criteria. Let's continue with the question. Did data science say they were septic? Well, the scientific method says that we should start with an observation or a question, research the topic area, form a hypothesis, test that hypothesis with experiment, analyze data, and then report the conclusions. But that doesn't quite capture what we're trying to do at the best. The scientific method around data analytics can be divided into three phases, a pre-analytic phase, an analytic phase, and in a post-analytic phase. In the pre-analytic phase, the user identifies a need. Some specific analytic model is selected. The analysis is requested and data are collected. In the analytic phase, data are prepared for the analysis. Data are analyzed. The output is verified that we've got the right context, the right patient, the right units. And in the post-analytic phase, an output is reported. And that output is interpreted in the context of a clinical scenario. Then an action is taken and patient care is somehow affected. This is what data science is trying to do in the framework of sepsis. Model specification, the key step in the pre-analytic phase, has seen enormous progress over the past five years. For example, in 2019, Chris Seymour and his colleagues described the derivation, validation, and even potential treatment implications of novel clinical phenotypes for sepsis. By scoring laboratory and physical data by scoring laboratory and physiological parameters in a variety of organ systems shown in the diagram in the center, they were able to extract four different phenotypes based on comparisons among several dozen of these laboratory and physiologic parameters and described the clinical courses of these endotypes as being somewhat different. The clinical application of these analytic frameworks has been somewhat less successful. For example, last year, Wang and colleagues performed an external validation of a widely implemented proprietary sepsis prediction model, the EPIC sepsis model in hospitalized patients. In the center of the frame, you can see the sensitivity and specificity curves based on different thresholds for the EPIC sepsis model. The finding was that despite generating sepsis alerts on 18% of all patients in their academic health science center, the EPIC sepsis model did not detect sepsis in two thirds of the patients who had sepsis. The positive predictive value was only 12%. The negative predictive value was 95%. And overall, the area under the receiver operator curve performed at only 0.63, not a terribly effective predictive model. Now, the post-analytic phase depends entirely on a clinician acting on the information and doing something to change patient care. In this paper reported from the University of Pennsylvania group in critical care medicine in November of 2019, they discussed clinician perception of a machine learning based early warning system designed to detect and predict severe sepsis and septic shock. Now, this was a very good algorithm, at least for the time. The time the algorithm fired, 11% of the patients were already septic. And within 48 hours, fully 30% had developed severe sepsis and septic shock. How do the clinicians receive this actionable information? Well, as you can see in the graph, neither the doctors nor the nurses were terribly enamored of the alerting system. In fact, only 11 to 12% of the providers said that the alert improved their care. And among the nurses, only about a third of them said that the alert improved their care. In contrast, more than half the providers and between a quarter and a third of the nurses said the alert was intrusive and didn't improve their care. So what's the problem? If the alert works, why aren't clinicians more enamored of these sepsis alerts? The answer is to be found in an examination of the sepsis patient continuum. The time to intervene obviously is at the point of infection through prevention or early treatment. Sepsis patients tend to present late. In fact, among Medicare beneficiaries, 93% of all sepsis patients are actually septic at the time they reach the emergency department. And that number is rising. But our contemporary definitions of sepsis, that is sepsis three and the adult sepsis event, those don't get called out until organ dysfunction has already developed. And it's pretty obvious to those who are at the bedside that the patient is either acutely ill or about to become acutely ill. So what's the problem? Even if we have a reasonably performing analytic model and a predictive algorithm, it feels late to clinicians. We wanna prevent infections or if we can't prevent them, at least detect them immediately when they occur or at the latest at the earliest physical signs. We don't wanna wait for the last critical leverage point, that is organ dysfunction, to be told that the patient is septic and something needs to be done. Timing's wrong. Did data science say they were septic? Our algorithms are only as good as the data available. And there are at least 13 categories of structured patient data stored by modern health systems that are relevant to machine learning. Unfortunately, the difficulty of extracting each type can vary quite substantially. Those types shown in green are easy to obtain. Minimal clinical experience is needed to interpret them and not much validation is required. Those shown in yellow, clinical experience is often needed to interpret them and there are often errors either in the creation of the data or their extraction. And finally, those shown in red, which can be very important, such as the chest X-ray and other radiograph images and the waveform data that come across our monitors, those may not even be available to our algorithms without custom software to facilitate storage and process the data for use in the algorithm. And it's not just a question of getting more data, more data types, more features, more elements. It's necessary to find and sort and integrate and process the data so that it's transformed into actionable information. More data does not equal more information. Typically, it's less. Much of the time, we don't need a data model or a decision tree or an algorithm. We simply need better visualizations of the existing data so that our human brains can readily detect the trends that aren't apparent in spreadsheets or other handwritten note form. Did the data scientist say they were septic? What exactly is a data scientist? Well, first, it's someone who has subject matter expertise. For sepsis, it needs to be someone who understands the biology, the physiology, and the care. That person has to have computer hacking skills, able to manipulate large sets of often complex and differently structured data, and use math and statistics, knowledge and methodology, including machine learning to create models that are going to be useful to clinicians and rigorous in their science. Now, the workflow to create a predictive analytic is complex. It proceeds from specification of the data needs through data extraction, validation of the raw data, often an iterative process, to initial model development, identification and correction of incorrect or missing data, final model development, and then validation of the predictive model itself. The success of this process depends on the tight integration of three individuals, a clinician who will identify clinically relevant data for the model, the data extractor or the data steward who can describe the capabilities of the data and what the limitations are going to be on the request, and the data scientist who will eventually build the model but has to articulate the preferred and necessary data format. There are several steps that are often omitted, such as creation of a data dictionary and prioritizing and tasking among many different requests. And all too commonly, the researcher either wants too much of the model or doesn't actually know what they want the model to do, or one of the participants of the team is not fully involved. We have to ask, who is asking the question about what data science can contribute? Different constituencies have different reasons for asking, is this patient septic? The public health official or clinical epidemiologist is trying to create a large picture, not care for an individual patient. A quality assurance or quality improvement officer is trying to make sure that no patient has slipped through. The inventor of a diagnostic cares about specificity and sensitivity. The clinical trialist wants to make sure only the authentic patients eligible for the trial actually receive the test article. The journal editor wants to know that what's being published actually represents a cohesive cohort of patients. The clinician just wants to do the right thing. And the patient, well, the patient's interested in getting better. Unfortunately, I've had most of those roles as a journal editor, as a clinician trying to care for hundreds of patients, and yes, as a sepsis patient. I did have sepsis according to sepsis three. The continuum does matter. We in critical care tend to focus our attention out here. The patient who comes to the unit already with organ dysfunction, perhaps with a development of shock, but our opportunities to intervene, our best opportunities to intervene, occur much earlier in the course of sepsis. And this is where data science probably has a chance to make an even greater impact to figure out who is at risk and get the immunizations done early, to figure out where patients are likely to become exposed to pathogens, to identify at the earliest possible moment a departure from physiologic normal that might represent the onset of the sepsis phenotype. Those are the opportunities where we have a chance individually and collectively to make the biggest dent in sepsis. All of that prior to the patient coming to the ICU. What we don't want to do is mismatch the data science with the clinical setting. So take somebody who's just run up a flight of stairs. Your heart rate was 101 and I'm clocking your respirations at about 24 per minute. Well, that's probably because I was just... You're triggering the sepsis warning. I'm going to need to get the doctor. Doctor? Sepsis warning, what does that mean? You probably have a serious infection. Infection? Your rapid heart rate and rapid breathing triggers SIRS and then your shortness of breath gives us a likely source, pneumonia. I don't have pneumonia. Hey, bud, why don't you leave the doctor into the doctors, okay? I feel totally fine. Good thing we have the sepsis watch to keep you safe then. Want me to get some blood work, doctor? Oh, they can do that in the ICU. I'm admitting him. Excuse me, what? Computers don't lie, pal. Computers don't lie, but what they say has to be interpreted in the correct context. What will be done with the answer? This is perhaps the question that should have been posed first. Are we benchmarking? Are we predicting? Are we making a revocable clinical decision like starting antibiotics? Are we making an irrevocable clinical decision like taking the patient to the operating room? We've seen that data science can provide more than one answer and the utility of the answer depends on who wants to know, what they want to know, and why they want to know. Thank you.

data science

sepsis diagnosis

limitations of current definitions

electronic health records

machine learning

Sepsis

Quality and Patient Safety

Sepsis

Evidence Based Medicine

Presentation

Select

2022

Intermediate

Advanced

Sepsis Resources