Hello and welcome to today's Journal Club Critical Care Medicine webcast. This webcast, hosted and supported by the Society of Critical Care Medicine, is part of the Journal Club Critical Care Medicine series. This webcast features two articles that appear in the March 2022 issue of Critical Care Medicine. This webcast is being recorded. The recording will be available to registrants on demand within five business days. Log in to mysccm.org and navigate to the My Learning tab. My name is Thomas Zagmani and I'm a Professor of Intensive Care at Cardiff University in the United Kingdom. I will be moderating today's webcast. Thanks for joining us. Just a few housekeeping items before we get started. There will be a Q&A session at the conclusion of both presentations. To submit questions throughout the presentation, type into the question box located on your control panel. If you have a comment to share during the presentations, you may also use the question box for that. And finally, everyone joining us for today's webcast will receive a follow-up email that will include an evaluation. Please take five minutes to complete it. Your feedback is greatly appreciated. Please note that today's presentation's content is for educational purposes only. Please read the disclaimer for me. And now I would like to introduce today's presenters. Dr. Yashir Taravichy is a practicing pulmonologist, intensivist, and clinical informaticist at the MetroHealth Cleveland, Ohio. He's an Assistant Professor of Medicine at the Case Western Reserve University School of Medicine. As Director of Clinical Research Informatics, he's dedicated to leveraging developments in health information systems and data science to advance the continued evolution of a learning health system he serves. Dr. Stephanie Taylor is an Associate Professor of Internal Medicine at Atrium Health Carolina's Medical Center in Charlotte, North Carolina. Dr. Taylor's research interests focus on applying health services methods and big data analytics to granular clinical data to advance the learning hospital system and improve outcomes for patients with sepsis. Dr. Taylor leads an R01-funded dissemination and implementation study evaluating the effects of a sepsis transition and recovery program on reducing readmissions and mortality among sepsis survivors. Thank you both for joining us today. I will now turn the presentation over to Dr. Taravichy. Hi. Thank you for having me, and thank you for allowing us to share our work with you in the setting. So I'm going to speak about the article here, Improving Timeliness of Antibiotic Administration Using a Provider and Pharmacist-Facing Sepsis Early Warning System in the Emergency Department Setting. As far as disclosures go, I do receive some consulting and funding fees from Beckman Coulter, but that's unrelated to this work. And it's worth noting that this study was designed and implemented after the introduction of our sepsis early warning system, and the vendor who created the early warning system had no role in the study beyond the provision of supporting information on the early warning system. So, oh, sorry about that. The outline for the discussion I'd like to generate today is on the slide, and I'm going to start with general notions of sepsis and how we might be able to do better in our health care system. And I think it requires no introduction, but the tension that we face when we're dealing with sepsis in the health care system is usually the recognition that early antibiotics are good as far as early intervention leading to better outcomes. And I think that most of the data we recognize is rather robust, but it tends to be in hindsight, kind of more of a pre-post setting. So that's less of a debate. The problem with being aggressive about early antibiotics is sepsis is ill-defined, and we've got historical precedent where forcing physicians to act faster can lead to unanticipated consequences. And I refer you to the issues that developed when we rushed ED clinicians to improve time to antibiotics or time from arrival to antibiotics in pneumonia as a quality measure back in 2007, which led to an increase in misdiagnoses and antibiotic overuse, as an example. Talking more about Metro Health in specific, what we can do better, well, we can always do better as far as sepsis goes. The admission is that we were not achieving one-hour sepsis response times, which I realize now with the updated guidelines is more controversial. And I think most are not what it's worth. We did not have a standardized team-based response to sepsis, and that's something that I think is considered more of a standard of care at this point. And our stakeholders, as a result of that, were not always aware of which patients to prioritize. And when I say stakeholders, I'm referring to the rest of the care team outside the clinical group, the physician group, I'm sorry, including nurses, pharmacists that might actually have a role in the response to sepsis. And so now the question arises, well, how can clinical informatics and predictive analytics help? So obviously there's a large degree of complexity in the data involved for the diagnosis and the prediction of sepsis and or a bad outcome in the context of infection. And that lends itself to a more sophisticated data science approaches. With the drive for earlier automated detection, we've got a significant development of rule-based sepsis screening tools and prediction-based early warning systems. And our focus with this study is on the lab. Most of the data that supports the use of such systems tend to be in the setting, in the form of a pre-post-intervention study. And they've shown at least historical improvements in mortality, time to antibiotics, and rates of sepsis-vulnerable compliance. So with that, we turned to our EHR vendor, which was Epic at the time, and we looked at their sepsis early warning system solution. Their solution is derived and externally validated. It's a penalized logistic regression that incorporates several structured EHR variables, demographics, vital signs, lab studies, diagnoses, and procedures. And the way that they designed or at least designed their early warning system, it relied on a gold standard diagnosis of sepsis, which is either a sepsis-specific order or a flowsheet completion. They set the prediction, as far as the assessment goes, to be six hours before that occurred. They validated it in a rather large, derived and validated it in a rather large set of charts, 405,000 encounters, a pretty decent size. And as you may already know, it's leveraged by, as far as we can tell at the time, over 100 institutions. Problem being, any of the data that suggested its efficacy tended to be little news clippings on the internet or kind of by report during interviews. There was nothing really in the literature to tell us what the actual efficacy and utility of such a system would be. So the way we approached the implementation, as we did what most reasonable learning health systems would do, is we brought in all the stakeholders, we formed a committee. We had great representation from quality, infectious disease pharmacy, emergency medicine, and a big showing from clinical informatics, which includes specialties like clinical decision support systems. We decided to go ahead and turn it on. And so the whole point of this was to see in the background what this was doing. So none of this was actually shown anywhere in the electronic health record. We actually left it on for a significant amount of time, nine months. It was somewhat planned, somewhat kind of a little bit of a snooze in terms of getting back to it when things got busy or elsewhere. And we monitored how often it triggered during the day. We studied the impact of the thresholds as suggested by the vendor, which is a 5% risk of sepsis based on their definition. And we did that over 1,600 ED encounters. We saw that on average it flagged less than 12 times per day. And I should say we are a level one trauma center with a pretty safety net healthcare system with a pretty busy ED. So that gives you a sense for kind of how things go, how active we are. You know, with that in mind, knowing that there are a large number of patients and physicians in the ED throughout the day, we thought that 12 times wasn't really that bad actually. Now, what we wanted to do for the validation piece, we didn't want to just look at diagnoses, sepsis specific orders or diagnoses of sepsis. We wanted to focus on clinically meaningful outcomes. And so we actually leveraged the sepsis three definition of infection, which is culture sampling followed by antibiotic administration or vice versa within a certain timeframe. So for the threshold level of five, we found that the sensitivity to predict that outcome, which is infection, which is what I've done, sepsis three definition of infection, and three day ICU stay or death, the sensitivity for that outcome was 89.5%, and the specificity was 68.4%. The positive predictive value in the ED specifically was 27%, negative predictive value 98%. And the interesting thing is we found that the alert actually fired slightly more than half the time before antibiotics were administered. And then we kind of regarded that as potentially a lead time opportunity. The figure on the right hand side is one of our internal validation plots that just gives you a sense for how we were kind of marching this out over time. And it's showing you the proportion of patients that ended up flagging based on a higher score of five or greater, and how many of those ended up in the ICU or dead. So it's consistent with the numbers that I quoted earlier. So the way we kind of move forward as well, we think that this is potentially useful. We think that the sensitivity and specificity at least compared to historical other well established systems. And we saw with the lead time opportunity, we thought that was probably a good idea. But based on these favorable characteristics, we're interested in implementation. We just don't know if it's going to work in our setting. And we fear the unanticipated consequences, such as clinician agitation and burnout, which is we've all struggled through the alarm fatigue issues, right? And the implementation resources we had at the time were limited. We had a pharmacist that was going to come in and help out. We didn't know how much time this was really going to take. We thought we wanted a good comparison group. So we're thinking about how we design this intervention. Do we roll it out in half of the emergency room intervention versus standard care? Do we alternate days, which can get incredibly confusing? Or do we do the most robust thing and randomize? And that's what we settled on. As we're thinking through this model, how the sepsis care pathway, we recognize that really, generally speaking, clinicians tend to have an inkling for sepsis when it's happening. There are uncommon situations where, for instance, they're busy in the trauma bay or maybe in a code stroke. And some new results have come back. And a patient might actually look potentially, quote unquote, potentially more septic in the moment. That's a situation where an early warning system like this may be able to prompt an intervention. But we thought, actually, the biggest value was likely to be with the rest of the team, which is the team charged with the actual intervention. And that's what the cogs on the right-hand side are showing you. And these are the points at which the early warning system could be potentially useful, the pieces in the puzzle that it'll touch. So we thought, well, hey, this early warning system could hasten each cog's input. But more importantly, it provides a common rallying point or trigger for a multidisciplinary interaction. And we talked about how we would actually alert and how and whom with all of our stakeholders. And we came up with this swim lane diagram to give folks a sense for what it is that I'm supposed to do when I'm suspecting sepsis. And this was a key thing. This was developed entirely with our stakeholders from the emergency room. We provided, essentially, the technology and the validation and asked them to take charge with how they think that this should be implemented. And we were really keen on making sure that we actually established a standardized response to sepsis, regardless of whether or not the score flagged, and made sure to educate clinicians about the performance of the model and what we expected. So again, whether or not that there was an early warning system score, we expected folks to actually trigger this pathway. So this was kind of our first real stab at a more of a team-based response. And much of what you see in the swim lane diagram is, I would say, standard in keeping with some of the CMS-based response to sepsis. So the question that we posed in the context of our work thus far is, will an EHR-integrated clinician and pharmacist-facing sepsis early warning system improve sepsis-associated process measures, which that was a primary outcome, that was time-to-antibiotics, and then sepsis-associated outcomes, because we wanted to include a clinical measure to see if that was actually going to make an impact. And the one we settled on was days alive and out of hospital, which has become a bit of a recent trend in the literature, and it kind of mirrors a lot of the other critical care outcomes we think about, like ventilator-free days, for instance, or presser-free days. So the way we thought through this implementation was anybody who came into the emergency room would be randomized to either have a flag should they cross the sepsis alert, which we set at five, versus a silently registered alert. Now, and that's the upper part of that pathway. The lower part of that pathway, after that one-to-one randomization based on the last digit of the internal patient identifier, the sepsis alert would lead to, in the augmented care group, an icon on the track board, which you can see on the right-hand side, middle figure, and that kind of looks like footprints, or it's really just two exclamation marks, and a notification directed towards the pharmacist specifically. We wanted to reiterate here, this is a non-interruptive alert, and this was essentially what we negotiated with the care team. We also asked whether or not they wanted an opportunity to provide feedback, and the answer was, don't give us more work. And so this was mostly just informational, and we did not mandate a response. I know that's going to come up later. Just to give you a sense for the timeline, we thought this was going to be a really quick go. You know, the simulation portion took a good amount of time. The planning and building took two quarters, and then the implementation, actually, was relatively quick. We would have published this sooner, but we got hit by COVID, so we ended up diverting our attention elsewhere. The way we kind of structured this was, we had automated reports that captured the data of interest. Since this was a quality improvement initiative, we were able to capture the data of interest since this was a quality improvement initiative, we were planning on iterating actively as we went on. And we wanted the data to be essentially, you know, to come in in somewhat of a stream so we can make decisions on iteration if needed. We had bi-weekly readings with data review, and we had multi-specialty representation at those groups, and blinded chart review when appropriate, actually chart review when any patients died. And so now I'm going to move towards speaking to the outcome of our randomized controlled quality improvement initiative. So, a total of 835 ED encounters triggered a substance early warning system flag. I should say, sorry, that we ran this essentially from August to December with the dates shown in 2019. 237 encounters were excluded, a priori based on discussion with the emergency room, because these were situations where we felt that the score was likely to be ignored. And we ended up with 598 encounters in the analysis, and then 313 randomized to the standard care group and 285 randomized to the augmented care group. These are the demographics of both groups. As you can see, relatively balanced. There were not any major outliers, both in terms of age, sex, race, ethnicity, weight, and also the timing of the alert from admission on the order of hours. And as you can see, and actually the median, these are all median values with interprofile ranges, the alert was actually, for the most part, you know, occurred pretty quickly into someone's admission, or arrival to the emergency room, usually within an hour. Here are our results for the primary process and outcome measures that we selected. This is Figure 2 and 3 in the paper. So the time to antibiotic administration from ED arrival was shorter in the intervention group compared to the standard care group. And what you're seeing there on the left-hand side is the violin plot, which gives you a sense for the distribution of the data, as well as the median and interprofile ranges. And you can see that the curve on the right-hand side with the visible alert group is a little bit kind of shorter and fatter, so you can see that the median is lower, too. It was 2.3 hours versus 3.0 hours, and that was statistically significant at 0.039. Figure 3 shows the primary clinical outcome, which is the days of alive and out of hospital, and we saw that they were greater. And that's, sorry, I should say days alive and out of hospital. You count those over a 30-day period. And the great thing about that measure is it's a bit of a hierarchical measure in the sense that if, you know, somebody stays in the hospital for 30 days or they die, those are similar issues. If somebody gets discharged quickly, for instance, if you're looking at length of stay, when they come back in, those days should go against you. And so days alive and out of hospital is a little more sensitive to readmissions and obviously death. So the median within 30 days, the median in the standard care group was 22.5 versus 24.1 days in the intervention group, also significant at 0.011. Looking at the components that go into that hierarchical construct of days alive and out of hospital, you can see that there are these, you know, I love to call them trends here, but there are non-statistically significant differences in terms of these individual measures like length of stay, hospital mortality, 28-day mortality, and 28-day re-presentation to the ED or hospital, and not just readmission, which account for the differences in the primary clinical outcomes. We also did an a-priority subgroup analysis, and we looked at days alive and out of hospital for situations where the score, the alert actually flagged before an antibiotic was administered, assuming that that would be the situation where the score would be most useful. And we saw significant differences in both the primary process measure and the primary clinical outcome, as you can see. The time to antibiotics looked like they were about an hour apart, as far as the median values go in the subgroup analysis. We were asked to do a couple of more supporting analyses, so we actually looked to see if days alive and out of hospital were negatively correlated to time to antibiotics, and they were, with the Pearson correlation coefficient stated there. We also looked to see whether the intervention group had a shorter time from alert to antibiotic ordering, and they did, 0.6 hours versus 1.4 hours. We also saw differences in time from order placement to antibiotic administration, again, 0.4 hours versus 0.7 hours, also statistically significant. We looked at some of our balancing measures that we decided, you know, we looked at ahead of time. You know, worth stating, I should say before that, is actually there were no differences in the comorbidity scores or day one SOFA scores with our study between the groups. And worth also noting that approximately 40% of those that scored, that flagged the score or an alert at the threshold we selected, ended up in the ICU. There was not a significant difference between both groups. We saw no differences in rates of antibiotic usage, which was a major concern, 67.7% versus 70%. No differences of rates of fluid resuscitation or even relative volume of fluid resuscitation based on weight. C. diff diagnoses were incredibly rare. We saw no differences there. And based on the chart review, no unanticipated events or missed opportunities were noted in either group. So we concluded that patients presenting in the ED who are randomized to a sepsis early warning system, notification that was visible to both pharmacists and clinical staff, had a reduction in time to antibiotics and a modestly greater number of days alive and out of hospital compared to those who had the alert hidden from view. We saw no significant differences in rates of antibiotic use, fluid resuscitation, volume, or C. diff diagnoses. So why do we think that we were able to do what we did? Well, at a high level going out, going into the design phase, we did an internal validation. We didn't rely on what the vendor told us the model did. We got a local positive and negative predictive value that we were able to contrast with established screening mechanisms. And I mentioned QSOFA there, although that's fallen out of favor too. We involved our stakeholders early in the discussion. We allowed them to mold the intervention and essentially to own it. We think that it was really great that the alert was simple and obtrusive and was integrated into an obvious workflow location. And we fought the urge to show numbers. So we were thinking along the lines of a D-dimeralactate where it's positive or not, and you should think about doing something. We also think that we looked out and that pharmacists were really well positioned to be the sepsis response champions in this whole process. And like I said, the post-hoc analysis showed that both the time from presentation to antibiotic order and time from order to administration were significantly hastened. So I think we were able to touch the process at multiple points. Now, this may come up a little bit in the discussion, but why do we think our results differ from others? Well, we limited the scope of the model to the emergency room, which obviously will enrich the positive predictive value, as opposed to, for instance, running this on the floors where you're likely to get a lot more false positives. And we validated our data individually to a different definition of sepsis, one that is more widely accepted and more clinically relevant, again, through the ICU stay or death. And then most importantly, we did not assess the value of the model in isolation. So we didn't just say, how well does the model do on its own in terms of just a validation study in a vacuum, really rather how it actually augmented clinical care. So how does the addition of the system to an already present and responsive clinician affect the management of sepsis? And it is the first prospective randomized control study of such a sepsis early warning system in the ED. The only other randomized control study, at least to date, for a sepsis early warning system was one that was done in an ICU setting. I have these up here just for reference. It's interesting to talk about clinical decision support and how often, unfortunately, we violate that in the world of clinical informatics. And quite a while ago, we've had the Bates 10 Commandments back from 2003. And if you look at that list, a lot of these things are really obvious, right? Speed is everything. Anticipate needs to deliver in real time. Fit something into a workflow. Little things matter as far as usability testing goes. Physicians resist stopping. And that's all those interruptive alerts that really end up going nowhere. You can change someone's direction, which in our cases, maybe if you're thinking about the antibiotic and kicking off the response, go for it. And then simple interventions work best. Ask for information only really when needed. And that's something where, again, our ED providers felt pretty strongly that they didn't want to give feedback in the moment. And then monitor, impact, get feedback, and respond. And that's kind of the iterative nature of the intervention that we did. And with that, I'll stop and say thank you and pass on the slides to my colleague. Thank you. I'll take over now. I'm Dr. Stephanie Taylor, Associate Professor of Internal Medicine at Wake Forest School of Medicine, Atrium Health in Charlotte, North Carolina. I'm going to discuss my article, the Effective and Multicomponent Sepsis Transition and Recovery Program on Mortality and Readmissions After Sepsis, the Impacts Randomized Clinical Trial. This is kind of a cool follow up to Dr. Terabici's work. He talked about how the good work they're doing to improve early management of sepsis, getting those antibiotics in as quickly as possible. And then we're going to switch gears a little bit with our study and talk about once you've done all that and the patient survives sepsis, how can we support their recovery so that they continue to do well and continue to improve after hospital discharge? So why is this study important? We know that sepsis is one of the most common causes of hospitalization worldwide. And through all the good work that people are doing to improve early management of sepsis, patients are surviving sepsis more than ever before. And so we've ended up with many, many sepsis survivors who end up getting discharged from the hospital each year. But unfortunately, we're learning that just surviving that initial infection is not the end of the road for sepsis survivors. Many patients continue to experience persistent problems and things like physical and functional decline, cognitive impairment, mental health problems. They experience recurring infections, whether it's exacerbation of the old infection or a new one, and then flares of their chronic diseases. And all of these problems contribute to really high rates of re-hospitalization after an index sepsis hospitalization. The estimates vary, but somewhere around one in three sepsis survivors will be readmitted to the hospital within 90 days after discharge. And there's some data that indicate that many of these re-hospitalizations are actually preventable. So these are data from Dr. Hallie Prescott and her team. They looked at a group of sepsis survivors who got readmitted to the hospital, and the things that they got readmitted for, 42% of those readmissions were for conditions that were considered ambulatory care sensitive. So conditions that, plausibly, if the patient had been able to get care in the outpatient setting, that condition could have been recognized and treated and have prevented a readmission for that patient. So in addition to high rates of readmission, sepsis survivors also have persistently increased risk of mortality after sepsis. And this study also from Dr. Prescott and a sample of Medicare patients shows that over one in five sepsis survivors in the Medicare population ended up having a late mortality. Even after they survived sepsis, they ended up experiencing mortality after that. And over half of those were considered attributable to the sepsis event. And these data, similar thing looking at late mortality after sepsis, but in a population that's a bit younger, similar thing. So patients who experienced sepsis compared to patients who were hospitalized for other conditions experienced higher rates of mortality that lasted even out to five years after that index hospitalization. So we are learning more and more that sepsis survivors continue to have poor long-term outcomes, but what do we do to improve those outcomes for sepsis survivors? And certainly continuing to focus on best practice care in the early management of sepsis. So work like Dr. Terabici's team and others that are really working on accurate and prompt identification of sepsis and prompt administration of antibiotics, source control, those type of things are really important. It certainly makes sense that the better treatment of early infection will reduce further organ dysfunction and immunomodulation and things that will promote long-term recovery. And then second, things that we can do in the hospital. We're starting to build up a literature of things we can do in the hospital. We're starting to build up a literature of things we can do for patients while they're in the ICU or in the hospital that promote recovery. So things like early mobilization, the whole ADF, A through F bundle, right? So identifying and preventing delirium, engaging families, those types of things that we are beginning to see. And I think the literature is building that we can do those things for our patients while they're in the hospital to improve long-term outcomes. But then the thing that's sort of been a black box up until now is how do we care for patients after they're discharged from the hospital? We know that sepsis survivors are not cured at the time of discharge from the hospital. They're a lot better. They're no longer in shock. They no longer have bacteria coursing through their bloodstream, but they're not recovered fully. Many patients are not recovered fully. And we still have a lot of work to do to support sepsis survivors after they're discharged from the hospital. And thus far, this has been something we really didn't know much about how to do. Kind of recognizing that need to guide post-sepsis care, Dr. Prescott, again, his research has just been really important in advancing sepsis survivorship. She set forth this kind of framework for how we should care for patients after sepsis hospitalization. And the framework included four care elements that are listed here. So one is to screen for common impairments after sepsis. Two is to review and adjust medications. Three is to anticipate common causes of health deterioration and four was establishing goals of care. So these are things that we think based on kind of extrapolation from other literature and just common sense would probably help sepsis survivors have better outcomes. And in fact, on the right here in the red box, you can see results from some observational data that we looked at in our healthcare system. We evaluated a cohort of about 200 sepsis survivors and looked to see how frequently these four care elements were provided after discharge. And in fact, for patients who received more of these care elements, for each additional care element that you received, you had a lower odds of experiencing hospital readmission or mortality. So there was kind of this idea that these care elements really are important and the delivery of them is really important for improving long-term outcomes after sepsis. The question that we had as a clinical and research team was how do we deliver those care elements or how do we operationalize this support for sepsis survivors? And the intervention that our team came up with was a nurse navigator driven care delivery system that we call the STAR program or sepsis transition and recovery program. This intervention is described in detail elsewhere, but I'll briefly kind of go over how the intervention works. So we have a patient identification algorithm that's built into our electronic health record that screens the health record for patients who meet a clinical definition for sepsis and then are deemed to be high risk based on our internal risk models for either readmission or mortality. And that list of eligible patients gets automatically sent to our STAR nurse navigators who will reach out to the patient and make contact while they're in the hospital. The STAR navigators start to develop a relationship with the patient, they do some assessments and work with the treatment team to facilitate the delivery of best practice inpatient care. All of that happens over the telephone. These nurse navigators are located remotely. And then when the patient's preparing for discharge, the STAR navigator provides sepsis specific education to the patient and caregivers, extra attention to medication around the time of discharge and really just make sure the patient has everything they need to support their discharge to whatever care setting that is, whether it's home, home with home health, skilled nursing, wherever the patient's going, the STAR navigator helps get them ready for that. Then the next step in the early transition period, within 48 hours, and we think this is really important, within 48 hours, the STAR navigators reach back out to the patient, go back over some assessments, review meds again, monitor for improvement of their symptoms and occurrence of new symptoms, and then sort of repeat that process of deciding if there's any barriers to their care and figuring out how to overcome those barriers. After that initial within 48 hour assessment, the STAR navigators continue to contact patients at regular intervals throughout the intervention delivery period. And I will say, I put in the wrong image here. So in the impacts trial, we had planned and did deliver the intervention for 30 days. The 90 days is from the next step, the next time we did this intervention. So this intervention was actually delivered for 30 days. And if the navigators noticed problems or there were problems that arose, they would facilitate the patient to be seen quickly by their primary care provider. And if the PCP was unable to take care of the problem, we have a transition service support clinic that can take care of some of those problems. If they arose. So we developed this intervention and kind of like Dr. Terabici said, we didn't wanna just roll it out and never know, does this definitively work? We wanted to test it in a rigorous way. So we designed the impacts trial, which was a two arm parallel group pragmatic randomized controlled trial. We enrolled patients from three hospitals around the Charlotte, North Carolina area. We enrolled from January, 2019 to March, 2020. And we followed up patients for 30 days after discharge. So I'll just take a minute and talk about our enrollment algorithm, our EHR embedded enrollment process, because I think this is really important to the pragmatism of the study. So we built a algorithm that we embedded into our electronic health record that screen patients for a clinical definition of sepsis. So a combination of antibiotics and cultures related to each other in time. And then we have an internally developed risk model that predicts risk of mortality or readmission. And patients who met that sepsis criteria and or high risk in either mortality or readmission were automatically populated on a list that gets sent daily to the STAR Navigator. So again, I think this really was important for the pragmatism of the trial. We didn't have to manually screen patients or rely on referrals. We didn't do individual consent. We had a waiver of individual consent because most all of these care elements are things that would be standard of care. They were just delivered in a bundle in this intervention. So this is how patients were identified and enrolled in the study. The primary outcome of the study was a composite of mortality or hospital readmission. And we have sort of a unique situation in Atrium Health that it's a very large healthcare system. So we're able to, we have visibility of hospital readmissions to 47 different hospitals and a really large market share. So we feel pretty good about the readmission data. We used the intention to treat analysis and we estimated the conditional odds ratio for the effect estimate. So this is a precise, whoops, sorry. Flags are moving slow. This is a precise two diagram that shows the pragmatism of the study. If you're not familiar with the precise two scoring, a one is considered most explanatory over here at the hub of the wheel and a score of five is considered most pragmatic out at the periphery of the wheel. And our study, as you can see, was relatively straightforward. It was relatively pragmatic on these nine domains. So for enrollment, we screened, 2,630 patients were screened. 930 didn't meet eligibility criteria. 1,918 were excluded. Many of them were excluded because of resource constraints. When we did the impacts trial, we only had one nurse navigator and we knew that she was limited to the amount of workload that she could do. So we capped randomization at a level that would allow her to feasibly deliver this support. So we ended up with 712 patients that were randomized. We randomized them one-to-one to either receive usual care, which was kind of just the regular stuff that patients get after discharge in our healthcare system, or the sepsis transition and recovery program. After randomization, we ended up excluding a few patients because they ended up having infection ruled out prior to discharge. We ended up with 691 patients in our analytic sample. Patients were well-balanced in the usual care group versus the STAR group. I'll just point out some overall cohort characteristics. They were relatively comorbid with a comorbidity index score around five. About half of them, a little under half of the patients required ICU admission. A little over a third had septic shock. And the hospital length of stay was around eight or nine days. This is the results with regard to our primary outcome of a composite of 30-day all-cause mortality or readmission. And that event occurred in 33.3% of patients in the usual care group. Versus 28.7% of patients in the STAR intervention group. This was a risk difference of about 5%, which was statistically significant. And a secondary analysis of only patients who survived the initial hospitalization. That was a risk difference also of about 5% that was also statistically significant. So for patient engagement, I think this is really important to point out, particularly for patient accepted STAR program follow-up. We had a lot of fidelity to the intervention throughout the study. So 77% of patients accepted STAR program follow-up. When you compare this to the rates of say, patients attending a ICU recovery clinic or transitions after the hospital clinic, this is much, much higher. Rates of clinic attendance, those types of recovery clinic attendances are 20, 30%, something like that. But we had 77% of patients that agreed to enroll in the STAR program and had some form of STAR follow-up. And then 60% of patients actually completed the full 30 days of follow-up. When you look at process measures, which we looked at to kind of see what's actually happening in the STAR group compared to the usual care group, to kind of see maybe what's making the difference here. It looks like I circled the wrong thing here. So the one under that, behavioral health screening. So depression screening happened more frequently in the STAR group than in the usual care group. Medication reconciliation happened more frequently in the STAR group compared to the usual care group. And then ascertainment and documentation of care goals happened more frequently in the STAR group than the usual care group. To put this all into context, in 2017, the World Health Organization put forth a resolution on management of sepsis. And one of the key things that they added was that we really needed to do better at identifying support to improve the long-term burden of sepsis. And kind of similarly, the 2021 update of Surviving Sepsis Campaign added a new section on long-term outcomes. And this is really a recognition of the fact that sepsis survivors do tend to have poor long-term outcomes. And a lot of the recommendations were weak with low-quality evidence or just best practice statements. So there really is this really important need to identify interventions that are effective. And in that setting, this is really the first intervention that has shown to improve outcomes for sepsis survivors and important outcomes like readmission and mortality. Some strengths and limitations of our studies and strengths. We designed this to be a really rigorous test with a randomized controlled study design. We enrolled patients from three hospitals that had a diverse population. Like I said before, I think the electronic health record embedded recruitment strategy was really important to make this a really pragmatic study to make sure that we enrolled a lot of patients and that they were a representative sample. And really just a sample that represented the real world impact of an intervention like this. Again, it was highly pragmatic, so it kind of maximized the real world impact. And we used readmission and mortality as our primary outcomes, which are meaningful both to patients and the healthcare systems. Some of the limitations, obviously with a multi-component intervention like this, it's impossible to disentangle the effects of the individual components. So what was the secret sauce in the STAR program? At this point, we don't know. It could have been just one thing that really had a big impact, could have been the constellation of things. So really teasing that out is something for future studies to evaluate. The second thing is that we use a 30-day follow-up period for this study, which in the grand scheme of sepsis survivorship is actually relatively short-term. So evaluating long-term outcomes and whether this type of intervention improves outcomes even longer than 30 days is important. And then factors that are associated with successful implementation of this type of program are really key because finding an effective intervention like this is great, but we also need to figure out how to disseminate it to other sites that may be different than our own. So with that, I will thank my fantastic team on this study, particularly my co-PI, Dr. Mark Kolkowski, our STAR and IVERC navigators, Alita Rios and Joan McSweeney, and just a wonderful team that worked together on this project. And I think we have time for questions. I'll turn it back over to Tomas to moderate the question section. Thank you very much. And thank you both for the absolutely fantastic presentations. I always learn a lot on these webcasts. So without further ado, there are a couple of questions. And the first one goes to Dr. Tarabici. And I think it's a question that many of us are asking that have you switched the early warning score help on for all patients now after your trial? Yeah, so actually we've turned it on for everybody in the emergency room before we even wrote the manuscript. So when we saw that positive result, remember it was a quality improvement initiative. We went ahead and just removed the randomization feature and left it be. And part of that assessment, right, this was a very pragmatic quality improvement approach. We wanted to know how much time this was taking from our pharmacist. And it turned out not a whole lot because they were ultimately involved in these patients' care anyways. And they were also potentially kind of screening patients on their own to start with. So this gave them a little bit of a triage mechanism. And we went around and we asked, hey, what do you think actually worked here? I mean, just like Dr. Taylor was saying, we don't know what part of this really actually worked. So it's a package, right? But the gut feeling was, hey, having the pharmacist involved that was great, that was crucial. They got the antibiotics in faster. Everybody got rallied around with that, with the alert. So I think that that was probably the primary mechanism that worked. And because it wasn't that much more time for them, we thought it was reasonable to unleash it into the entirety of the emergency room. Now, we're not using this on the inpatient setting. We haven't validated it yet on the inpatient setting. That was kind of one of the initial use cases for the model. I'm not optimistic, to be honest, just looking at how it does overall, and also based on talking to other institutions that are using it. Thank you. Thank you very much. Dr. Taylor, like with any bundle of interventions, people will ask that what do you think are the most important parts which help the patients? What can you tell us about that, please? Yeah, again, that's a great question. And it's something that we're studying. We have a kind of a follow-up project to the impact study, the ENCOMPASS study, which is a type one hybrid effectiveness implementation study that we're doing some really in-depth implementation science evaluation to figure out what's really going on and kind of tease out that secret sauce. What is it that is necessary and sufficient to make this program work? Because you really wanna distill out those characteristics for dissemination. You don't wanna necessarily propagate the fluff of the program. You wanna be able to disseminate the really important aspects of it. So we're studying that and we should be able to get some results that are able to guide that question. Honestly, I think at the end of the day, it's gonna be some things are important for some people. And I think we still have some work to do to individualize. Substance survivors are so heterogeneous. And I think sometimes it's really just the case management and the kind of more social determinants being taken care of. Sometimes it's more medication rec, sometimes it's symptom monitoring. So I think there's gonna be an individualization process and hopefully we'll end up with a set of things that work that can be targeted towards the group of patients that need it. Thank you. Going back to Dr. Tarabicci, and you mentioned the pharmacist's involvement that they were already screening the patients. Now, in the study, have you had any extra resources for the pharmacist? And if yes, is it maintained now that you have switched it on in the ED? So it turns out that this is actually just a small fraction of what they do in the emergency room. When you think about the remainder of their roles, they're involved in all of the code strokes, the traumas, anytime really an antibiotic is ordered or if there's some dosing issue in the emergency room, they've been involved. And so when we brought in the pharmacist, to be frank, the ED leadership wanted a pharmacist in the emergency room anyways. We were trying to figure out whether or not the additional burden of this early warning system was going to take them away from other things and the sense was no. So to answer your question more directly, it wasn't so much, how much more is this gonna cost us by putting some more bodies in the emergency room? It's a little bit more of, are we distracting them from the rest of their expected workload? And the answer was no. So we were, again, reassured that by doubling the patient load, we were not gonna distract them any further or at least to an undue degree. Thank you. Dr. Taylor, if you think about the readmissions and you mentioned that they are very frequent within the first 90 days of discharge, is it likely or is it possible that if you would extend the study and the bundle to be implemented for 90 days, then you would have even bigger effect size? Yeah, that's a fantastic question. And actually, that slide that I put up incorrectly was a slide for the ENCOMPASS study, which is the follow-up study to the IMPACTS trial, where we are extending the intervention out 90 days. So we sort of felt anecdotally that the 30 days for the IMPACTS study was a little too short. Our navigators were telling us, you know, I don't feel like patients are actually ready to be let go at 30 days, but that's the end of the program. So I had to hand them off back to their BCPs. And then ENCOMPASS trial, we did extend the intervention to 90 days. We're still, still that study is ongoing. It's still enrolling. We're about halfway through with a target of over 4,000 patients in the ENCOMPASS trial. But I'll tell you sort of anecdotally, it almost seems like 90 days might be a little too much. Our navigators are saying, you know, some of the people are losing engagement as you get towards that 90-day point where their problems are mostly solved by 90 days. So maybe 60 days is the magic point. We're still trying to sort that out, but that's a great point. Thank you. That's really interesting. Dr. Taradici, you mentioned that the pharmacists were not sort of disturbed by the extra workload because they were already there. And the other question which comes through is that have you found any other unintended consequences in other patient groups, like you mentioned, MI, stroke, trauma, et cetera, which needs timely care? Have you shifted the risk from one high-risk group to another, by any chance? That is a fascinating question. You know, that is the only way really to prove that, I think, is, well, I'm not totally sure it's actually possible to prove that. I would say that at least we support that that's not the case. Based on looking at historical trends for our time-to-antibiotics for patients coming into the emergency room, the standard care group looked to be consistent with historical prior. So, you know, it didn't see, you know, the concern would be maybe more valid if perhaps time-to-antibiotics increased from the baseline measure in the standard care group, but it did not. So I think that that reassures us. Now, were there other things we may have missed? There's, you know, we're definitely underpowered for some of these balancing measures. I'm not gonna pretend that we've definitely answered that this doesn't increase risk of C. diff or antibiotic overexposure. You know, that is to be determined, but I would say that if there is a difference there, the effect size is not gonna be, it's not gonna be huge necessarily. So thank you. Dr. Taylor, do you have any cost estimates for the intervention? Have you done any health economic analysis on this? Yeah, I feel like I keep giving you such a cop-out answer to all these questions, but that's part of our encompassed trial. Our aim too is valuation of cost effectiveness. So we'll have a better estimate from that, but, you know, readmissions are so expensive. And our nurse navigators are actually not that expensive because they're remote, because they're virtual. They can have a pretty large panel of patients. They don't require a lot of resources to support. So I'm just gonna, just based on the face value of it, it seems like it will be quite cost-effective, but we're still kind of crunching the numbers and figuring out definitively what the cost effectiveness estimates will be. Thank you very much. The other question to Dr. Tarabici is a more probing one. That is it really plausible that giving antibiotics about 40 minutes earlier has such profound effect on days alive and out of hospital? There are randomized controlled trials which have trialed really early antibiotic administration in severe sepsis, and they didn't really show that much of a difference. So how can we square that circle? Yeah, I would say, great point. And I would never profess that the 40 minutes is the sole reason for any improvement in outcome. I would also be, you know, I'd be careful to say that these are modest changes. Now, we are thrilled with modest changes because really our improvements in sepsis care are incremental, you know, much like, I really think that a lot of the emphasis, I mean, we've really pushed this time to antibiotics concept as much as we possibly can. I think the intervention worked for us in our setting. It may not be useful in all care settings. I mean, if there's juice to squeeze, by all means, think about an early warning system. And I wanna point out my, you know, my co-presenter, Dr. Taylor's work. I mean, that's where I think there's probably a lot of other possible places where we can squeeze for improvement in overall outcomes. So, you know, the answer, it's a cop out, but it's complicated. I think because it's a component of a care path, the fact that it happened earlier probably means an overall more robust response happened earlier. And it's not just the antibiotics necessarily. Okay, so you say that it is really, your early antibiotics is a marker of system success. Yes. And that's, okay, that's, I can take that. I can take that. I mean, a lot of these predictive models for risk of deterioration and whatnot, I mean, you know, you could be predicting early sepsis, or the guy having an MI may be having some complication that warrants immediate investigation, right? So just because they didn't get antibiotic, they happen to fall into the group, and even if they were ruled out to have a quote unquote infection by sepsis three definition, they may have had a more prompt response. Again, that's not to say that they were distracted by other situations, but this is a system that's flagging when continuous streams of data into the EHR change. And so I think, in general, an early warning system for risk of deterioration or a bad outcome might be useful in any context for prioritization in a room setting in general. Yeah, no, that's, thank you very much. That's very insightful. Dr. Taylor, if we think about your group of patients with severe sepsis, you showed that there are other risk groups like heart failure, COPD, et cetera, who are at similarly high risk. Is this what you have described and what you have implemented? Is this the model that a ICU follow-up clinic should do for all patients? Or is it really specific to sepsis? Yeah, I think there might be two questions in there, is whether this can be applied to other conditions like heart failure. And actually, this model is based on work that's sort of related to heart failure transitions, which has received a lot of attention being a penalty diagnosis for heart failure readmission. So there is some work like this being done in conditions like heart failure. We also borrowed some from cancer survivorship to kind of think about how that works. As far as whether this would be a good model for ICU follow-up care, I think yes. I think it's not, certainly not to replace an ICU follow-up clinic, because I think there's certainly a place for that brick and mortar clinic where patients can come and get all of their multi, you know, component needs taken care of in one clinic. But the studies of those clinics, like I mentioned, have really low attendance rates. So when you try to enroll patients in trials of an ICU recovery clinic, the follow-up or attendance is really low. And so you're limited in what you can do. And so I think a program like this is ideal for the select patients that need you to actually reach out to them and support them where they are. Whereas I think there is a subset that would benefit from ICU recovery clinics, keeping it the way it is. There are some ICU recovery clinics that developed a more virtual model of care delivery during COVID. And I think that's here to stay. So probably some combination of things. And a lot of the lessons that we've learned in sepsis survivorship, I think is really applicable to the general ICU population. Although ICU follow-up clinics are, many of them are sepsis survivors. About 50% or half of the patients in ICU follow-up clinics are sepsis survivors. So I think a lot of this information is interchangeable and we can learn from each other's care delivery models. Thank you very much for answering all these questions. And this concludes our Q&A session. Thank you to our presenters and the audience for attending. Again, everyone who joined us for today's webcast will receive a follow-up email that will include an evaluation. And I would like to ask you to take five minutes to complete it. Your feedback is greatly appreciated. And on a final note, please join us for our next Journal Club, Critical Care Medicine on Thursday, 28th of April. This concludes our presentation today. Thank you very much for both.

sepsis

early warning system

emergency department

antibiotic administration

electronic health record

mortality

readmissions

sepsis transition and recovery program

nurse navigators

post-discharge support