false
Catalog
Leadership and Management Skills to Enhance Your P ...
How to Utilize Unit-Based Metrics and Financial Da ...
How to Utilize Unit-Based Metrics and Financial Data
Back to course
[Please upgrade your browser to play this video content]
Video Transcription
Good afternoon, everybody. My name is Derek Wheeler. It is my pleasure to be here again. This is a great Event. I'm so glad that the society is selected to do this So I'm currently privileged to be the chief operating officer at Lurie Children's Hospital in Chicago But in that role for about 14 days some would say that I've gone completely over to the dark side, but I'm still practicing. So that was one of the requirements when I accepted the position But I'm going to talk a little bit about ICU metrics and financial data. I have no real relevant financial relationships to disclose The objectives we're going to review the Donabedian model of quality and safety in the context of commonly used ICU metrics discuss how we can use data to improve care in the ICU and Most importantly discuss how not to use metrics in the ICU and certainly if you're interested in this topic There is a great, I hope, I think it'll be a great pro-con debate tomorrow. I think it's at 1030 That I'm I'm arguing the con And one of my colleagues is going to argue the pro, but it should be a lot of fun So just to bring up the context of our discussion So I think if you work in the United States in the health care system You've heard this statistic before the total national health care expenditures. It's about 20% of our gross domestic product so quite a lot of money is spent on health care in the United States and you know, one of the arguments that I hear a lot is that Healthcare has become too corporate. It's become too much like a business Guess what? You cannot be on 20% of the gross domestic product without being a business So we we do work in a business That doesn't necessarily mean that we are still not professionals But health care today is big business. We spend a lot of money on health care. And so Steve Pastorius is this is a lot of his work actually So if you look at where all that money goes or how that money is spent About 35 cents of every health care dollar is spent in the hospital setting 33% is spent with doctors and so, you know 10 cents of every dollar is administrative support and everybody's frequently argues that If we had a nationalized health care system, we would eliminate all the administrative costs And while 10 cents of every dollar spent is still a big number You know, I think there's a lot of other opportunities here for us to look at really how we spend money in health care today ICU care it's about 13 cents of every dollar that's spent In the United States and so a pretty significant component everything that we do and we all know this, right? we know that ICU care is very expensive and There's living proof of it right there So, you know that led Warren Buffett to call health care costs the tapeworm of economic competitiveness and a lot of that, you know, he would suggest that we cannot be Competitive as a country in the United States with all the money get that gets spent on health care And so if you look at just the statistics since 1960 If other prices had grown as quickly as medical costs, we'd be spending $78 on a gallon of milk $43 on a dozen eggs and $57 on a pound of coffee and I guess maybe for Starbucks a lot of people would be willing to spend $57 on a pound of coffee, but You know the inflation that is tied to health care costs in the United States has really gone out of control And so it's really not a long-term sustainable issue for us and and we really need to be very focused about how much we are spending on health care and identify ways that we can cut costs and So the argument is frequently. Okay. Well you get what you pay for, right? And so if you look at this This is just comparing the United States health care system to the Canadian health care system and then to the Mexican health care system so all of You know the three biggest countries in North America and if you look at the the money that we spend we spend more money than both Canada and Mexico combined on health care and so the frequently argued statistics, you know They look at the quality of our health care delivery system and they typically put up infant mortality under age 5 mortality I'm a pediatrician. So those things are important to me and then long-term life expectancy and if you look at the statistics compared to Canada Spending about half of what we spend they do much better from a health care delivery perspective, right? Everybody argue agree with that So so my question to you is infant mortality and under five mortality and life expectancy Truly measuring the delivery that we provide in the hospital setting and in the health care setting I would argue that it's not and so hopefully I'll be able to show that to you. So this is a great book It's by Elizabeth Bradley. So she's a researcher at Yale Wrote a book probably about five years ago Maybe ten now called the American health care paradox why spending more is getting us less And so what she did is she looked at Healthcare spending among all the g20 countries and then the spending on social services things like Food stamps welfare all of that and you can see that the US most of our dollar That we spend of those two things social services spending and health care spending goes towards health care, right? Compared to other countries they're spending about The an equivalent amount of money on both of those But they're spending a lot more on the social services side than they are on the health care side So they're taking a lot and so the argument maybe we are investing our money in the wrong place And so if you look at that that makes sense this is a well-known statistic that only about 20 percent of Healthcare outcomes are really due to the care that is actually occurring a lot of that 80% is due to all of the other factors the social determinants the social influencers of health So I think all these other countries have kind of figured this out and and they're actually looking at ways to invest in health Delivery as opposed to health care delivery. So if you take anything from this 20-minute talk It's you know, whatever metric that you use to measure whatever you want to in the intensive care unit Make sure that that metric is actually measuring what you think and what you want it to measure Because I would argue that statistics like under age 5 mortality infant mortality and life expectancy Truly are not assessing the quality of the health care delivery system as much as the social services system So so and and when we look at metrics, that's probably the biggest Biggest issue that we have with any ICU metric whether it's a financial metric or whether it's a quality metric It's really not measuring what we think it's measuring or what what we want it to measure So, you know Donna Bedian came up with this model probably about 30 or 40 years ago But it's it's really looking at three types of metrics. You have structure metrics Process metrics and those things together drive outcomes So so what is a structure metric? And so this is this is I wrote this for The pediatric current concepts course probably about 10 years ago So some of these metrics are a little bit outdated, but if you look at structure metrics They're going to be things like safety culture your nurse patient staffing ratio whether you have Looking at your case mix whether you have rapid response systems when I wrote this that was whether you have the computerized physician order entry So all those things are structural characteristics of your health care delivery system that then would combine with the process measures so things like bundle compliance Your threshold for Blood cell red blood cell transfusions, etc. The time to antibiotics for kids with sepsis or adults with sepsis They lead to your outcome so structure plus process equals outcomes and those are the things that I think we're really interested in whether we're Actually improving ICU mortality ICU length of stay those kinds of things So It's also important that whenever you look at a set of metrics use a portfolio of measures There's really not one specific Measure that we can use in the intensive care setting that really tells us whether we're a good ICU versus a bad ICU rather we should be using a portfolio of measures and so Again, you talk about process measures and outcome measures just as important is to use Balancing measures just to make sure that we're improving at the expense of something else. So if you're Improving ICU outcomes, but you're doing so with a much higher cost of care Maybe that's not necessarily what you're you should be focused on and so unfortunately I think what we frequently have is a lot of hospitals emphasize process measures because they're really easy to measure Okay, and so this is this is absolutely one of my favorite stories, so this was our PICU ventilator associated pneumonia bundle compliance back when everybody was really focused on ventilator associated pneumonia for a period of time we went 522 days In our PICU without having a ventilator associated pneumonia It had been so rare in our ICU that we actually held a funeral for ventilator associated pneumonia Probably not our best idea, but we had a funeral in the ICU where we celebrated the fact that we went This long without having a ventilator associated pneumonia And I can tell you during that time our compliance with the ventilator associated pneumonia bundle was only 65% Right and and so when we compared ourselves with the cardiac ICU and the neonatal intensive care unit their bundle compliance was 95% but they had a much higher rate of ventilator associated pneumonia, okay, so that what does that tell you well? Maybe the VAP bundle didn't necessarily drive improvement in outcomes I will tell you that wasn't the reason so the reason was the way that we monitored bundle compliance So the cardiac ICU and the NICU they actually used self-report So they would ask the nurses at the end of every day. Did you do all the elements in your bundle? Yes or no? Right and and well, you know, they didn't rate themselves perfectly because their compliance was only 95% But it was all self-report whereas in our PICU it was all direct observation So it was an individual watching the nurse and the respiratory therapist in the physician making sure that he or she Did all the elements in the bundle and oh, by the way, if they saw that they were not being compliant They would say, you know, you really need to do that and scrub that maybe for 15 more seconds You need to wash your hands for the full 30 seconds. So they would correct the behavior in real time But they would still rate it as being non-compliant with the bundle. So so you've got to be really careful about your process measures When when you work on that It's also important to use kind of a set of cascading or roll-up measures. So make sure that what you're measuring and what you're following in the ICU is something that the other ICUs in your hospital can utilize. And then all of that should roll up to system level measures at the hospital level. So things like standardized mortality ratio at the hospital level, cost of care, all of those should be kind of at the top level and the individual units should kind of roll up to those. So that's what we mean by a cascading or roll-up measure. It's important to be transparent about your data. You have to show it. And there's all kinds of things that you can find online about how to display your data, whether it's a compass, as you can see here. One of the popular ways is a dashboard. So you have a dashboard of your portfolio of measures. Looks like an automobile dashboard where you have green, yellow, and red, depending upon how you're doing with that particular metric. The balance scorecard, something that was really popular for a while. But it's also important that you show that data. So regardless of whether you use the compass, the balance scorecard, the dashboard, make sure that it's displayed prominently and in such a fashion that it's actually getting used. So this is a good picture of that. So this is actually in our cardiac ICU where they do a huddle at the beginning of every day in front of their outcomes board. And they talk about their different ICU metrics. And that kind of is really important on driving further improvement with all of those. And then this is a good example of our PICU improvement board, kind of very similar. So it looks really clean, easy to see, very transparent. The data is displayed in a way that is easily understood. So unfortunately, and so I remember going to a hospital, I don't know, five years ago, and I walked into this big room. It wasn't quite as big as this room, but it was a good sized room and every inch of their walls in that room, all four walls were covered with metrics, right? And they were really proud of it. And they're like, look at all this, this is awesome. You know, we know everything that's going on in our hospital today because of all these metrics. And then I asked, well, so all of these are hand collected. So who's collecting all this data and how long does it take somebody to actually display all this? So it just, you have to be really careful that you don't fall into that trap where you're just using a ton of metrics because they lose their meaning and they're no longer important, they're no longer relevant. And so that's a trap that I've seen a lot of different hospitals fall into. And this is a good example from Lurie Children's Hospital, right? So we're part of that. And so we're guilty of this, just like every other hospital. This was an outcomes board for one of our inpatient units. It was in the break room. So it was in a place where people frequently went to, right? So it was displayed in a prominent location of that particular unit. But if you look at this, I mean, there's, can anybody make any sense about what they're trying to show there? It's just completely covered with all these different charts and notations and key driver diagrams. And it loses a lot of meaning. It no longer becomes important and it no longer is useful to actually drive improvement, which is really kind of what you want for any metric that you use in the intensive care setting. So this is another great book. If you'd like to read this, I would say this is an excellent book. It's called The Tyranny of Metrics. It's not just healthcare, but it looks at all of these issues, right? And it talks about something that's called Goodhart's Law. Has anybody heard of that? Raise your hand if you've heard of Goodhart's Law. Couple of you. So basically Goodhart, I think he was a economist from the 1920s. I believe I'm not historical as Robert is. But it really is, it explains when a measure becomes a target, it ceases to be a good measure, okay? And so the classic example, at least in this cartoon, if you measure people on the number of nails that are made, you're gonna get thousands of really, really tiny, small nails, particularly if you tie the measure to any kind of incentive. If your metric is the weight of the nails that made, you're gonna get a few really heavy, giant nails, okay? And so you're all looking at this going, okay, that's good, those are okay examples, but they're not really relevant because none of us really make nails, right? Here's another great example of it, okay? So this was back in the time in British India when they had a cobra problem and all these cobras were killing people. So they decided, you know what, we're gonna, every dead cobra that you bring me, I'm gonna give you some money, okay? And so what do you think happened? So at the very beginning, people were bringing in cobras, but then when they started studying the problem, the cobra population actually wasn't decreasing, it was actually increasing. And why was that the case? Because the people figured out, hey, if we bring in all these dead cobras, we get more money, let's just start raising cobras. And so they developed all these cobra farms. And so again, that's a really good example of Goodhart's Law. And while it's a nice, funny one and a few of you are laughing, again, not very relevant to hospital care, right? Well, here's a really good one, a good example from healthcare. So this was the National Health Service in Britain. England was very focused on reducing the incidence of boarding in the emergency department, okay? And so if you look at the definition, if you've spent beyond four hours in the emergency department, at least in the United States, that's the definition of boarding. And there's some concern that if you spend more than four hours in the emergency department, the quality of care deteriorates. Certainly patient satisfaction gets worse, but maybe the quality of care. So there's an incentive potentially to admit patients and to get patients out of the emergency department and make a decision, are you gonna admit them or are you gonna send them home in that four hour timeframe? So the National Health Service started incentivizing and starting penalizing hospitals based upon their four hour length of stay in the emergency department. So what do you think happened? So you can see here in this green line, everybody at right around four hours started getting admitted to the hospital. So in order to meet the criteria, whether it was the right decision or not, right around four hours, if they were still in the emergency department, we're just gonna admit them to the hospital, right? So again, not kind of what you want out of a metric. It's not a behavior that you're trying to incentivize and promote. How many people have seen this graph? A few of you. So this is kind of the rate of growth of healthcare administrators like me versus the rate of growth of physicians really since 1970. So you can see here in the royal blue that the number of administrators, the growth has increased by 2,500%, right? So compared to the growth in physicians, which is only like, I don't know, 50%. So significant growth in physician administrator or non-physicians healthcare administrators. And people throw that around a lot and they're saying, you know, is this really what we want in our healthcare system? And so it's interesting. So has anybody heard of Google Ngram? So if you go to Google, you know, the website on Google, you can actually, it's called Ngram, and you can actually determine the number of times in all of the published literature, right? How many times specific words have been stated, right? Am I explaining that the right way? Does that make sense? So you can go and say, how many times was the word, you know, black mentioned in all of the literature that has ever been published? And so I did that with metrics, benchmarks, and performance indicators, right? And you can see that this blue line, the growth over time, really since 1970, of the number of times that metrics has been mentioned in the press, in published literature, both scientific literature and non-scientific literature, mirrors almost exactly the growth in administrators, right, in healthcare. And so what does that tell you? So getting all these metrics, getting all these key performance indicators takes a lot of work, it's a big investment, and it leads to us having to hire a lot of non-clinical folks to be able to measure and interpret and collect all of these metrics. So I think that just means that we should be more careful about the metrics that we use, and make sure, one, that we're using them, number two, they're measuring what we want them to measure, and three, by measuring them, they're being used to actually drive improvement in outcomes, which is really the goal. So Edwards Deming, who's really the father of quality improvement, has been mentioned in a lot of different articles. He came up with these 14 points of management, and so point number 11 was all about metrics. He said, eliminate management by numbers and numerical goals substitute leadership, right? So again, minimize the number of metrics and key performance indicators in the targets. Number five, he also had kind of a seven deadly diseases of management, number five was management by using only visible figures. So again, we need to be a little bit more careful about the metrics that we use in the intensive care unit in the hospital setting. Another big famous person is Don Berwick, was one of the founding leaders of IHI, the Institute for Healthcare Improvement, was the CMS administrator for a time, ran for the governor of Massachusetts, I think, but very well-known patient safety advocate, one of the leaders in healthcare quality improvement. And in an article a few years ago, he published the seven roadblocks to patient safety, and number four was metrics clut, right? So he said, stop excessive measurement. So here's a guy who firmly believes that using the proper metrics drives improvement. He actually said we're using too many metrics. So I would argue that we should pay attention to these, they're the experts, Deming and Berwick, and really measure what matters for our patients. So in summary, use a portfolio of measures. I like the Donabedian model of structure, process, and outcome, but also make sure that you use those balancing measures as well to make sure you're not having unintended consequences. I think using cascading or roll-up measures is critically important. Display your data prominently. Don't get too carried away, though. Beware the tyranny of metrics. Remember Goodhart's Law, remember the COBRA and the four-hour emergency department rule, and then lastly, measure what matters. And with that, I thank you for your attention. Thank you.
Video Summary
In the video transcript, Derek Wheeler discusses the importance of using metrics in healthcare, specifically focusing on ICU metrics and financial data. He highlights the challenges and pitfalls of using metrics, emphasizing the need to ensure that metrics accurately measure what they are intended to measure. Wheeler references key concepts such as the Donabedian model of quality and safety, the impact of healthcare spending on outcomes, and the importance of using a balanced portfolio of measures. He also cautions against excessive measurement, citing examples like Goodhart's Law and the unintended consequences of incentive-driven metrics. Wheeler advocates for transparency in data display, the use of cascading measures, and focusing on measuring what truly matters for patients in order to drive improvement in healthcare outcomes.
Keywords
metrics
healthcare
ICU metrics
Donabedian model
transparency
patient outcomes
Society of Critical Care Medicine
500 Midway Drive
Mount Prospect,
IL 60056 USA
Phone: +1 847 827-6888
Fax: +1 847 439-7226
Email:
support@sccm.org
Contact Us
About SCCM
Newsroom
Advertising & Sponsorship
DONATE
MySCCM
LearnICU
Patients & Families
Surviving Sepsis Campaign
Critical Care Societies Collaborative
GET OUR NEWSLETTER
© Society of Critical Care Medicine. All rights reserved. |
Privacy Statement
|
Terms & Conditions
The Society of Critical Care Medicine, SCCM, and Critical Care Congress are registered trademarks of the Society of Critical Care Medicine.
×
Please select your language
1
English