Good afternoon, everybody, and welcome again to the Critical Care Congress. My name is Valeria Valbuena, and I am a general surgery resident and national clinician scholar at the University of Michigan in Ann Arbor. Today we are talking about when we can all trust our monitors. I will be talking about the pulse oximeter and how the current design is limiting our ability to care for all patients equally. I have no relevant financial disclosures today. And our objectives for the next 20 minutes are simple. I will introduce this concept of measurement bias and what it means for medical design, discuss the evidence behind pulse oximetry racial bias, and explore strategies for addressing this issue. Before I can answer the question I was given for my title, we have to begin with what is race. The definition of race has changed and evolved, and we have gained a better understanding of the motivation behind creating human categories. Race is a descriptor of grouping of individuals based on shared physical or social qualities into categories which are generally viewed as distinct within a given society. Race is a social construct, though, and an identity which is assigned based on rules made by society and assigned in such a way so that some people get benefits and some do not. The concept of bias is an even more contentious one, but we will start by focusing on its strict statistical definition. The bias of an estimator or a test is the difference between this estimator's expected value and the true value of the parameter being estimated. In the graph on the right side, you can appreciate how bias lives between the true value and whatever measure value we obtain for the estimator. When we consider the concept of bias in medical design, it is quickly apparent how some of the most common tools we use in medicine might be fraught by it. In April of last year, the journal Science published an editorial by Dr. Ashuta Kadamy, was an assistant professor of engineering and computer science at UCLA, where he explored in detail how the hardware and software that operates medical devices can be biased. In his article, he elaborated on three broad forms of medical design bias that can affect medical devices, diagnostic algorithms, and prediction tools. Those are computational bias, interpretation bias, and physical bias. Computational bias pertains to the distribution and processing of data that are used to operate a device. Think about databases used for clinical prediction that have mostly mail samples or inputs, and because of that, under- or over-diagnosed women. Subsequent implementation in clinical settings can lead to interpretation bias, where clinical staff or other users might interpret device outputs differently based on demographics. This parameter is a good example here, where patients of certain races are assumed to have an inherently higher or lower lung capacity. Or the GFR equation, which is adjusted for race based on the belief that Black patients have higher muscle mass and consequently higher baseline levels of creatinine. And of course, a device can exhibit physical bias, where the physical principles are biased against certain demographics, and this is what we will discuss today. Physical bias and the measurement bias associated to devices designed with a white standard is a little less intuitive than the other two types. How can a machine encode bias? You know, the physical working principle of a medical device is bias when it exhibits an undesirable performance variation across demographic groups. An example of physical bias occurs in the context of the optical bias sensors of pulse oximeters. The currently widely used pulse oximeter uses two colors of light, one that is near infrared and the other one in visible light, to measure blood oxygenation. Unfortunately, there are a number of things that can affect this interaction between hemoglobin and light waves, and one of them is melanin, making it so that the device does not work as well in patients with darkened skin tones. And here is where our story with the pulse oximeter started. During the first wave of the pandemic, Michigan was hit very hard, and of course, like all of you, we care for hundreds of critically ill patients all around the state, and relied more than ever on pulse oximetry in a number of settings. And at this time, clinicians at my institution started to notice occurrences of undetected, occult hypoxemia, where patients with an otherwise normal pulse oximetry reading were profoundly hypoxemic on blood gas evaluation, and most of these patients were Black. Our working definition of occult hypoxemia is an arterial blood gas saturation of less than 88%, despite a pulse oximetry reading of greater than 92%, which is what most of us will consider a normal range. At the same time that the phenomenon of dispirited rates of occult hypoxemia in Black patients was being noticed in intensive care units during the first wave of the pandemic, Dr. Amy Moran Thomas, who is an associate professor of anthropology at the Massachusetts Institute of Technology, published this eye-opening piece in the Boston Review, where she explored decades of accumulated evidence, demonstrated the systematic measurement bias of pulse oximeters in dark-skinned patients. And so prompted by their observations at the bedside, clinicians at my institution aimed to answer the following question. When the pulse oximeter reads 92 to 96, what are the odds that a patient will have an oxygen saturation of less than 88% as measured by an arterial blood gas? Of course, this is our definition of occult hypoxemia, and are these odds different for patients of different races? And here, of course, race was used as a surrogate for a skin color. And here's how they went about answering this question. This was a retrospective study of two large patient cohorts, one at the University of Michigan and a larger multicenter one abstracted from the EICU Collaborative Research Database, which included information from 178 hospitals. Adult inpatients receiving supplemental oxygen were included, and pair arterial blood gases and pulse oximetry measurements within 10 minutes of each other were extracted. And they tested for occult hypoxemia for patients identifying as either Black or White in both cohorts. The SpO2 range selected for their analysis was very intentional. This is the SpO2 range where, debatably, many of us will be reassured about the oxygenation of a patient with the right clinical picture. A pulse oximeter reading below 92 will raise alarms, and honestly, anything higher than 96% is likely stone-cold normal. So that SpO2 range of clinical interest is very important in their design. Their estimates of occult hypoxemia rates were adjusted for age, sex, and SOFA score for the U of M cohort. Here's a box plot depicting the accuracy of pulse oximetry in measuring arterial oxygen saturation according to race. In the x-axis, we have the oxygen saturation as measured by pulse oximetry as a percent, and in the y-axis, we have the arterial oxygen saturation as measured by IBG as a percent as well. The shaded area indicates an arterial oxygen saturation of less than 88%. And in the box plot, the horizontal line within each box represents the median at the top and bottom of each box, in the top and bottom of each box, represents the upper and lower limits of the inter-flarctile range. You can appreciate how at every pulse oximetry reading in the range of clinical interest, Black patients have true lower arterial oxygen saturations overall compared to what the pulse oximeter measured, and lower relative to White patients, frequently crossing that 88% line. In the unadjusted analysis of the U of M cohort, the rate of occult hypoxemia was 11.7 for Black patients versus 3.6 for White patients, and these rates were very similar after adjusting for age and sex and SOFA score. In the multicenter cohort, the rate of occult hypoxemia was 17% in Black patients and 6.2% in White patients. So my co-authors demonstrated that in two large cohorts, Black patients had nearly three times the frequency of hypoxemia that was not detected by pulse oximetry as White patients. The results of this correspondence have now been widely circulated, and it prompted a number of reactions and public engagement around the issue. Interestingly, this was not the first time this phenomenon had been described in the literature. Here are two of the most salient pieces of historical evidence. The original study on pulse oximetry racial bias was actually published by Dubran and Tobin in 1990. The purpose of this study was to determine if SpO2 could be reliably substituted for measurements of arterial O2 tension, PaO2, when adjusting FiO2 in ventilator-dependent patients. They evaluated pulse oximeter saturation values of 54 critically ill patients. The graph on the left shows their principal findings. In the x-axis, we have arterial oxygen saturation as measured by blood gas as a percent, and in the y-axis are pulse oximetry measurements from 70 to 100 as a percent as well. The solid line in both grids is the line of identity for full correlation of the two measurements. In the dashed lines are the isoplets of different levels of bias. There were 55 measurements obtained in white patients, which is the top red, and 43 measurements obtained in Black patients, red B. And you can appreciate the difference in the distribution of the measurements in relation to the solid line. There are significantly more measurements for Black patients that cross the higher levels of measurement bias compared to the measurements in white patients. And thus they concluded that pulse oximetry was less accurate and frequently overestimated SAO2 for Black patients. Over 10 years later, Feiner and Bickler tested measurement bias of three separate pulse oximeters. The graph on the left shows the result of two of those devices. The x-axis in both plots corresponds to SAO2 as measured by blood gas, and in the y-axis you have the mean bias for the measurement difference between SpO2 and SAO2. The blue line in both grids is at zero bias for the device. So you see light-skinned subjects are indicated by open circles, dark-skinned subjects by closed circles, and intermediate skin pigments with gray circles. You can appreciate how bias was generally the greatest in dark-skinned subjects, intermediate for intermediate skin tones, and the least for lightly pigmented individuals. The majority of the white circles in both charts are hovering over the no bias line. And of note, the range of SAO2 they chose for this particular experiment went back all the way to an SAO2 of 60 to 70, where bias between the groups is objectively the greatest, but the effect remains in the higher and clinically relevant SpO2 levels. So these studies had some limitations, of course, including limiting comparisons to only Black and white patients, and patients had a range of critical illnesses. The New England Journal of Medicine study had no data on occult hypoxemia in the higher SpO2 range, and it used timestamp electronic data that had not been confirmed by a clinician. In the most recent contribution of my group, which was published in Chess, we aimed to address some of these limitations. Our research question this time was the following. Do pulse oximeters less effectively detect arterial hypoxemia in Black, Hispanic, and or Asian patients compared to white patients in respiratory failure who are about to initiate extracorporeal membrane oxygenation? And these are the results of our analysis where we compare the unadjusted rate of occult hypoxemia between patients of different races using chi-square tests. This is a bar graph representing these results. The x-axis is race and the y-axis is the percent of patients in each group with occult hypoxemia. The rate of pre-ECMO occult hypoxemia was 10.2% for 186 white patients and 21.5% for 51 Black patients, 8.6% for 70 Hispanic patients, and 9.2% for 65 Asian patients. In a logistic regression, which we adjusted for sex and measure SpO2, Black patients with respiratory failure had a statistically significantly higher risk of occult hypoxemia with an odds ratio of 2.57 compared to white patients. And these results were similar to the findings by Jodin et al. No difference in occult hypoxemia rates were noted for Hispanic or Asian patients in our study. When we changed the definition of occult hypoxemia to an SaO2 of less than 88% when SpO2 was greater than 96% of the higher end of the SpO2 range, Black patients had a statistically significant higher risk of occult hypoxemia with an odds ratio of 3.52 compared to white patients. And that was statistically significant. Although most of us could imagine the effects that a degree of hypoxemia will have on patients, there has been little clarity on what this disparity means for clinical outcomes until recently. Wong and colleagues sought to bridge this gap in their multicenter retrospective cross-sectional study, which included three publicly available databases spanning 215 hospitals and 382 ICUs. Their JAMA Network open publication both demonstrates the higher rates of occult hypoxemia among Black patients compared to white patients, but also demonstrates this difference in Asian and Hispanic patients, which is a notable finding. Using a multivariable regression, they associated the higher odds of occult hypoxemia with higher SOFA scores, higher in-hospital mortality, and higher lactate levels, which are all important markers of worse global clinical outcomes. So given these findings, we can see how that pulse oximetry has some limited usefulness when it comes to predicting true arterial hypoxemia in patients of different races, and how compared to white patients, minoritized patients have higher prevalence of occult hypoxemia and worse clinical outcomes associated with it. So now we are going to discuss some of the implications of these findings and their historical context, which is of particular importance to answer that question at the beginning of the talk. The growing body of evidence on pulse oximetry racial bias indicates this phenomenon is real and clinically relevant. More importantly, this has been a known and reasonably well-described issue in the academic community for a long time, but somehow we have failed to address it. This realization brings us to an even more important question, which is, what do we do about it? What do you do when your equipment is broken, or in this case, systematically misperforming for a group of patients? Thankfully, this issue has received enough public attention that it has become of interest to some key stakeholders. Earlier last spring, Dr. Elizabeth Warren, a member of the Senate Committee on Health, Education, Labor, and Pensions, and Senators Broecker and Wyden sent a letter to the acting commissioner of the FDA urging them to quickly conduct a review of the accuracy of pulse oximeters across racially diverse patients and consumers. A month later, the FDA issued this safety communication on pulse oximeter accuracy where it both summarized the available evidence and provided succinct recommendations for clinicians to be cautious of the interpretation of oximetry findings. At this time, the FDA is also evaluating published evidence to reassess both the content of their safety warning, but also their recommendations for device design and calibration. Less covered, but probably just as important, has been the push by regional legislatures to raise awareness of this issue. Representative Patricia Dillon from the 92nd District of Connecticut was the primary sponsor of this bill, which was aimed to inform healthcare providers and insurance companies about the issue and ensure alternative routes of reimbursement for things like home oxygen that are not based on meeting a certain measurement criteria in pulse oximetry. More recently, we have seen the revitalization of this discussion when the UK Health Secretary ordered a formal review of the devices in November of last year, citing the increased deaths of minoritized citizens in the UK and the glaring inequalities associated with the care of these patients as a motivator. The UK and US health authorities are currently collaborating in this review. In the micro level, nothing is more important for patients right now than clinicians making informed decisions about care that account the accuracy issues of the device. Our collective reliance in pulse oximetry measurements needs to be tempered by following trends rather than single measurements and using additional diagnostic tools, including laboratory tests, to globally assess tissue oxygenation in patients, especially those who are critically ill. I believe the current body of evidence is strong enough to elicit some explicit advocacy by the governing bodies of our specialties. In their pulse oximetry position statement, the Intensive Care Society of the United Kingdom issued some strong and succinct expectations for industry leaders, health care organizations, and colleagues around the world demanding immediate action to design, distribute, purchase, and utilize pulse oximetry devices that have been tested on a diverse population and that work well for all patients, not just some of them. I cannot help but wonder if SCCM will want to co-sign this document or release a similar statement. Critical care trainees have also been very vocal about their desire for pulse oximetry reform. In their lens of perspective piece, Drs. Coloni-Dalgo, Lunzaya, and Harlan, which were all pulmonary critical care fellows at the time, highlighted the evidence behind the phenomenon and the urgency to stop using white patients as the measurement standard. And finally, when in doubt, I would just say get a guess. Our threshold for doing this has become significantly higher over the years, but these findings should increase your level of suspicion of hypoxemia in patients with the right clinical picture and decrease any reinsurance you might have gotten in the past from single pulse oximetry measurements alone. So there you have it. This is the evidence landscape we're looking at. There's about three decades' worth of data indicating that this device that we use every day to take care of patients is systematically not working as well for certain groups. And somehow we are all here today grappling with the consequences of historical negligence around this issue and what to do about it. So is the pulse oximeter racist? Well, you know, inanimate objects cannot be inherently racist, but we as a professional community can certainly be by inertia if we choose not to do something about it. I believe the larger and more existential question as a field and as clinician and advocates and patients ourselves is what we are willing to tolerate. This has been a recognized problem for so long, and yet today several of your patients are decompensating in the floors and ICUs without being noticed as promptly just because of their skin color. And although, you know, oximetry is the main topic of the discussion today, I want you to recognize that our tolerance for this or anything else for that matter truly globally affects the moral compass of the House of Medicine because we cannot provide excellent care as a profession unless we provide it for all. Thank you all for your attention, and please let me know if you have any questions.

pulse oximeter

medical device

design bias

physical bias

undetected hypoxemia

racial bias