SCCM Resource Library-Barriers to Real-Time, Multidisciplinary Clinical Implementation of AI Applications

Video Transcription

Hello, my name is Tyler Loftus. I'm a trauma and acute care surgeon at the University of Florida.  Thank you for the opportunity to present on this panel, Barriers to Real-Time Multidisciplinary Clinical Implementation of Artificial Intelligence Applications.  The rationale for this part of the panel discussion is that artificial intelligence in critical  care has largely failed to emerge from the valley of death, or the chasm between model development and real-world implementation.  I'll be presenting several challenges and proposing solutions. One major challenge is that lack of data standardization impairs multicenter validation and dissemination  of artificial intelligence models. One potential solution for this problem is that we can use common data models or federated  learning to maintain data security while sharing models without sharing data. Here's an example of a federated learning workflow.  Several local models train on local data and they send model hyperparameters or coefficients, some information that the model has learned, to a central model.  And that central model can then test on new unseen data and can be deployed widely without having to share data between institutions.  This approach has the potential advantage of sometimes increasing model performance. Here's an example published in Nature Medicine in which the blue bars represent the area  under the curve or discrimination of the model in predicting the outcome of interest using local data only.  And then the green bars are a federated learning model that was trained on data across all the different institutions and data sets.  And as you can see, the federated learning model actually had superior performance in many cases.  Another major problem is that patients and clinicians often mistrust black box models and fear the possibility of egregious errors.  This is a somewhat famous example of a computer vision model that is able to successfully  identify several objects in a room, like a laptop, a couch, and a person playing video games. When the investigators put an elephant in the room, the model was occasionally fooled  and sometimes labeled elephants as a chair, for instance. And so as a solution for this mistrusting of black box models, especially when they're  capable of egregious errors, we suggest implementing model interpretability, explainability, and uncertainty estimation mechanisms.  For example, this is a conceptual framework for optimizing certainty and deep learning predictions.  There are two main sources of uncertainty. One is aleatoric uncertainty, which is due to noise and variability in data.  And the second is epistemic uncertainty, which is due to model parameters. It may be possible to overcome aleatoric uncertainty by making uncertainty estimations and mitigate  epistemic uncertainty with learning curves. So uncertainty estimations measure variance in the distributions around point estimations created by a model.  So for instance, you can vary the model by changing the weights of different neurons or dropping certain neurons from the network.  And then you'll have, instead of a single estimation, you'll have a distribution of estimations made by all the permutations in the model.  And if those have a wide variance, then you have low certainty. And if they have very little variance, a very tight distribution, then you'd have very little uncertainty.  Learning curves can be created by adding instances or rows to a file or variables or columns to a file until accuracy peaks.  And at that point, you know that you cannot enrich the predictive performance by adding more instances or more variables.  By doing these things, you can determine whether aleatoric uncertainty is sufficiently low  for clinical application and determine whether model accuracy is sufficiently high for clinical application after ensuring that you have the right variables and enough of the data.  Another major problem is that models applied outside of their training environments and patient population could cause harm.  And we suggest technology readiness assessments and performing model stress testing with simulated data to mitigate this potential challenge.  Technology readiness assessments were first conceptualized after the Space Shuttle Challenger tragedy.  After this, NASA developed its own technology readiness assessments, and these have been adapted to scientific investigation as identified in this report in this BMC journal, Health  Services Research. Another major problem is that manual data entry requirements and additional work can drive clinician apathy towards artificial intelligence models.  Essential solution to this problem is to integrate automated decision support systems with existing clinical and digital workflows.  This is an example published in Scientific Reports by Ben Schickel and colleagues in which a deep learning model made continuous patient acuity assessments by taking SOFA  variables or the variables that are included in calculating the SOFA score and updating them continuously to measure the risk of hospital mortality.  On the top of the screen, you can see the changes in some of the SOFA variables, and on the bottom of the screen, you see the bedside SOFA, which is sort of a SOFA lookup table,  and it would be the application of static variable threshold cutoffs, which really didn't estimate a high risk for death until it was within about 12 hours of the patient passing,  whereas the deep SOFA model seemed to respond earlier and more accurately to changes in patient physiology.  Another problem is that artificial intelligence models cannot incorporate some subjective findings and the wisdom of experience that's accessible to clinicians.  And for this, we think it's critically important to preserve human intuition and decision-making  processes that are augmented by recommendations made by artificial intelligence models. This is discussed in greater detail by Eric Topal in one of his Nature Medicine publications  In the current state, we have interactions that are almost exclusively between doctors and patients.  With increasing use of artificial intelligence technologies, they could play a greater role in affecting clinical decision-making.  Eric Topal points out that it's unlikely that computers will ever play a dominant role.  He argues against a scenario in which computers would interact solely with patients, and we would endorse this perspective by Dr. Topal.  Finally, there's also some evidence that machine learning models, in many cases, don't actually make predictions just based on the variables available in the model.  Sometimes they're making predictions just based on the presence of a variable. For example, these investigators found that clinician-initiated data is a filtered representation  of patient physiology. And this is likely because clinicians tend to order tests that they expect to be abnormal or expect to affect their decision-making in some way.  And so simply the presence of the test being ordered is sometimes a more important predictor in even a very effective machine learning model than is the result of the test itself.  Finally, accountability for errors associated with artificial intelligence-enabled decision support remains a challenge.  For this, we suggest guiding the development and implementation of artificial intelligence models towards social benefit with altruism, creativity, and clinical expertise, which  are some very important human characteristics that remain inaccessible to machines.  For example, in surgical decision-making, decisions are often challenged by time constraints and uncertainty.  Doctors have only so much time to make a decision, and they're working under uncertain conditions,  often with incomplete information, and more information may become available  as patient physiology evolves.  Under these circumstances of time constraints and uncertainty, doctors and surgeons in particular tend to fall back on standard hypothetical deductive reasoning.  Unfortunately, under these circumstances, cognitive shortcuts can unmask implicit bias,  and that can adversely affect patient decisions.  Instead, we would propose artificial intelligence-augmented decision-making, in which case the surgeon is working with an AI model to produce more reliable,  effective decision-making.  Thank you for the opportunity to present in this panel.

Video Summary

Tyler Loftus, a trauma and acute care surgeon at the University of Florida, discusses the barriers to implementing artificial intelligence (AI) in critical care. One challenge is the lack of data standardization, which hinders the validation and dissemination of AI models across multiple centers. To address this, Loftus suggests using common data models or federated learning to share model information without sharing data. Another challenge is the mistrust of black box AI models. Loftus proposes implementing model interpretability and uncertainty estimation mechanisms to increase trust. Additionally, the talk explores the need to preserve human intuition in decision-making processes augmented by AI recommendations and the importance of accountability for errors.

Asset Subtitle

Quality and Patient Safety, Professional Development and Education, 2022

Asset Caption

This session will explore the potential for artificial intelligence (AI) to augment critical care. Speakers will introduce relevant concepts using terms and examples that are familiar to critical care professionals, describe state-of-the-art applications of AI in critical illness, identify barriers to real-time clinical implementation of these applications, and propose desiderata for ideal algorithms.

Meta Tag

Content Type Presentation

Knowledge Area Quality and Patient Safety

Knowledge Area Professional Development and Education

Knowledge Level Intermediate

Knowledge Level Advanced

Membership Level Select

Tag Evidence Based Medicine

Tag Innovation

Year 2022

Keywords

artificial intelligence

data standardization

model interpretability

uncertainty estimation

accountability

	Society of Critical Care Medicine 500 Midway Drive Mount Prospect, IL 60056 USA Phone: +1 847 827-6888 Fax: +1 847 439-7226 Email: support@sccm.org	Contact Us About SCCM Newsroom Advertising & Sponsorship DONATE	MySCCM LearnICU Patients & Families Surviving Sepsis Campaign Critical Care Societies Collaborative	GET OUR NEWSLETTER
© Society of Critical Care Medicine. All rights reserved. \| Privacy Statement \| Terms & Conditions The Society of Critical Care Medicine, SCCM, and Critical Care Congress are registered trademarks of the Society of Critical Care Medicine.