SCCM Resource Library-56-Clinician Prediction of Functional Outcomes Following Pediatric Critical Illness

Video Transcription

Hi, and thank you for attending. My name is Suzanne Gouda, and I will be presenting our study, Clinician Prediction of Functional Outcomes Following Pediatric Critical Illness.  Morbidity is increasing among PICU survivors. In our primary study, the Chicago Survivor Outcomes Study, we found that both morbidity  and mortality increased from discharge to three-year follow-up in a cohort of children who were admitted to the PICU at the University of Chicago.  Recent data also supports that health-related quality of life and functional status are important to families of PICU survivors.  When facing medical decisions in the ICU, families rely on clinicians for essential guidance, and communication around prognosis is a key aspect of this process.  There is a lack of evidence to support the accuracy of clinicians' predictions. Our objectives were to examine whether PICU clinicians could predict long-term functional  outcomes accurately when compared to an objective tool. We also evaluated the level of agreement between objective functional status, clinicians' predictions,  and parent perceptions. We hypothesized these would all be poorly aligned. This study is a secondary analysis of the Chicago Survivor Outcomes Study, our prospective  cohort study that originally described longitudinal outcomes of children after PICU discharge.  In this study, clinicians were asked on hospital day one to predict patient survival and, if applicable, the level of functional impairment at six months post-discharge.  And functional status scores, which I will describe on the next slide, were obtained from electronic health record review.  Parents of eligible patients then participated in a telephone interview at six months post-discharge  with an objective assessment of functional status and asked their perception of their child's functional status at that time.  In brief, the functional status scale score evaluates six domains of function you can see listed here, with a total score ranging from six to 30.  Lower scores represent normal function, and higher scores represent abnormal function. The original patient cohort characteristics for which the predictions were made are described  here. The patient cohort was about half male and half African American, which is an accurate reflection of our local PICU population.  Regarding clinical characteristics, about a quarter required mechanical ventilation and under 10% required vasoactives.  Clinician predictions were made in various roles on the medical team, including attendings, fellows, rotating residents, and bedside nurse and nurse practitioners.  Of note, we did combine the RN and NP responses to preserve anonymity, given a low number of NPs in our PICU.  In this graph, we are looking at clinician accuracy in predicting survival. We have each clinician role represented on the x-axis and the percentage of correct predictions  on the y-axis. We see that across all clinician roles, over 90% correctly identified patients who survived to discharge.  Now looking at patients with objectively normal function per the FSS score at six months post-discharge, we see clinician accuracy across all roles decrease.  We still saw that over half of clinicians correctly predicted normal function at six months.  Moving on to patients with mild dysfunction at six-month time point, we again see clinician accuracy generally decrease in less consistency between the provider groups.  Interestingly, the fellows performed best with 57% accuracy. Finally, in patients with moderate dysfunction at six months per the objective score, we  again see a significant drop in accuracy, with clinicians being incorrect over 50% of the time across all roles.  In this table, we look at agreement and diagnostic performances of clinician predictions compared to the objective FSS category at six months.  When assessing the strength of agreement overall, we see that all provider roles had only fair  agreement with the six-month FSS score, as defined by the kappa coefficients seen here.  No matter their role on the medical team, clinicians' predictions demonstrated low sensitivity,  specificity, and most notably, positive predictive values of less than 20%, a reflection of very  poor accuracy when dysfunction does exist in the patient population. However, the negative predictive values were much better, 85% or greater, for all clinician  predictions of six-month outcomes, a reflection that accuracy is much stronger in those with no dysfunction present.  Now moving on to the parents' perception, this graph compares reported parent perception in gray to the objective FSS category in blue.  We see that for those with normal FSS, only about half of those parents perceived normal status. Parent perceptions seemed best aligned in the moderate category.  Although some parents did perceive their child to have severe dysfunction, no child met objective scoring criteria for severe dysfunction.  There was poor agreement between the FSS at six months and parent perception of functional status overall, with a kappa coefficient of 0.16.  Now that we've looked at how clinicians align with the FSS and how parents align with the FSS, next we will look at how clinician and parents align with each other.  In this table, we then compared clinician predictions to parent perspective to evaluate for any improvement in the following performance measures.  We see, again, only fair agreement between the two groups across all clinician roles.  We do see a modest improvement in the positive predictive values, but overall, little difference of the other performance measures. Our study has a few limitations to note.  Given the logistic challenges associated with follow-up, our study has a relatively small cohort at a single center.  We also only enrolled English-speaking parents due to limitations of telephone follow-up. Overall, these factors may limit our generalizability.  We also do not have information on the varying levels or years of experience of providers, which may have an impact on prediction accuracy.  Parents and clinicians were not given a detailed reference guide to the categories for each level of dysfunction to anchor their assessments.  In conclusion, our results demonstrate that clinicians have poor prognostic accuracy with long-term morbidity following PICU hospitalization.  There was poor alignment between clinician predictions, objective functional status score, and parental perception.  These findings illustrate limitations in our ability to guide families in prognostic discussions.  Because parent and clinician perspectives may not be aligned, prognostic conversations may benefit from incorporation of objective scoring tools and use of a shared language,  a topic for future studies. We would like to give a special thanks to the following people, and thank you to all of you for attending.  Thank you.

Video Summary

The study presented by Suzanne Gouda focuses on the ability of clinicians to accurately predict long-term functional outcomes following hospitalization in pediatric intensive care units (PICU). The study found poor alignment between clinician predictions, objective functional status scores, and parental perceptions. Clinicians' ability to predict survival was generally accurate, but accuracy decreased when predicting functional outcomes. There was also limited agreement between clinicians and parents. The study suggests that incorporating objective scoring tools and promoting shared language may improve prognostic discussions with families in the PICU. The study acknowledges limitations such as a small cohort size and lack of information on provider experience.

Asset Subtitle

Pediatrics, Quality and Patient Safety, 2021

Asset Caption

The Society of Critical Care Medicine's Critical Care Congress features internationally renowned faculty and content sessions highlighting the most up-to-date, evidence-based developments in critical care medicine. This is a presentation from the 2021 Critical Care Congress held virtually from January 31-February 12, 2021.

Suzanne Gouda

Introduction/Hypothesis: Clinician prognostication impacts medical decision-making, but the accuracy of predictions of functional outcomes after critical illness is not well studied. We evaluated the accuracy of clinicians' predictions of functional outcomes compared to an objective tool and assessed agreement with parent/guardian perception of their child's functional status.

Methods: A prospective, observational cohort study at an urban, tertiary pediatric intensive care unit (PICU) was performed. Functional Status Scale (FSS) scores were obtained for patients â‰¤ 18 years old at baseline through electronic health record review. Parents were interviewed to determine child FSS at 6 months and 3 years after discharge. Clinicians were asked to predict the level of functional deficits after discharge. The accuracy of clinicians' predictions compared to FSS scores was assessed. Concordance between parent/guardian perception of their child's functional status, FSS scores, and clinician predictions was also assessed.

Results: One hundred twenty nine patients were enrolled, 77 of these patients had follow-up data available. More than one-third of these children had worsening of their functional status or died by three years after discharge. While more than half of clinicians correctly predicted normal functional status, the accuracy of clinician predictions of mild, moderate, or severe dysfunction compared to FSS scores was poor at both time points, with kappa coefficients <0.25 and positive predictive values (PPV) <20%. Clinician level or training did not correlate with improved accuracy. Parent/guardian perception of their child's functional status poorly aligned with measured FSS scores and clinician predictions with all kappa coefficients <0.25. PPVs of predictions by clinicians compared to parent perceptions was modest (< 45%).

Conclusions: Our findings demonstrate clinicians inaccurately predict long-term dysfunction and that parent/guardian perceptions of functional status differ from both FSS scores and clinician predictions. Clinicians' poor prognostic ability and the lack of alignment with parental perception of functional status complicates the ability to guide families regarding complex medical decision-making in the PICU and long-term outcomes.

Meta Tag

Content Type Presentation

Knowledge Area Pediatrics

Knowledge Area Quality and Patient Safety

Knowledge Level Advanced

Membership Level Select

Tag Scoring Systems

Year 2021

Keywords

clinician predictions

functional outcomes

objective functional status scores

parental perceptions

prognostic discussions

	Society of Critical Care Medicine 500 Midway Drive Mount Prospect, IL 60056 USA Phone: +1 847 827-6888 Fax: +1 847 439-7226 Email: support@sccm.org	Contact Us About SCCM Newsroom Advertising & Sponsorship DONATE	MySCCM LearnICU Patients & Families Surviving Sepsis Campaign Critical Care Societies Collaborative	GET OUR NEWSLETTER
© Society of Critical Care Medicine. All rights reserved. \| Privacy Statement \| Terms & Conditions The Society of Critical Care Medicine, SCCM, and Critical Care Congress are registered trademarks of the Society of Critical Care Medicine.