Good afternoon, friends. Thank you for joining us for our session on Viral Infection Respiratory Illness Universal Study COVID-19 Registry. We're very glad that you could join us. We have very interesting topics that will be covered today from very eminent speakers in the field, not just from the United States, but across the globe. You will listen to people who will be discussing interesting results from our first iteration of virus registry. We will be talking about patients, not just on the adult spectrum, but also in the pediatric age group. My co-moderator is Ms. Razan Almoody, and we will be with you throughout this whole session. If you have any questions or comments, please post them in the chat box. We will be managing that as we go along. We will be happy to chat and do breakout rooms for you as well, as and when the questions arise. So sit back and enjoy this interesting session. Thank you. Hello and welcome. My name is Rahul Kashyap, and today I'll be opening our session on VIRUS, which stands for Viral Infection and Respiratory Illness Universal Study COVID-19 Registry. Those who are doing live tweeting and using social media, please feel free to tag us and ask questions during real time. This registry is supported by Mayo Clinic, as well as Gordon and Betty Moore Foundation and Jensen & Jensen Research and Development. The registry is pending some grants from CDC and FDA as well, and myself, I received grants from NHLBI for unrelated clinical trials. Let me give you an overview of, for next eight to ten minutes, what we'll be talking. So I'll be introducing the registry, those who are not familiar with it. What we did with REDCap capture, which stands for Research Electronic Data Capture, how much progress we have made, show you a sneak peek of a dashboard, which is publicly available, and then we will give you a sneak peek of a critical care dashboard and data explorer, which is not publicly available, and some of the collaborations who will follow after me in the next half an hour. So let's get started. VIRUS is a collaboration of international investigators in the field of critical care medicine, and these efforts are managed by Society for Care Medicine's Discovery Clinical Research Network. This is a cross-sectional observational study of all eligible COVID-19 patients who got into the hospital, both adult and pediatric. The study is registered on clinicaltrial.gov as well as IRB approval. There is no intervention plan as drug or any devices. The data is totally de-identified and only enrolled patients who are either on the floor or in the ICU. So we claim it to be one of the most accurate and generalizable near real-time data on COVID-19's practice variations and outcome, and very quickly it's becoming the largest U.S. registry of hospitalized patients. How it come about? We were able to take the study in almost three weeks' time back in March 2020 last year, and this gives you our timeline where the idea was conceived before the pandemic was announced. Quickly, we wrote the proposal. First IRB was submitted. Some of the social media contacts were created. The REDCap data collection form was created. Then send out the approval to other sites as well, and then very quickly, even before April 1st last year, we were able to enroll our first 10, 15 sites with 10 patients as well. Feel free to reach out to us if you have. This has been published in Critical Care Explorers. Dr. Walkie, one of the co-PI, is the primary author on this publication. We talked about how we were able to do the data harmonization, so we used the REDCap online data collection tool. It's a little busy slide, but to give you an idea that Core Data 1, 2, and 3 are one time point. They include some of the administrative data, testing, trials, demographics, symptoms, and so on and so forth. Then we have daily data collection for up to 14 days, and then day 21. That doesn't mean that all day of data will be available. Patients will be coming to different days to ICU, so we have fields where they can mention about which hospital day or which ICU day data they are collecting, the hospitals are collecting. Then we have enhanced data in terms of vitals and SOFA and PILOT for pediatric as well, APACHE, and some of the data for PRISM and PSS, which are some of the scores for pediatric as well. We have our day 28 instrument. We are noting down the outcome of the patient, not necessarily on day 28, but any time if they have died in the hospital or got discharged from the hospital. There is an associated SOP, standard operating procedure with that, and now we are mapping our data points with OMOP modeling to make sure that there's a standard definition and certain data points are there in the study. Weekly enrollments graph is presented here, and it gives you roughly last eight or seven or eight weeks as well. You would see that almost every week we're making progress to adding 500 to 800 patients every week from more than 250 institutions. At the time of recording of this slide, we are near 50,000 patients. By the time this is displayed, we will have more than 50,000 patients enrolled so far from 40 U.S. states and 24 countries. I would like to present to you the dashboard. If you have not seen it, please go online. You can just type in sccmcovid19.org. That takes you to this nice dashboard, which has both adult and pediatric versions, and then we have some charts listed there now for last several weeks. You can click on world locations and see the presentation in 24 countries, almost from Japan, Russia, India, Pakistan, Saudi Arabia, Iran, Middle East, and Africa as well, and then you have Egypt, a couple of European sites, South American sites, Central American, Hawaii, Puerto Rico, U.S., and Canada as well. This gives you the distribution of the state's contributing data. Some hospitals from each state contribute data. We have up to 40 states now, and not to mention Hawaii and Puerto Rico are contributing data in that regard. This is a sneak peek of that dashboard as well. It kind of shows you the distribution in terms of who sought help in terms of gender, race, ethnicity, and so on and so forth. What type of support these patients are requiring? Roughly out of 16,000 patients, 30% patients require mechanical ventilation. This is full-blown intubation as well. Then 12% requires non-invasive, and then high-flow nasal cannula and so on and so forth. Then what happened to these patients in terms of how long they're requiring some of these interventions, how long they are staying in ICU, and what is their status, ICU discharge status, as well as hospital discharge status. Now, let me show you one of the internal dashboards we are working on. This is critical data export in collaboration with Inference. It kind of let the investigators allow to dissect the data and slice it. We are hoping that the participating institutions can create their own cohort for comparison without revealing other institutions' identifications or privacy for that part. It is still in the working, just a sneak peek. Those who are taking pictures for social media, I would suggest to not use these pictures for especially these slides. You can display the complication, comorbidities, medication taken, and so on and so forth. It goes down a little bit deeper in that regard to see what is happening with these 50,000 patients, and then these numbers are growing every day. Let me talk about the collaboration as well. Very early, some of the partners, either from society, from foundations, from as well as the other groups who are doing the registries, we were very quickly able to partner with them. Those here which were marked in red with asterisks, these are already a concrete partnership which has been written, and we are moving forward with that. Point and click here, American College of Radiology, ISRAC, and FDA. Out of these three, you will be hearing from my colleagues, Dr. Sri Murthy, Ms. Christina Davis from ACR, representing ISRAC, and then Ms. Heather Stone from FDA about these collaborations. Others' collaborations with the American Heart Association, SOAP, and Infectious Disease Society of America are still in progress, and we hope to add their logo once those collaborations are set in stone. So what's our global story in some of my concluding slides? So we are sitting at roughly 50,000 unique patient data, and numbers are there more than 125,000 patient days, more than 8 million data points, 3,000 volunteers who are volunteering their time from different, different institutions at SCCM, at Boston University, at Mayo Clinic, and other institutions in the U.S. and outside. More than 130 ancestry ideas have been submitted, and then I think more than 45 have been approved. Four manuscripts are either published, accepted already, and then at least a dozen manuscripts are in preparation. So, and this number is only going to grow as well, so that's a huge testimony to our collaborators, our partners, as well as our patients who let us collect these data to learn about the disease. If I give you a take-home message of what would be the, what, what to describe as a virus or COVID-19 registry in one sentence, so this sets an example of a rapidly deployed international pandemic registry that seeks to provide near real-time analytics and information of intensive care treatment, hospital care patterns, as well as outcomes for COVID-19 patient. I'll leave you with our contact information and some other social media hashtags as well as handles. Please feel free to reach out to any of us, my co-investigators, Dr. Kumar and Dr. Waukee. Please do visit our web page, which gives you information about the registry and how to enroll for that as well, and if you're doing live tweeting or LinkedIn or Facebook, please use appropriate hashtags and handles. We'll be happy to take questions at this point. Thank you. Hello, my name is Christine Davis and I'm a senior director at the American College of Radiology. I'm here today to talk about my society's collaboration with SCCM on the virus registry. I have nothing to disclose. The American College of Radiology is a nearly 100-year-old membership organization. We are a professional society predominantly for radiologists, but also for radiation oncologists and medical physicists. The college's efforts are centered around a half a dozen primary programs in the areas of research, education, government relations, and others. The core purpose of the ACR is to serve patients and society by empowering our members to advance the practice, science, and profession of radiological care. The Center for Research and Innovation is where I'm based and is the college's research program. It's entirely housed in Philadelphia, Pennsylvania. We've been conducting research for over 50 years and we are about 150 employees strong dedicated to clinical research. We don't create the science at the CRI. We facilitate the success of others in the areas of interest to them. We do that through collaborations with research, PIs, and commercial companies supported by both federal and state grants, commercial contracts, and partnerships with other societies like SCCM. You can think of the Center for Research and Innovation like a full-service clinical contract research organization. We have staff expertise in everything from medical and technical writing to pursue grants and author commercial contracts, all the way through to statistical analysis of the data that we do collect on our research projects towards the goals of regulatory authority submissions or publishing. Our services are modular, meaning partners can plug in and out where needed, what they want to outsource to us, collaborate with us where they'd like to, and maintain other scopes of work outside of the CRI. In response to the pandemic earlier this year, the ACR stood up its own COVID-19 research program. The ACR stood up its own COVID-19 registry called the COVID-19 Imaging Research Registry. When COVID-19 came, it was clear that creating a searchable data platform housing both clinical and imaging data to inform and empower radiologists during the pandemic was going to be crucial. The ACR Registry is a joint effort across the ACR Center for Research and Innovation and the ACR Data Science Institute. We also collaborate closely with the Society of Thoracic Radiology. The Registry's aim is to aggregate diagnostic imaging data and clinical information in an effort to translate new evidence-based research into clinical practice against the disease. Participating sites can contribute demographic information, clinical data on signs and symptoms, diagnostic images, laboratory test data, and outcomes data for U.S. patients tested for COVID-19. And it was actually through outreach efforts for our own ACR COVID-19 registry that I was introduced to the Society of Critical Care Medicine team and learned of their virus registry effort. ACR is partnered with the Society of Critical Care Medicine to evaluate the safety and efficacy of COVID-19 practices through the virus registry. ACR will collect diagnostic images for virus COVID-19 enrolled patients and then establish links between those images that we collect in our repository and the clinical data housed by SCCM at Mayo Rochester. Archived images will then be made available to participating sites through combined dashboard reporting. It's these types of collaborations that are very important as we continue to monitor and work to predict the course of the pandemic in the months ahead. Merging both clinical and imaging data is critical and virus is laying the groundwork for future collaborative efforts that really harness the power of bringing data together in an accessible model. The ACR accomplishes image collection on virus through an ACR developed tool called TRIAD and that's an acronym that stands for the Transfer of Images and Data. TRIAD is a data exchange platform developed by the college and we use it widely across all of our programs where data and cases are being collected. TRIAD supports image and data sharing and anonymization between imaging centers and the ACR. If you want to learn more about ACR's role on the virus registry, there's a few ways that you can reach out. You can reach me and my team through the virus leadership team. ACR has also stood up a dedicated email for the virus registry that I list here and I also include my own personal contact information. I want to thank SCCM for this time and forum today and I wish everyone a great Thank you. Hi, I'm Srinivas Murthy from the University of British Columbia in Vancouver, Canada, and I'll be speaking about building international linkages using the virus framework and building that linkage with other international collaborations. Specifically in this context, Itharic, which is the International Severe Acute Respiratory Infection Consortium, which is the International Severe Acute Respiratory Infection Consortium. I do have some disclosures. I'm on the board of directors of Itharic and also a co-chair of the Clinical Characterization and Management Committee of the World Health Organization's COVID-19 response. I also receive salary support from the Health Research Foundation and Innovative Medicines Canada as a chair in pandemic preparedness research. So a brief history of where we're coming from. So after 2009, after the H1N1 outbreak, there was a collective movement to better improve research readiness and part of that was better to scale up observational research to make sure that we have real-time pandemic-ready research ready. So the goal was overall to improve research readiness for that future pandemic outbreak. And so Itharic was born and with it a project of Itharic, which is called Sprint Sari, which is a short-period incidence study of severe acute respiratory infection. The goal of this study was effectively to establish common data tools and elements that could be scaled out urgently and emergently in the context of a pandemic, having pre-existing approvals and pre-existing infrastructure for research and data collection. And so what it did, it had existing ICU registries, which collected standardized data elements. You could directly contribute to a common data collection tool as a member of a network or as an open-access tool. You could have a localized instance of this REDCap database that you can collect your own data infrastructure. And so for that first component, some existing ICU registries, we provided open-access tools so that other registries could use these tools for ultimate harmonization. Also, so for example, for this pandemic, we've collaborated with folks and given them our tools. This is the ECMO CARD registry, which is another global ECMO or critical care specific registry that has used these tools to sort of leverage specific questions and answer them accordingly. Alternatively, there was localized instances of databases. And so countries around the world took the data variables that were standardized in terms of their language and had their own localized instance, whether at a hospital region or national level, to make sure that they had real-time data to monitor their local outbreak at the clinical level. Or you could collectively, directly contribute to a platform as a member of a network. So for example, if you have clinical data that you had to collect and you could enter that globally. So for coronavirus, given this multiple years of readiness work that we've been working on over this past number of years, coronavirus provided that situation where these tools needed to be operationalized. And they were subsequently operationalized in places like the United Kingdom, which initially characterized many thousand patients in hospital with COVID-19, providing a framework for baseline data. The original China reports, many of them use these tools to report their initial experience with this disease. Subsequently, other countries, France, United States, small localized instances of the database with standardized tools provided an ability to harmonize elements across regions and use high quality data across different parts of the world. Similarly, randomized trials could be embedded just by integrating a randomized element and standardized outcome assessments. The original Calitra trial performed in China, for example, used these tools to integrate a randomized trial for COVID-19. And so up till now, this has been scaled out across the world with a large number of countries that have subsequently participated and entered data, either into the global data set or into a localized instance of a database. In terms of a timeline, the first case obviously isolated late December of 2019. As of August 2020, it was the 100,000th record. At the time of recording in December, upwards of 200,000 records, which have provided a great deal of information to the larger community about practice patterns, epidemiology, microbiology, and outcomes. For example, Sankey plots and probabilities of some deaths and differential outcomes based on gender or comorbidities and prognostic scores have all been populated by these reports that are publicly available. Obviously, linking up with other registries around the world, such as virus, has been a goal as well to make sure that questions asked could be as useful as possible for all stakeholders. The historic team has long had a strong relationship with the Discovery Network at SCCM, with a number of collaborators and collaborations over these past number of years as part of pandemic preparedness work, and virus provides a further opportunity for that. Obviously, with more patience and the need for more power to better analyze and integrate new questions, these relationships and collaborations for harmonized data and harmonized analyses become even more crucial and more important. And so, as part of this work, mapping shared elements across the two registries is absolutely crucial, finding common questions across the registry participants, including collaborators and policymakers to make sure that the questions asked are responsive to the needs of clinicians around the world, and then building large-scale collaboration. And I think that large-scale component is absolutely crucial. I think we've learned over the course of these past 12 months of this pandemic that without collaboration that is big and without collaboration that's large, we won't be able to answer the appropriate questions in a timely way. And so, building these collaborations across borders, across regions, and understanding the logistic and implementation hurdles to build those large-scale collaborations is absolutely fundamental for our pandemic response activities currently for COVID-19, and then building that to move forward across future years, future pandemics, and future research questions. And so, for those initial sort of harmonization and standardization for those first analyses, there are initial analyses underway incorporating the large power across the tens of thousands of patients across the number of registries to make sure that the questions asked can be as robustly answered as possible. Obviously, one of the challenges is ensuring that all legal and implementation hurdles are overcome, and I think those are mostly achieved now, and then answering a question in a scientifically rigorous way. The benefits of this collaboration, obviously, with reasonably rare conditions within the COVID-19 framework or rare outcomes, ensuring that enough patients are present to answer those questions is absolutely crucial. And then building towards the future, I think we've all acknowledged that ensuring that a common data set for elements for acute disease, specifically, and sorry in this example, but any acute disease can be scaled out to answer questions and have a large-scale registry. I think this capacity and willingness for sharing that's been exemplified by this pandemic needs to be pushed forward so that more data sharing across international networks and international collaborations is the rule rather than the exception. I think we can continue to push those boundaries and we can move it forward. So, to conclude, pandemic research is hard and requires coordination and collaboration. Obviously, power in numbers is key and making sure that we have as many patients as possible participating in these research that we can learn as much as possible and make it as generalizable as possible would be the goal. Thank you so much for your attention. Good afternoon and thanks for having me. My name is Heather Stone and I'm a health science policy analyst at the Student Drug Administration for FDA. I'm here today to tell you about CARE-ID and our partnership with the FCC and Mayo Virus Registry. CARE-ID is an internet-based data repository that has been developed as a collaboration between the FDA and NCATS at NIH, the National Centers for Advanced and Translational Sciences. It was also developed with the support of professional societies such as IDSA, the Infectious Disease Society of America, CDC, and the WHO. The point of CARE-ID is to give the global clinical community the opportunity to report novel uses of existing drugs for patients with difficult-to-treat infectious diseases through a website, a smartphone, or another mobile device. You can visit CARE-ID at care.ncats.io or by downloading the CARE-ID app, and it's available to all healthcare professionals globally. There are three main features of the platform, which include the ability for users to enter a case report and view others' cases, for them to explore the cases shared by other clinicians as well as those from the published literature, in addition to clinical trials and other data sources. And finally, there's a discussion forum where users can contribute their treatment experiences, ask questions, and get feedback from other clinicians globally. We're very excited about a new development, which has been a partnership with SBCN Mayo's virus registry to automate the data extraction from EHRs and other registries, and this will enable us to overcome a significant challenge with the platform, which was the ability to get clinicians to submit cases. So in order to do this, this will enable us to share data and make a limited but high-value set of case data on drug treatments for COVID-19 initially and eventually for other diseases completely openly available via the CARE-ID platform. We'll be working with Johns Hopkins to develop an automated data extraction tool that will enable us to extract data from the virus registry into CARE-ID, as well as from a number of other sources, and to provide this technology for further adoption by virus sites to reduce the burden of case sharing and make it easier for sites to contribute their information. Our goal is to facilitate the development of a sustainable infrastructure that will enable clinical care and clinical research to be seamlessly integrated. So where are we now? Right now we have a working group of SBCN Mayo virus experts, as well as leading clinicians from Hopkins, Emory, UCSF, and others who have been adapting the CARE-ID case report platform that was developed specifically for COVID-19 and adapting it for EHR and registry data extraction. Because obviously the information that one could potentially gather from an EHR is different than that which you would ask a clinician to enter as a case report. In addition, we're extending the partnerships to include other health systems, additional registries and programs, and this is all being facilitated by our partner, the Critical Path Institute, which is the convener of our public-private partnership, the Cure Drug Repurposing Collaboratory. So this has been a wonderful expansion of that partnership. To tell you a little bit about the infrastructure that we're planning to build, the Cure Drug Repurposing Collaboratory will function as a partnership coordinating center, which will be at the heart of the organizational structure and will collaborate with FDA and NCATS NIH. NCATS will also serve as the data coordinating center, leveraging the existing infrastructure that they've built with the N3C program to capture COVID clinical data and expanding upon that infrastructure. We will have four primary institutional partners, of which the SBCN Mayo virus registry is one. The others include Johns Hopkins School of Medicine, the University of California, San Francisco School of Medicine, and Emory School of Medicine and their associated healthcare systems. All told, we expect to have about 350 different sites, including sites in both the U.S. and internationally. And one of the things we're most excited to partner with the virus registry on doing is expanding our international presence and hope that providing this mechanism for sharing data in an automated fashion, as well as having an option to share a more limited set of information, will make it more accessible for global stakeholders to participate. Our hope is that by June of 2021, we will have included 100,000 patients, de-identified data at case reports, and that that information will then be openly available on CureID in partnership with Hopkins and the SBCN Mayo virus registry. And we estimate including at least 250,000 cases within two years in collaboration with all of our partners. And as I mentioned before, at the heart of this initiative is the idea of developing a sustainable infrastructure for capturing and sharing this kind of information with both the virus registry, as well as the open source edge tool for sites to use to automate the extraction of this critical data from their electronic health records. We're delighted to be working with the SBCN Mayo virus registry and are thrilled to have this collaboration and look forward to more work to come. Thank you. Hi, my name is Dr. Juan Pablo Domecq. I'm an assistant professor at Mayo Clinic in the Division of Nephrology and Hypertension and Critical Care Medicine. I'm here on behalf of the Virus Society of Critical Care Medicine Discovery Network, and I'm here to present outcomes in COVID-19 patients from the virus registry. For the ones that are very active on Twitter, please feel free to share our findings in your tweeting and other social media. I have no conflicts to disclose. So what are we going to talk today? It's this first observational study that is coming out from the virus registry that was focused on outcomes in patients with COVID-19 that require an inpatient admission. We're not going to focus on methodology because we don't have the time and there is another session that is going to explain this. We're going to, I'm going to show you the eligibility criteria and the flow of the patients that we end up enrolling. We're going to talk about how we stratified our outcomes and interventions. And then talking about outcomes, we decide to choose three outcomes, mortality, length of stay and distant home. We feel that these outcomes are very important for patients and also for clinicians, because as you all know, when a patient gets admitted to the hospital, they have three main questions. If they're going to die during the hospital admission, or if they're not going to die for how long they're going to stay in the hospital, and if they're going to be able to go back home from the hospitalization. Then when we were practicing in, during the pandemic, we noticed that there was a significant amount of a practice variation between hospitals. And we were wondering if these practice variations were leading to a mortality variation. So we felt that it was very important to evaluate the mortality variation and try to come up with some explanations if this was significant. And then I'm going to talk about the implications of our findings in research and in practice. So this is the eligibility criteria and the flow of the patients that we end up analyzing in our observational study. As you can see here, when we download the data, there were 49,000 patients already enrolled in the virus registry. Some of them were newly enrolled and they were not having any outcomes yet enrolled in the registry. We end up assessing for eligibility 29,000 patients, and we exclude 9,000 patients because were either not adults, or they did not have a laboratory confirmed COVID-19 infection, or their discharge status was not yet recorded. At the end, we end up analyzing 20,608 patients, and we divide these patients in three main categories. The patients that they were requiring no organ support, that there were like 15,001 patients. The patients requiring invasive mechanical ventilation, there were 5,005 patients. And a small percentage of these patients, 602, they require either vasoactive drugs or acute renal replacement therapy without the need for invasive mechanical ventilation. These patients are very unique, and probably they have specific indications why they are requiring this organ support without requiring invasive mechanical ventilation. So we decide that probably it's not going to be very useful for all of you to really know what's happening with these patients. In terms of the stratifications, we divided our population in H categories, and then we focus on different types of organ supports, and we present the outcomes that I already mentioned based on these different types of organ supports. As you know, this is an observational study, so each of these categories are not randomly selected. And all of these, being part of these categories are because they were some selection. For example, patients that they were, do not resuscitate or do not intubate, they were not part of the invasive mechanical ventilation for obvious reasons. And patients, for example, that they go for ECMO, they're highly selected. So some of the outcomes that the ECMO patient has is because they were selected for successful. So they were either having only single organ failure, or they were having not many comorbidities that they made it appropriate to receive ECMO. The same for invasive mechanical ventilation, as I already mentioned. In terms of the organ support therapies spread, as you can see here, the 5,000 patients that they were requiring invasive mechanical ventilation, around 1,700 of them, they were requiring only invasive mechanical ventilation. 2,000 of them, they were requiring invasive mechanical ventilation and vasoactive medications. Around 650, they require these two intervention plus acute renal replacement therapy. And we have the patients that they went for extracorporeal membrane oxygenation were around 390 patients. In terms of the mortality, as you can see here, we're not trying to present this data to try to prove that all their patients have high mortality or patients that they were receiving more than one organ support, they have high mortality. What we're trying to present here is accurate recent estimates on patients with COVID-19 to help clinicians and patients to know what to expect when somebody is admitted to the hospital and when somebody is requiring some of these interventions. As you can see here, patients that they were young and they were not having any organ support, the mortality is less than 1%. When the patients are getting older, of course, the mortality goes high and probably can go high as to 23% when the patients are not having any type of organ support. Of course, probably a lot of these patients were having the status of do not resuscitate and that's why the mortality is high. But when you add a level of invasive mechanical ventilation, you can see that the mortality goes up and mortality can go as up to 40% in these patients. When you keep adding other type of organ supports and mortality keep growing to the point that the patients that are requiring invasive mechanical ventilation, vasopressants, and renal replacement therapy, they have around 70% chances of dying during that hospital admission. This is very important for clinicians and for patients to discuss before considering these type of organ support because they need to be aware that the mortality is pretty high. As you can see here, the mortality is very high in the age of 75, almost 80%, which is very important to be sure that patients and family are aware of these situations. The overall mortality in our cohort was around 19%. We were involving patients that they went to the ICU and also patients that they went to the wards. In terms of daycare home, also a very important outcome. As you can see here, the chances of going home if you are young and you don't have any organ support are higher than 90%. Overall, if you don't have any need for organ support therapy, the chances of going home is higher than 70%. As you can see here, the patients that they were going for ECMO, their chances of going home was lower than the overall chances of going home from the cohort. It probably goes they need a lot of super acute rehab or acute rehab and do it to the prolonged hospital stay. As you can see here, the chances of going home for patients that they were severely ill with multi-organ failures are less than 10%, as you can see here. In terms of a hospital length of stay, we know that the patients that they are not requiring any organ support, the length of a stay is around like a week to six days. When the overall length of a stay for the cohort was around a week. When we are adding organ support therapies, the length of a stay increases. As you can see here, and we're talking about a hospital stays from two weeks to three weeks to almost a month, especially in the patients that they are requiring ECMO because we know that ECMO is a very complex, challenging and labor intervention to perform and it's time to be successful. So, we're dealing with some prolonged length of a stay for patients that they are requiring ECMO, another type of organ supports as well. In terms of the mortality variation, as I mentioned, we plot all the hospitals that they were enrolling patients on invasive mechanical ventilation and the mortality variation was from 21% to higher than 75% in hospital they were having higher mortality. So clearly a very high mortality variation across hospital. So what this means? What this means is that if a similar patient presenting to a random higher mortality hospital versus presenting to a random lower mortality hospital had a 1.79 odds of dying. So this is very significant and we adjust these results to age, race, ethnicity, gender and comorbidities. As well, we did a sensitivity analysis on SOFA score and hospital case volumes and these sensitivity analysis show that there was no differences in the availability. What we found is the country where the care was provided was playing a minor role in the variation of the mortality, but the majority of the variation was not explained by these characteristics. So there was a lot of hospital mortality variation left unexplained in our analysis that is going to call for further research and some other implications. In terms of implication for practice from our estimates, as I already mentioned, we are providing novel prognosis estimates that they can be used at the patient and clinician level to have probably a better informed decisions and also for a clinician to paint a good, clear expectations for patients and family members when they are admitted into the hospital with a COVID-19 infection. The hospital levels, I think that leadership and hospital administrators can take these findings for resource allocation and making sure what to expect when patients are getting into the hospital. In terms of the mortality variation, clearly this calls for an improvement in some of the practices in some of the hospitals and also kind of review what is working and what is not working in terms of taking care of patients with COVID-19. In terms of implications for research, so this is a live registry, this is only the first observational study from this. We have several ancillary studies underway and now we have more than 50,000 patients already enrolled in the registry. So we're going to keep asking questions based on this analysis and other analysis that we have been doing and try to answer the questions using all our patients. One of the questions that we have is that we're going to explore why we have such a high interhospital mortality variation and we, of course, this calls for quality improvement projects but also for research projects to try to identify different type of characteristics that are introducing this mortality variation. Because the virus registry is an international registry, we're going to have international collaboration and research projects that is going to be extremely exciting and important to help patients and clinicians in order to help the COVID-19 pandemic. We need to thank all the patients and their family members to let us help them and also help other patients using their data and sharing their data with us in the virus registry. From our team at Mayo Clinic, I need to thank Dr. Kashyap, Dr. Gayak, Dr. Lau, Dr. Gonzal, Dr. Taken, and the whole team that it was very, very collaborative environment and we really were able to pull this through. Also the team from the Boston University, Dr. Waukee and Dr. Sheldrick were incredibly helpful in analyzing this data and helping us also with the manuscript writing. In terms of the SCCM team, Dr. Kumar, Mrs. Bowman, and Dr. Greg Martin were very helpful with their advices on writing the manuscript that we're presenting as well. From the virus team, Dr. Scott Molesta was very helpful in different stages of the project. And of course, we cannot thank enough to our virus registry collaborators all around the world because the registry wouldn't have been able to exist without their help. So thank you very much and have a good day. Hi there. I'm Alan Waukee from Boston University School of Medicine. I'd really like to thank the Society of Critical Care Medicine for the invitation to speak today. What I'll be talking about is COVID-19 practice variation across the USA and beyond. I'm an Associate Professor of Medicine and Pulmonary and Critical Care Clinician at Boston University School of Medicine. I have a few disclosures. The most pertinent disclosure is that I'm a co-principal investigator on the SCCM virus registry. The overview of what I'd like to discuss today includes first a discussion of the rationale for evaluating practice variation. So first I'll introduce why we study practice variation. Then we will discuss three specific groups of practices. First, use of high flow nasal cannula and non-invasive ventilation for patients with COVID-19. Next, use of repurposed or experimental medications for patients with COVID-19. And finally, use of therapies for ARDS and refractory hypoxemia within the virus registry. Finally, we'll discuss where all of these results, looking at practice patterns within the COVID-19 virus registry, where all of these results will lead for directions for next studies and follow-up studies within the virus registry. So why should we be talking about practice variation? Why study it at all? Well firstly, it's really helpful to know what is the epidemiology of practice. So by that I mean how do different groups approach similar patients with similar problems in different ways? We all think that our way that we practice is the only way until we see how the hospital across town or across the country would manage a very similar patient very differently. These large idiosyncratic variations in practice can actually be leveraged for research purposes to do comparative effectiveness studies, to help explain outcome variations like presented in the talk about outcomes for COVID-19, and also to help show where there might be equipoise for pragmatic clinical trials. Evidence where evidence-based practice is underused can then be identified for quality improvement interventions to try to improve practice. And then last but not least, it's helpful to understand trends in uptake and adoption and de-adoption of different practices to help inform what might be successful implementations and de-implementation strategies. The first group of practices we will evaluate is high-flow nasal cannula and non-invasive ventilation use for patients with COVID-19. These practices were chosen because they are commonly used to support acute respiratory failure prior to COVID, and there was some controversy early on in the pandemic. Some people expressed concerns for an increased risk of infectious aerosolization with use of high-flow nasal cannula and non-invasive ventilation. Ultimately, guidelines recommended use of high-flow nasal cannula, plus or minus using continuous positive airway pressure in COVID-19 acute respiratory failure, but usage patterns are unclear. And so for this reason, we leveraged the virus registry to look at practice patterns. We started with 38,466 patients in the virus registry who had a positive COVID-19 assay and then had to exclude patients that were not adults. We're looking at adults in this study, patients who we did not have discharge status on, patients who were not requiring or receiving any supplemental oxygen. And then we excluded patients from hospitals that really didn't have enough data for precise estimates. So we excluded hospitals with less than 10 patients, and we're left with about 9,500 patients included in this analysis. And we found about 60% of patients received nasal cannula or face mask without other modalities of oxygen delivery. 20% received high-flow nasal cannula, 9% received non-invasive positive pressure ventilation, and 8% received both high-flow nasal cannula and non-invasive positive pressure ventilation. Among those who received each of these oxygen delivery methods, about 5 to 10% died without receipt of invasive mechanical ventilation. And this is the same percentage, the same group of patients who had limitations on care or do not intubate orders. Next we found that about 40 to 50% of patients required invasive mechanical ventilation. And that amongst those who required mechanical ventilation, again about 40 to 50% ended up having in-hospital mortality and that is similar to the results that you will see with the outcomes analysis. But in total about 40% of patients who received high flow nasal cannula or non-invasive mechanical ventilation survived without requiring invasive mechanical ventilation. What did the patients in this study look like? The average age was 62, 43% of patients were women, about 50% of patients were white, 30% self-identified as black, and then 5% Asian and 15% other races, and 25% had Hispanic ethnicity. 92% of patients were from the United States. Patients had a variety of comorbidities. Here you can see the top most common comorbidities. 12% of patients did not have a full code status and had some kind of do not intubate or resuscitate order. The average SOFA score on ICU admission was 4 and then 57% of patients had an ICU stay. Most of the patients were from the pre-July period, the early pandemic time period. What did we find? We looked at case mix adjusted variation in use of high flow nasal cannula and non-invasive ventilation. The data I'm presenting to you is data from about 3,500 patients that met additional inclusion criteria including that the hospitals had the month of admission and entered both patients who were admitted to a ward or an ICU. These analyses that I'm presenting to you were adjusted for age, race, sex, country, comorbidity, SOFA score, time of the year, code status, and whether they were in the ICU. So what we found was that use of high flow nasal cannula and non-invasive ventilation increased over time. So pre-July versus post-July, there was a 27% higher odds of getting these post-July. And we found wide variation across hospitals and use. On the x-axis here, you can just see hospitals de-identified ranked according to their risk adjusted use of high flow nasal cannula and non-invasive ventilation. In the y-axis, you can see the percent of patients at each hospital who received these interventions. And what you see is that some hospitals never used these interventions and up to 70% of patients within this hospital using the intervention. And so there's a wide range of use across hospitals. And in our model, we found that about 27% of the variation in use of high flow nasal cannula and non-invasive ventilation was due to the hospital you were admitted to. And what that means is there's about three times the chance of getting high flow nasal cannula or non-invasive ventilation if you were randomly admitted to the higher use hospitals as compared to the lower use hospitals. And that's accounting for patient characteristics. We did similar analyses looking at use of invasive mechanical ventilation. We found that there was a non-significantly lower risk of having, receiving invasive mechanical ventilation if a patient was admitted to a higher use of non-invasive mechanical ventilation hospital. We found no change over time in use of invasive mechanical ventilation and much less variation where hospitals explained only 9% of the variation in use of mechanical ventilation. So in summary for high flow nasal cannula and non-invasive ventilation, we found wide variation in use of these modalities across hospitals that was unexplained by patient characteristics. We found 40% of patients were treated with high flow nasal cannula or NIV, non-invasive ventilation survived and did not require invasive mechanical ventilation. That mortality of invasive mechanical ventilation was not higher for patients treated previously with high flow nasal cannula. These results are limited by some unmeasured confounding within our analyses, some missing data and the fact that this was a volunteer registry and some data might not have been uploaded. We also looked at repurposed use of experimental medications. Multiple medications were used empirically early in COVID-19 including corticosteroids, anti-interleukin 6 or interleukin 1 inhibition, hydroxychloroquine and remdesivir, which were the most commonly used repurposed medications within our registry. So we looked at preliminarily at patterns of use and implementation of these medications to inform future work. We used the same cohort as for the high flow nasal cannula non-invasive ventilation study and we looked at unadjusted measures of overall use, range across hospitals, range across countries and range across time of use. When looking at medications, we found that overall 21% of hospitalized patients received corticosteroids with a wide range across hospitals, 0 to 90%. There was some slight differences whether use in the U.S. or not U.S. with non-U.S. sites using more steroids overall and a significant increase over time with a near doubling of use after July when new evidence came forward. Similar to remdesivir was used in 9% of patients, wide variation of use, wide variation across countries with the U.S. using much more than international non-U.S. sites and again increasing use over time. Anti-IL-6 inhibitors and anti-IL-1 inhibitors, even less use, about 6% of patients received these again with wide variation across hospitals, less variation across sites and a decreased use over time. And finally hydroxychloroquine with similar use to steroids, similar variation across hospitals and across countries but with a large amount of de-implementation over time from a quarter of patients early in the pandemic to 1% after July. So rapid de-implementation for hydroxychloroquine. Looking at some specific ARDS therapies that we looked at including inhaled vasodilators, we found less use, about 5% of patients received these, much more who were on invasive mechanical ventilation but some patients who were not on invasive mechanical ventilation received inhaled vasodilators, a little less variation across hospitals of 0 to 30% and no change over time. Prone positioning, 24% use, mostly among patients who did not receive mechanical ventilation but we do see 12% of patients having prone position who were not on mechanical ventilation and this is something we've seen with COVID, the awake non-intubated proning. No change over time for proning and again we looked at use of APRV among those who were mechanically ventilated and found wide variation in use across hospitals but not wide variation across time or country. So in summary, looking at medications for an adjunctive ARDS treatment, we found wide variation across hospitals and uptake. Practices changed over time for as evidence changed with de-adoption of hydroxychloroquine and adoption of steroids and remdesivir over time whereas ARDS associated practices were stable over time with fairly robust use of prone position that was higher than reported rates in non-COVID studies. Again similar limitations of unadjusted, these are preliminary results that I'm presenting to you with some issues with missing data as well. So in conclusion, the use of nearly every therapy in hospitalized adults that we looked at with COVID-19 varied widely across hospitals and high flow nasal cannula and non-invasive ventilation, this variation persisted even after adjusting for patient characteristics and so the hospital that patients with COVID went to really determined much of the care that they received even above their clinical characteristics and degree of illness severity. We did find strong relationships between the strength of evidence and as new evidence accumulated for different therapies like corticosteroids or hydroxychloroquine that there was an increase in adoption and de-adoption of those therapies. So what this means is one, it's unclear why there was such variation and so the results of this study will drive a qualitative analysis of institutional approaches to evidence uptake which we can then link to practice patterns and outcomes and will drive us to turn our attention towards looking at improving on this wide practice variation by doing a quality improvement trial within the virus registry, the SMART-ICUQI trial that I think you'll be hearing more about soon. And this will present tools and resources to facilitate evidence-based practices for COVID-19 and hopefully beyond. I would really like to acknowledge our study team from Boston University, the Society of Critical Care Medicine, Mayo, the virus team, and really especially a special thanks to all of the sites that contributed data to the virus registry. Thank you so much. Hello and thank you for having me come and give this talk today. My name is Svet Hevner. My pronouns are he, him. I'm a registered nurse and site PI for Prisma Health in the Viral Infection and Respiratory Illness Universal Study and also head up one of our subcommittees which I want to talk to you about today. I have no disclosures to share, no vested financial interest in any of the pieces I'm going to be discussing, no income outside of my own institution. The big caveat that I want to give before I get into this, because this is about automating abstracting data from an electronic health record, is that I'm not an informaticist. Prior to the COVID-19 pandemic, I was a research manager managing a program seeking to implement evidence-based routine HIV and hepatitis C screening in our emergency department. And so I had a lot of interaction with our infectious disease team, with our informatics team, and with aspects of critical care, because any time you're doing anything to ED, it does overlap. When the COVID-19 pandemic hit, I was the person inside my institution who had the right stakeholders nearby to kind of head up our data efforts. And then there came an opportunity for us to collaborate with the Mayo Clinic and Society for Critical Care Medicine on the virus registry. And I attended, and I ended up opening my mouth during one of the first meetings to ask, you know, is anyone else interested in looking at how can we speed up this data collection process? Several people have talked about the registry itself, but I want to kind of point out some of the barriers that we're facing. You know, electronic health records, the idea was that it was going to give us all this incredible data, that we would be able to pull that data together quickly and be able to study things. But the reality of the implementation is that they're so customizable, and there's so much variation in the use, even within an organization down to the individual level, that it's really hard to get some of that data out, let alone the reality that we don't have common data models. There's a lot of renewed energy there to get, to map to OMOP or N3C or FHIR, but that, we're not there yet. There's also a promise to have artificial intelligence and natural language processing that we just don't have available, readily available for everyone. And on top of that, there's barriers to actually accessing the data within your institution. Those can be regulatory or organizational barriers, but also the electronic health records themselves frequently require you to be credentialed with the corporation before you can access the full data markets. And so why does this become a problem? Because if you're mainly abstracting data for something like virus, this is an ongoing pandemic. There are 549 variables for each patient record. 265 of those can be captured up to 17 times, depending on how long the patient is in the hospital. So a complex intensive care patient could have upwards of nearly 5,000 data fields that have to be collected. We're estimating that probably takes about four hours per patient. And with the numbers as they are, my institution, we're approaching 4,000 patients. That's just not sustainable. We're not able to get that kind of data out with manual abstraction. And so as we brought this team of people together who are interested in talking about how do we automate some of this data collection, we came up with this idea to have this group called PEEP, Practical EHR Export Pathways. There are 38 systems. We wanted to develop some tools and resources, and the name kind of started off as a joke PEEP on a ventilator. We're giving a little bit of support, trying to help people open up and get started on their own process, as opposed to delivering a whole package for them to implement themselves. The way we approached that was to develop these series of tools. It started out with my institution sharing our SQL queries that we use to pull data out of the Epic Clarity Database. These aren't drag-and-drop queries, of course, because everyone calls their variables different things and we're not on common data models. But what was helpful about that is that it's shared the logic of some of the data cleaning. So not everyone had to reinvent the wheel to figure out how to link the diastolic blood pressure, the systolic blood pressure, so you actually get the highest set, so you get the highest map, or the lowest as we're going through and getting all this data. We also developed some Excel macros and some red cap calculating tools to help people that still were relying heavily on manual extraction, just to speed that process up. So instead of having to collect all the vital signs and labs and then collect SOFA, we made a calculator to calculate SOFA based on the data that was already entered in Enhance Table 1. We also recognized really early on the value of peer coaching in this work, to have a sounding board, a group to discuss how difficult some of these variables are and collaboratively develop some solutions. And through that whole thing, we're trying to develop this conceptual framework of what is it that we're doing and how is it working, because the reality is that COVID-19 is hopefully going to be the largest pandemic any of us will live through, but it certainly won't be the last, and it certainly wasn't the first, and we're going to have this problem again. We had weekly meetings in our online forum. We've recorded all of that information, so it's there for posterity. We can go back and refer to it and find out why we made decisions the way we did, in addition to notes that CIRL has taken. And the first thing we developed was this kind of schematic of what does this process look like, looking at all the different ways that we're pulling data together. One of the biggest tools, that Clarity database, is analyst-driven reporting, and this requires a high level of skill, it requires a high level of access, it requires training and infrastructure that some institutions simply don't have. It's a very robust tool, it's a very powerful tool. We call this the scalpel, because we can really get the data we want, but for some sites, that's not available, and they need to use end-user reporting. In Epic, this function is called Workbench, where an actual physician or a nurse can get in and kind of build a report to pull information together that they might need. We call this the sledgehammer, though, because it's not always as fine of a tool. You end up having to go through several steps to clean that data up, because you might be able to get a whole dump of blood pressures, for example, but it's harder to get it cleaned the way you really need it for a registry like this. It's hard to get it to give you the highest set, or the lowest set, from each calendar day. And so, both of those require some transformation, certainly the end-user reporting more than the analyst-driven, but we also recognize that there's always going to be a need for manual abstraction. There are some data points, particularly for virus, we've found most people can't get signs and symptoms or imaging interpretations through automated means. These are variables that are captured in narrative form. It's in the HNP, it's in the radiologist dictation. They're not checkboxes where you can just export them in an Excel field. And some sites have access to some beta natural language processing, but it's not readily available to all of us. And so, all of that data then ends up in essentially a local registry, a data warehouse. For me, I have my local registry captured in Redgap and another platform called Snowflake, but then I still have to do a little bit of transformation to get it into Mayo Clinic. So, there's a lot of steps that have to happen to get this data out of Epic, for example, into the platforms itself. But we found that through this collaborative coaching, through this resource development process, we were actually able to get 10 sites to 80% automation in nine weeks of starting. So, and there were three sites that actually got more than 90%. And for those sites in particular, we realized that they were, that meeting the requirements of virus were part of their core operational priorities around approaching the pandemic itself. So, from my, at my institution, our COVID registry that I am developing in order to meet the requirements of virus is part of the data structure that informs our operational reporting, informs our incident command, informs our response strategy, bed management. All of those pieces are coming out of this same data. And so, there's definitely an incentive for me to be able to get the access that I need to get the data that I want. The other thing that we realized is that there's not an even distribution. We certainly recognize that data automation is going to let us get information a lot faster. And so, sites that are automating are going to have more reporting. But for example, the columns highlighted here are, in green, are the sites that are participating in PEEP. The orange are sites that are not. This first column definitely has some automation going on in their site, but they're not part of this peer coaching. But if you look at the third column, that's me. That's my little institution down here in South Carolina, just me and an informaticist plugging away at this information. And if you look a few more columns down, one of the top contenders is another small, it's a single hospital. But through the resources we've developed, they've actually been a big source of a lot of the coaching around end-user reporting. They're one of the top contenders, despite being a very, very small hospital with almost no resources. And so, as we've continued to explore this, to look at how would we expand this to other EHR platforms, recognizing that there's something happening here because the PEEP sites make up about 10% of all the virus sites that are participating, but they're contributing around 49% of the data. And they're not the big tier one institutions, as I said from these two examples, but still half the data is coming from 10% of the sites. And so, as we're continuing this work, we're looking to implement the same process in other electronic health records, particularly we're focusing on Cerner right now. And we're using the Dynamic Sustainability Framework to help us conceptualize this process. This is a continuous quality improvement framework that you end up looking at the multiple ecological levels at the same time. So, as you're developing one resource, you're thinking about, okay, this works to meet the goals of the project, is it going to work in this practice setting, is it going to work in other practice settings? This is something that is broadly applicable. There's some wonderful information from Chambers et al to look if you want to dig deeper, but we're finding that the other really big important thing is having in your team, having access to the data platforms, of course, is essential, but also having a knowledge of the data structure and the clinical culture so that you know what is happening at the bedside helps you understand how you're going to be able to get that data out and how you're going to be able to get it clean. And as we develop this framework, we have an ongoing evaluation that is going to help us look at what is different between some of these sites. There's a survey that went out the week before we recorded this, so I have some preliminary results, but one of the validated items that it included in there is called the Organizational Readiness for Implementing Change. It's a 12-liker type item instrument, prepared down from about 72, Shea et al published the Factor Validation and Implementation Science back in 2014, but it includes two subscales for commitment and efficacy. We made a slight adaptation to specifically say automated data collection instead of this change. There's two item examples there, one under each subscale. And what I have for you today, hopefully by the time of Congress, I'll be able to share some more robust results in the comments and other documents we're able to share. We have preliminary results from 20 sites. It's about a 7% response rate. We're hoping to get that response rate closer to 20, but we do have nine sites participating in PEEP and 11 that weren't. Just real quick and dirty ANOVA, looking at the two subscales, there is significant difference. The sites that are participating in PEEP are significantly more committed to implementing change and they have a higher degree of efficacy. P values were well below 0.01, but the R-squared values were close to 50. So there's clearly other factors that are exploring it, other factors that could explain this relationship, but we're continuing to dig into that analysis and hopefully we'll have more robust results to share by the time of Congress. The other piece that we are actively working on is a validation study. So we have 13 sites that are working to validate the priority variables. Those include things like SOFA scores, mortality, length of stay, and your co-morbidities, along with several key vital signs, lab results, other clinically significant findings. We are in the final stages of collecting that data, and we'll have some preliminary results to share at Congress, there will be a manuscript that we'll be looking to push out around that time as well. So we'll have some tables that we'll be able to drop in the comments to show you. But so far, we're feeling very encouraged. Congress have conducted some of their own validation studies and are finding validation as high as 90%, 90% agreement between automated data and manually abstracted data. So what we're hoping is that we're able to really demonstrate this collaborative effort had a huge impact on us being able to rapidly stand up this registry, gather this information in near real time, and have a reliable data source for future research. Once again, thank you so much for having me here today. It's a real honor to get to present to this organization. Please feel free to reach out to me. This is my email and Twitter handle. If you have any questions, comments, or if you want to get involved, either in the registry or the automation part in particular, all of us on the VIIRS team would be happy to help get anybody involved or answer any questions. My name is Valerie Dinesh, and I'm an assistant professor of nursing at the University of Texas at Austin with a joint appointment as a research scientist at Baylor Scott & White Health in Central and North Texas. My clinical training is in adult ICU nursing, and I joined the VIIRS registry team as the site PI representing 26 hospitals within the Baylor Scott & White Health system. My contribution to this series of presentations on the COVID-19 VIIRS registry is to offer a perspective from the site and health system level. I'll use our time to share lessons learned from our local team formation to workflow through the team science strategies that we're employing for internal data sharing and for team writing. As was the case for all of us, the startup of COVID research has been an added role and responsibility that gets addressed while keeping up with everything else. As researchers, I think we're accustomed to the practice of cultivating deep expertise in a narrow field, and COVID gets wrapped into that since the entire world of researchers have pivoted to fold their individual program of research into the context of COVID, and that collective effort shows. As a result, by early spring, there were dozens of observational registries competing with each other for volunteer effort to share data to develop a collective deep expertise in COVID. While I don't think a curated registry of registries for COVID has been created yet, even cursory searches of specialty societies include more than 40 registries relevant to any particular specialty, and more than 1,800 observational studies are listed on clinicaltrials.gov. There's no shortage of registry invitations. So with so much work to be done, while we all try to maintain continuing responsibilities with non-COVID research, the selection of a comprehensive observational registry to centralize effort was an attractive alternative compared with the dozens of specialty registries. This slide highlights just a small handful of some of the specialty registries that we compared the virus registry to in the early days in the spring. The approach to adopt a ready-made data field set and align efforts through SCCM and the Mayo Clinic was relatively easy, but developing the strategy for operationalizing the team and navigating data entry workflow and developing data governance and team science in the context of all of the competing demands of COVID and non-COVID commitments are what really represent the countless challenges and creative solutions that I want to share with this lessons-learned approach. With the selection of the virus registry, structuring the variables across these broad five categories, and providing the operational definitions, identifying the workflow and data governance were the next key steps. I think that one of the challenges in managing observational data during COVID-19 has been how to harness all of the data generated for the real-time operations decisions and strategy projections to then overlay all of the granularity required for acuity adjustments and analysis and research and then link data from all of these multiple sources with relevance to every acuity level within the hospital. I came into this project with more than a decade of research operations and research management experience and a floated half a dozen ideas for feasibility review in March to think about different ways to operationalize and kind of source all of the effort that would go into a registry of this magnitude and with the volume that was projected and that we're seeing for COVID. Given the broad relevance to ICU, to hospitalist medicine, to emergency medicine, and to all of the specialties, I really floated a lot of different ideas from a site level perspective, including things like the potential for temporary support for data entry from direct care clinicians that were temporarily converted to remote work due to COVID. Another kind of existing route for research effort can sometimes be to look at rerouting effort for health care workers that are on light duty due to workers comp claims or things along those lines. Also looked at the feasibility of integrating operational data entry directly into the COVID-19 database by modifying existing workflows for departments like infection prevention and or the lab team. Looked at options related to redirecting temporary support for data entry from research teams that were put on emergent hold due to COVID-19 restrictions while they converted all of their projects to be COVID-19 compliant and really just asking everyone for recommendations and variations on these approaches. Ultimately, our solution at the local level was to collaborate closely with quality and safety departments and combine forces with a temporary redirect of some work hours from research teams that were on temporary hold due to COVID-19 restrictions. We applied for competitive philanthropic research funding and were awarded funding which we used to help offset work hours for analysts and used that to write custom queries to extract data from the electronic health record. These workflow approaches enabled data entry hybridized with a combination of manual chart abstraction and custom query writing in the back end of the EHR. The custom queries for our health system are through SQL or SQL programming which is described in more detail separately by Smitty Hebner-Sullivan in the virus automation portion of this presentation series. What I really want to share is that the approach that we are moving forward with is really founded on a two-way flow of data so that as teams enter data they can also export from the data set to leverage data entered by others. Our local site approach is reducing redundancies in data entry and chart reviews across internal teams seeking quality and safety team data while also reducing redundancies in research data entry and chart reviews across smaller specialty research teams. So to offer really another visual of workflow strategies from the health system site perspective this is a screenshot of the categories of data elements captured for each COVID positive hospitalized patient enrolled in the virus registry. Each yellow box represents a bundle of variables related to patient characteristics interventions or outcomes and each of these bundles could contain more than like 80 clicks worth of manual data entry for each patient for each day. If aiming for 100 completion a complex patient could easily consume upwards of six hours of manual data entry if addressing each and every field. At the outset we relied on manual chart abstraction and initiated team training institutional validation procedures and time studies and quickly learned that a hybridized approach to blend automated data extraction with minimal chart abstraction would be the most viable approach to get at the volume of hospitalization cases across 26 hospitals. It's been an iterative process to validate automation for fields that our team hasn't previously queried so a lot of our lessons learned on the data validation side are related to detailed multidisciplinary reviews of flow sheet data to really work on the accuracy and quality of pulling proning data and calculations on the duration of high flow nasal cannula use during each and every hospitalization. This is the visual of the hybrid approach and I'll just click backwards to kind of do the comparison and contrast between the two and so this is the hybrid approach with the long blue bars with the SQL or SQL written into it and this is really isolating all of the manual chart review to a small subset of variables that are clinically important but technically very challenging to extract from the electronic health record without using manual chart abstraction. Of note, the data field percentages in the header here don't add up to 100% and that's because each variable was reviewed for the combination of feasibility and relevance for research questions and for example some of the elements for event management are excluded because of the feasibility of wrangling that kind of flow sheet data at least for now and overall the completion of every field for every patient could demand as much as 1,200 data points in some cases so scaling that back and developing well-validated analytic data sets to answer the majority of research questions has been the priority. So all of this data granularity being generated through virus can really contribute to large-scale observational research and as analyses are churning and developing to leverage data aggregated across health systems representing hundreds of hospitals and health systems participating in virus through SCCM the demand for insights from within individual health systems is also high. There are procedures for performing analyses on data generated by all of the health systems banding together through the virus data set but from an individual health system perspective we also took an approach to how to navigate and support internal data sharing to accelerate research that aligns and expands on the procedures established by the larger virus team. Part of our project setup within our individual health system site was to be attentive to data governance and team science principles from the outset. We established a procedural structure to prevent duplication of effort and analyses between teams to address data governance and to track data use and outputs at the health system level. We formed a registry steering group to review internal proposals of single or multi-hospital data for research from within the system and as with non-COVID research the proposals and research questions are reviewed for scientific merit for data availability and also considers proposal duplication with other analyses. Following approval we have structured project team and authorship plans that are developed collaboratively and aligned with ICMJE principles and support the use of the credit statement for contributions. Overall we're also cultivating writing groups related to our local data that we're contributing to virus and that strategy is to really support the lead of each manuscript or analysis as the first author who commits to independent development of the first draft and really elicits all of the contributions within co-authors. Overall the urgency of research operations solutions for the observational data as it's transforming into a kind of semi-permanent demand we're experimenting with additional approaches to further crowdsource a subset of the manual chart abstraction with clinician researchers within our system. For example we are looking into implementing a fee schedule of sorts but not a fee schedule from a financial perspective a fee schedule from a chart abstraction contribution perspective and looking at ways that we can ask teams to contribute a nominal volume of chart abstraction cases to the master file which could then translate to additional validation while expanding the data set and further distributing the burdens and benefits of collaboration. Overall these site perspectives are contributing to our lessons learned with the virus registry as a whole and I'm happy to share any resources related to team science and publication manuals and things along those lines that we've developed along the way. Another perspective is from the core registry team with the intricacies of aggregating data from across all of these sites and that is presented now by Dr. Tripathi. Thank you Dr. Dinesh. My name is Sandeep Tripathi and I am a pediatric intensivist at OSF St. Francis Medical Center in Peoria. I'm also the co-PI for the pediatric virus registry and I have been involved in registry design data cleaning and analysis. I'll be presenting on virus registry design and implementation lessons learned. So the registry is funded by the Gordon and Betty Moore Foundation and Jensen Research and Development. They had no influence on the analysis. I personally do not have any conflict of interest to disclose. So this brief presentation I've divided into four sections. First I will talk a little bit about registry design. Then we'll talk about data cleaning and analysis and finally about manuscript preparation and publication. So the registry is designed for research but it is not designed for answering any research question per se. So that adds complexity in registry design and we have learned a lot as we went through the process which I will talk about in the next few slides. So why registry and particularly why registry during pandemic? As Valerie has alluded to there have been a lot of registries that have been developed during the COVID-19 pandemic. In absence of registry investigators would be limited to presenting data from single center studies. Registry allows to get data from a large population of diseased individuals from geographically diverse areas over a very long period of time. And it also allows investigators to generate hypotheses and answers when randomized control trials are not feasible. But because of the amount of research that goes along once the registry is established it is also important to spend a lot of time at the onset in the design of the registry. Because data that is not entered in the registry will not be available to the investigators when it's time for analysis. So once we decided that we need to start the COVID-19 registry what next steps would we take? So first step is to develop the case report forms. After that you have to define common data standards and terminology. Then write standard operating procedures and then train for data entry. The next step would be to develop protocols for IRB and obtain IRB approvals from a central site and then all individual sites and then also data use agreements. Typically for a multi-centric study these things take months to years. For virus registry you are able to accomplish all of that within a few weeks. Once the registry is up and running the next step is to establish some sort of governance structure. And out of which the most important we've learned is the quality control subcommittee because a lot of steps go along with data quality control automation and validation testing which is not something that you do at once. It is an ongoing process. Then you also need a publication and presentation subcommittee and ancillary proposal review subcommittees. So the registry is based on RADCAP. The RADCAP most of the investigators are very familiar with. It seems like this has been around forever however this RADCAP first was developed in 2004 at Vanderbilt and it has gained rapid popularity and now it has more than 4,500 partner institutions in 139 countries. 12,000 publications have resulted from RADCAP. CDC, NIH and multiple state health departments are also using RADCAP for COVID-19 symptom tracking. The main advantage of RADCAP is its simplicity. It allows investigators to develop the data collection form without any background in coding and it's also available at no charge to the institutions. But there are some limitations as we have learned as we went along in this complex registry. One is that RADCAP stores all the project information or all data in a single master table which makes the data collection rigid and static and does not allow for complex schemes. Now what we mean by that is this is a typical Excel export from a RADCAP. The column A represents the hospital ID and the column B are the various data collection forms data. And as you can see the day zero which is the demographic information, demographic field or demographic row does not have outcome data while day 28 which is the outcome data collection form does not have demographic data. So this creates problems from a data entry standpoint it doesn't matter but at the back end it creates challenges for the investigators. The second is that there is no cross field validation. So what what we mean by that is a for example a five-year-old 35 kilo child is entered as a height of 100 centimeters. Now in isolation five years 35 kilos and 100 centimeters are valid data points. However a five-year-old 35 kilos cannot have 100 centimeters as a height and RADCAP does not have ability to track that. So that that can lead to errors in data entry. Moving on to data cleaning. So I have done the most of data cleaning for pediatric patients. So I will emphasize more about pediatric data cleaning or the lessons that we learned in pediatric fields. So age RADCAP or virus is an exempt study from an IRB standpoint so we cannot ask for age or date of birth because that's a PHI. So we have to ask age in years which is fine for adults but for children we need a little bit more detail than just years. So initially we have the registry was designed as age in years for more than five and age in month for less than five which was leading to a lot of errors. So then we have to change it later on to just three decimal points for pediatric patients. The other interesting problem is just identifying pediatric patients. So ideally registry should branch off if age is entered as less than 18 years. The challenge here is that you can you can select them based based on age. So anybody less than 18 would be pediatric but there is a potential of error. So if somebody is 90 years of age and by error you didn't write 90 but wrote as nine that would be counted as pediatric and there is no way to validate this. Now the registry does ask this question is this a pediatric patient at different levels like comorbidities, diagnosis of MIS-C, severity of illness but sometimes they also have conflicting answers. So this can lead to erroneous inclusion of adults in the pediatric analysis or erroneous exclusion of children in the pediatric analysis. The next is the comorbidities. So we designed it initially as a adult and pediatric comorbidities defined as separately. So the first question ask is if it's a pediatric patient then pediatric set of comorbidities open and if it's an adult then an adult comorbidity which is great if you are analyzing them separately but it limits evaluation of adults with pediatric comorbidities. For example, adult congenital heart disease is challenging with the way it's designed currently. Then there are still some redundant questions that we have learned and gradually tried to eliminate. For example, is the patient admitted to ICU is asked in the beginning and the demographic and then also in day 28 or outcome data. Some of these patients we realize that people have entered as no for ICU admission but then entered ICU length of stay. So then we modified the registry so that it is not the only ICU length of stay will only open up if you are mentioned as or entered as yes for ICU admission. So avoiding as much as we can about redundant questions. Laboratory values is another challenge. Initially we only entered one laboratory unit but one of our co-investigators pointed out that different hospitals would enter values have different units. So people are directly entering values from the EMR without consideration of the laboratory units. So now we have modified it and we provide multiple laboratory units but still there is challenge in cleaning up the data which was already entered with potentially incorrect units. Severity of illness scores is very important for any sort of registry for comparing the two groups of patients but it is something which has very standardized definitions and time periods of data and there is a high possibility of error when entered by untrained clinical or research personnel. Free tax entries. Now free tax entries are essential because there is so much that we do not know about COVID. So we have free tax entries for everything, for signs and symptoms, comorbidities, home medications. Fortunately they are almost impossible to analyze at the back end without especially when the data extends to 40,000 patients. Free tax entries have been a challenge. So moving on to data analysis part. So what questions can be answered from a registry is a fundamental concept and one of the things that we deal with a lot is missing data. Missing data is found in almost 50% of all retrospective studies and the amount of missingness is assumed to be around 15 to 20%. What type of missing data is, there are three types. One is missing completely at random which is basically researchers going through raw data forms and randomly discarding and then there is missing at random which is the more common one and it is also called as ignorable missingness which is a pattern of missingness is predicted from other variables and lastly not missing at random. Here the probability of a response is missing is directly related to the data that was collected. For example if you are asking about smoking history the people who are smoking are less likely to respond to that question so that missingness of a smoking question is not missing at random and this is a non-ignorable missingness. What do we do for missingness? The investigators have to decide whether they are going to do a complete case analysis or available case analysis and if not then they have to use some of the imputation methods. Now multiple imputation is a very common strategy and it is available in most of the statistical programs depending on the resources investigators can choose some of the more complex imputation methods also. Bias. So the greatest threat to any study and the one most commonly encountered is bias and bias for this virus registry is like selection bias. So most of the co-investigators are intensivists so it is more likely for them to enroll patients who are in ICU in the registry so that leads to inflation of the ICU admission rate and overall mortality rate being higher mortality rate being higher because ICU patients get priority in entering in the database. Similarly patients in registry may not be representative of the population and then there is information bias because of missing data or measurement errors and then recall bias which is more sick patients more likely to recall being exposed or for example smoking or some comorbidities as they are seeking to explain why they developed a severe disease. We also have to be mindful while making any form of causal inference in the observational studies because as a database research the investigators cannot control for all the variables that related to the primary research question so then they use experimental statistical designs one of which is matching and multiple regression where you don't balance the group but use a model to account for the confounding variable. The challenge is that since investigators does not have control over what data set has been collected there can be a lot of uncontrolled confounding so it's always challenging to identify measure and include as many confounders in the model as possible. So now once it's all done you have the data you have the analysis and you have the draft the last thing the investigators have to be careful is to go back and review the reporting of observational studies and epidemiology or stroke guidelines. Most of these checklists is pretty straightforward but some of them like missing data analysis or sensitivity analysis are easy to overlook when the investigators has a tight timeline for completing the project. So future directions our work is not done just by design of the registry we have to pay continued attention to data quality data completeness is very dependent on the cooperation between hospitals and the registry we've been very fortunate to have a very enthusiastic group of investigators and hospitals and the next next step for us would be to take a leap from the observational studies and implement some of the best practices that we have learned from this registry into prospective studies. Thank you.

COVID-19 Registry

observational registry

COVID-19 patients

treatments

interventions

practice variations

outcomes

high flow nasal cannula

non-invasive ventilation

collaborations

data completeness

data accuracy

experimental medications

data analysis