Hello, my name is John Stavransky. I'm an intensivist at Emory University Hospital in Atlanta, and I'm delighted to talk a little bit about clinical trial design during the COVID-19 pandemic, whether traditional or frequentist designs or adaptive designs are better for patients and for clinicians. You can see here the financial grant support to my institution and some of my intellectual disclosures, but perhaps the most important disclosure is that I'm not an expert in adaptive trial design, nor am I a statistician, but I have participated in the design of both frequentist and adaptive clinical trials. During a pandemic, it is extraordinarily difficult to improve care to patients with a novel pathogen, and this is for a number of reasons. First, almost everybody has increased clinical workload, and people often wonder, are the usual treatments that I know of, the usual treatments that I do for other diseases, are they appropriate for this novel pathogen? There's substantial noise from your colleagues, from the lay press, and importantly, from the medical press in terms of what people think is best, and that sometimes can distract from improving patient care with a novel pathogen. And you often will talk to colleagues who have experience, and they'll tell you about their experience, but that is clearly inferior to having a clinical trial that tells you how best to care for a patient. So even high-impact journals, in their rush to deliver information to clinicians, have erred, and sometimes some of the things that showed up in the literature turned out not to be true. This happened both in journals that we read, in journals that general internists read, and there were enough retractions during the pandemic that there was actually a study on how retractions were published. One may feel that when you're trying to improve clinical care during a pandemic, that you're a little bit like the myth of Sisyphus. You're rolling a boulder up the hill, and it keeps on rolling back down after you. And that's rarely the type of work that we want. We would like to be efficient at delivering better patient care to our patients with a novel pathogen. So again, I was asked to talk about both the differences between frequentist and adaptive designs, but also which one might be better for patients and for a clinician. And it's, I think, useful to spend a few minutes just reminding everybody and myself what these two types of trials are. So a frequentist trial is a traditional randomized controlled trial in which patients are randomized, usually to either get an intervention or usual care or a placebo. And this is determined by a metaphorical flip of a coin. This is different from an adaptive design that allows information from patients who have already been enrolled into the trial to modify important characteristics of the trial, including to which arms patients are allocated to, to the dosing of the drug, to potentially the number of treatment arms, the study endpoints, and even the sample size. And this is often done in conjunction with a platform trial that, rather than testing an intervention, tests a group of interventions according to a disease process or a patient population. And perhaps one of the better examples of that is the REMAP-CAP study that was designed to look at patients with pneumonia, designed to look at patients admitted to the hospital with pneumonia. And as of a couple of days ago, and this is the beginning of February right now, they had enrolled almost 20,000 patients into this platform trial. And this particular trial was designed by the well-thinking investigators to be up and running, to have a so-called warm base in the event that a pandemic might occur during the course of the clinical trial. So again, you've got a base platform of a type of disease, namely pneumonia, and you can have different arms that are added on in the case of a particular type of viral pneumonia. So what are the advantages of frequentist designs? Well, most of us are familiar with them. In fact, the first clinical trial back in the days of scurvy was in the use of vitamin C versus no vitamin C, in which you compared one treatment with placebo. And most clinicians are fairly comfortable with a traditional frequentist trial. And in the past, the FDA had mostly approved trials based on the frequentist design. There's also, for an investigator, there's a lower barrier for entry, that all you have to decide is what you're going to test, and most statisticians can help you design this. But there are a number of disadvantages. Generally, you can only test a single hypothesis. So you can say, does this drug, does this treatment pathway work? And if it doesn't work, you're back at the beginning. And because of that, especially in critical care, where many of our trials have not shown useful treatment effects, they can be extraordinarily inefficient. And when you do sequential trials of things that don't work, in aggregate, you may actually spend more money on a trial, even though it may, the startup costs, if you will, for an adaptive trial may be higher. And finally, if you're randomizing patients to either get placebo or a treatment, that means that half of your patients are not going to get a treatment. And some people think that may be less patient-centric, rather than a design that would allow more people to get a potentially beneficial treatment. So what are the advantages of adaptive designs? Well, you can test potentially multiple agents. And because of that, they may be more efficient. And you can actually design the trial to have a responsive adaptive part, so that if there is a drug that's working, you can randomize more patients towards that drug. And if there's one that's not working, you can randomize fewer patients towards that drug. So you can stop interventions earlier if it looks like it's unlikely to have benefit, and perhaps, again, randomize more patients towards an arm that looks like it does have benefit. So the disadvantages, you've heard that many people are less familiar with that. The FDA, while it has some experience now, certainly less experience than with the frequentist design. And it does take longer to design and may cost more money to do so. And certainly, the initial design can be more complicated. And I just wanted to briefly show you what you could do in this three-arm trial, that potentially, if you find out that treatment two seems to be leading towards a signal that it might be better than treatment one or placebo, then you can randomize, rather than equally to all three of them, you can randomize more of the patients to treatment two. And you won't stop randomizing them to the other treatments. But again, you can shuffle the randomization. And when you, I mean the computer that is allocating people, can put more people towards the treatment that might be more effective. So potentially, you can do other things with an adaptive design. And this slide comes from the design of the VICTIS trial. We actually didn't, we weren't funded to get to this point. But we designed the trial so that if we showed our primary outcome measure, which was vasopressor and ventilator free days, because that is a less compelling outcome than mortality, the trial was designed with the help of statisticians who are adept at using adaptive designs, so that if we showed our primary outcome measure, we could continue the trial as long as the predictive probability that we were going to reach the mortality end point was neither very low or very high. So in other words, it allowed us to, the flexibility to potentially continue the trial as long as the predictive probability stayed within two sets of bumpers for either success or futility. So how do you decide which of these is better? Well, one way to do it is just to toad up the number of trials that have been published in high-impact journals. And in the next two slides, I'm going to do this. You will see studies that have informed patient care in patients admitted to the ICU or to the hospital with COVID, including the dose of dexamethasone, the use of an antibody to interleukin 1 beta, dexamethasone again, convalescent plasma, helmet ventilation, hydrocortisone, high-flow oxygen, anticoagulation. On the adaptive side, you've got therapeutic anticoagulation, convalescent plasma, interleukin 6 receptor antagonists, and dexamethasone. And if you toad them all up, you'll find that there were more studies done with frequentist design than adaptive design. So if you're strictly going by the number of trials done, maybe more of them were done with frequentist design. Having said that, I think that it's pretty clear that each of them have potential advantages, and that while it's perhaps easier to get a frequentist design up and running and off the ground, that there are a number of potential advantages of the adaptive design. And similar to the group that designed the remap cap, if you're already up and running, if you have a continuous trial going on, for example, that even though it may take longer to start, that it may be more efficient once it's started and may turn out to be more patient-centric, because you can potentially randomize more people to a treatment that might be helpful. I think that you will see that adaptive trials may become more common over the next few years. And the real winner, it's probably too early to tell. It's probably too early to tell which of them. But I suspect that many of us will be both doing and seeing more adaptive trials in the next few years. With that, I'd like to thank the organizers again for giving me the opportunity to talk about this. And if there is a live portion to this, I will do my best to answer any questions that people might have. Thank you very much.

clinical trials

COVID-19 pandemic

frequentist trial designs

adaptive trial designs

advantages and disadvantages