This talk will be a little different. Previously we are told what to do. I'm going to say what you should not do when you're developing a simulation-based research. So these are the objectives for this talk. I'm trying to identify certain barriers and pitfalls in the conduct of simulation-based research and then try to develop some game plan how we can avoid those pitfalls and challenges. It is important to know when you're traveling that what path you are taking and what are the hurdles, what are the obstacles, what you should not do, what you should watch for. So this is important to recognize. So simulation is very flexible. You can do according to your needs, your objective, and what you want to do. But it can be very complex and it can be very uncertain. So as a researcher, we always want to watch for what are the risk. We try to minimize the risk for the potential research participant. We want to balance that. We maximize the benefit and we minimize the risk. As an IRB professional, we always look for those and we want to make sure no participant is exposed to unnecessary risk. And if there is a risk, you cannot do that study. So we have to avoid and minimize risk for the participants. So that is number one strategy we want to make sure when you're designing a study that these risks are minimum. And then in the simulation-based research, this checklist is very important that we try to understand that what are the important points that you want to watch for, whether there is a rigor, is the study is scientifically developed. There has to be a research question. Without a research question, there is no study, there is no research. So that scientific, it should be driven by science. That is very important. It's going automatic, okay. And research requires scientific rigor and simulation is used strictly to enhance learning and not in an evaluative manner. And simulation design or the study design should match the study objective. The objective should dictate what design we are going to develop, what design this study should have. In terms of fidelity, I know we talk so much about fidelity in the simulation, but fidelity is not the purpose, is not the main goal or main purpose of the study. Although it can help, it can help to improve the believability or reality of the study, but this is not the primary focus. We should aim for the fidelity to the extent that it helps the study objective. So what degree of realism or fidelity is necessary or appropriate? That we have to ask first. So we should not be going too crazy about the reality or realism or the fidelity of the study. In terms of fidelity, another term I would like to interject is interventional fidelity. It means this is the fidelity to the study protocol. We validate the scenario with participants who are similar to the target population and write the study protocol to clarify the conditions of the scenario that must remain constant to maintain the interventional fidelity. In other words, this intervention fidelity means is the fidelity to the research protocol. Sample size is also important that we want to make sure that we enroll subject that are necessary for the study. We do not want to be crazy and enroll too much and become so over-ambitious. So if the sample size is too small, it limits the generalizability and you cannot make any meaningful conclusion, but we have to make sure that it is enough to answer the study question. Also another concept that we sometime we equate research with education, although education is different. When we have simulation-based study for education purposes, also in terms of research also, sometime what happened that the patient or subject or participant, they think that this intervention is going to help and sometime that may or may not be true. So you have to make sure that these concepts are clear. Research and educations, they have a different objective. Choosing the right instrument is important to match the desired outcome. So first want to make sure, we want to find out that whether there is a instrument available. We don't want to invent the wheel. We want to identify that if there is already existing instruments available, we want to find out. And we want to assess whether these instruments are valid. These instruments should be valid and should be reliable. So reliable means that they are reproducible and validity means that they are supposed to do what they are supposed to do. And in terms of inter-rater reliability, which is the level of agreement, meaning that if I and you both are doing the similar study and testing those instruments in the similar condition, we should have similar results. So that is inter-rater reliability. These errors are important, that type one error is fatal. It means that you are convicting an innocent person, type one error. So this is very, very serious that we should not have these errors in our study design. So type one error means that we assume that one intervention is better than another, although in reality, there is no difference. Both are same. So these errors, recognition of these error and trying to avoid those errors is important. Also in terms of outcome, sometimes in the educational research, the participant think they may overestimate and they may think that everything is oversimplified and in retrospect, they think that this is too simple. So this is one of the potential challenge in those simulation-based research. There are tons of barriers that I cannot describe in this limited time, but the most important, I would say this is potential cost. The cost of those high fidelity simulators is very expensive and many centers may not afford it. So space is also concerned, especially when we have COVID. We were doing, it was very challenging to do simulation-based research, particularly in the clinical environment. So space in the facilities is another problem. We don't have unlimited time in our clinical practice, so always a challenge is the time. How you're going to motivate your staff to participate in the research? I think they are telling me to conclude it here and I would stop after this, I promise. So we always watch for those barriers and see that how we can avoid them in order to develop a research which is sustainable, feasible, and have a scientific rigor and always minimize potential risk for the participant and maximize the benefit. I think I'm going to stop here.

simulation-based research

scientific rigor

study design

participant benefits

avoiding barriers