Greetings and salutations to the Critical Care Congress at SCCM 2024 in Phoenix, Arizona. My name is Dr. Nicholas Pisa, my partner, Dr. Connor McNamara. We'll be giving the second half of this talk. We are both physicians at University Hospitals Cleveland Medical Center in Cleveland, Ohio, associated with Case Western Reserve University School of Medicine. Currently we're both assistant professors in the Department of Anesthesiology and Perioperative Medicine. We have no relevant disclosures for either of us in terms of funding. The main objective for listeners of this talk is that they are equipped with the stepwise approach we outline here in order to create their own curricula within their own institutions. The six major steps we will be focusing on are identification of the problem, identification of the learners and the surrounding learning environment, the focus of the objectives of the curriculum, creating educational strategies, implementation of the curriculum, and finally step six, evaluation and solicitation of feedback. The first step is identification of the problem. Development of a curriculum can be a long and arduous task made more difficult by a poorly defined problem because all subsequent steps depend on the initial step of problem identification being accurate. The curriculum designer needs to identify the specific health care problem while considering the problem's epidemiological characteristics and its impact on patients, society, educators, and learners. Exactly who is affected and how they are affected should be well delineated. An example could be post-intensive care syndrome, where the who would be ICU patients post-discharge that are affected by PICS, and the how would be the need for physical therapy, cognitive decline, psychiatric impairment, etc. After the problem has been identified, the next part is to analyze the current approach that is being used by all the stakeholders, including patients, health care professionals, educators, and society. For example, if a lack of post-intensive care syndrome competency in anesthesia residents is the identified problem, the next task would be to ask the following question about the current state. How are patients and their families currently dealing with PICS? Are they aware of what it is and what its consequences are? What are they doing for care after discharge from the ICU currently? And is there any education for the patient side that's occurring outside of the ICU? After looking at it from the patient side, it's then important to consider the current approach from the medical community. Who from the medical community is currently treating these patients with PICS? Is it psychiatrists, psychologists, therapists, primary care physicians? Is it, in some cases, intensivists, either physical therapists, occupational therapists doing this? Or is no one from the medical community treating the current problem? And then it's important to look at all these individual members of the medical community and see in what ways are they addressing PICS. Are they using a standardized multidisciplinary approach, or is it a bit disorganized and different from site to site? It's important to identify any standardized care models that do exist. After evaluating what the current approach of the medical community is, it's important to look at what educators are doing. What are educators currently doing to address teaching about, our example, PICS? Is it covered in medical school? Is it covered in another venue prior to residency matriculation? Is it taught on rounds during the course of the day during residency? Is it assumed that learners will seek out resources on their own? On their own time? Another important resource for looking at what current educators are doing is MedEd Portal. This was formed in 2005 by the AAMC. And within that space, there is an interprofessional education collection, the IPE, that houses a peer-reviewed repository of journal articles regarding the development and implementation of different curricula. This can serve as a source for those looking to design a curriculum to see what others are doing. The next part is to look at what society at large is doing to address the problem. In our example of PICS, are there third-party payers recognizing the diagnosis and treatment of PICS? Are there community networks offering support, such as church groups, support groups, or others, like Facebook groups? And does the federal government and the Centers for Medicare and Medicaid Services acknowledge the issue? After analysis of the current approach or lack thereof, the next step is to develop an ideal approach. It is useful at this stage to use a strategic planning process with visioning. Incorporating other stakeholders in the curriculum or a team of curriculum designers may be useful here to facilitate the creative thinking process and keep things moving in a positive direction. It is reasonable to acknowledge barriers in this step, however, they should not be allowed to impair the visioning process and the creativity that goes along with it. Statements like, we will never have the time for this, or they won't let us do it that way aren't useful at this stage and only serve to derail the creative process. It is important to balance the visioning with practicality, but only in a way that continues to encourage creative thinking. If the defining problem step was well-focused, the ideal approach and strategic planning processes will go easier. Creating an adaptable and flexible ideal approach will also be helpful in subsequent steps when dealing with the barriers. Step two is to identify the learners and the learning environment. Identification of the target learners is usually already complete for the curriculum designer. Regardless, it is still important to take a dive into what those learners will have for current and future job responsibilities, the competencies they are expected to obtain, and the training requirements needed to get the learner to competency. This might also be the step where the curriculum designer expands or contracts the pool of targeted learners. Some knowledge of the learner's current state may also be important, especially understanding their current levels of competency that the curriculum may address, as well as any of the underlying basic knowledge that may be a prerequisite for your curriculum. It may also benefit the curriculum designer to know how best the learners in the targeted group obtain skills. Are there any generational differences or influences that may play a role here in terms of different types of media consumption that can best educate the learners? An examination of the environment in which learners will experience the curriculum is the next task for the curriculum developer. What is already in place in terms of a curriculum and how has it worked or how has it failed? Who are the other stakeholders aside from the learners? Are the faculty educators who will be part of the implementation of the curriculum competent enough in the skill to subsequently teach learners, or do their skills need to also be developed further? It is during this stage that the curriculum designer should start to identify some of the potential barriers, including time, resources, money, and exposure. However, it's important not to be discouraged when identifying barriers, as this is merely part of the process. The way in which the data is gathered regarding the learners and the clinical learning environment can vary in effectiveness based on many factors. Ultimately, data should be gathered by the curriculum designer through a thoughtful and data-driven approach and adapted to fit the specific needs of the situation. There are a multitude of possibilities when examining methods of assessment, including informal discussions, focus groups, questionnaires and surveys, direct observation, personal history of the curriculum designer, testing, audits of the current state, and strategic planning sessions with your stakeholders. An organized and well-informed approach will serve the curriculum designer well during this task. Survey and questionnaire design may seem like an easy task at first, but obtaining a high response rate with usable data is not a given. Optimization of web-based questionnaires is beyond the scope of this lecture. However, many resources exist out there that can help maximize response rates and effectiveness. I try to apply the Goldilocks principle, where I examine a survey or set of questions and make sure that it's not so complicated that no one will complete it or take it seriously, as it may take too much time from them. But also, I don't make it so simple that I don't get the data that I need. Step 3 is focus the objectives of the curriculum. Well-developed objectives are necessary for any type of successful management, including curriculum development. Although they serve as a guide to the consumers of the curriculum, they also force the curriculum designer to fully think through the entirety of the impact of their curriculum and their project. Objectives can help set boundaries, define evaluation methods, and communicate intent to various stakeholders. They can also help set measurements for effectiveness after the curriculum has been implemented. A framework and guide for developing effective objectives that is often used in the business literature is SMART. Your objectives should be specific, measurable, assignable, realistic, and time-based. An example of this is at the end of the anesthesia intern's ICU rotation, they will be able to demonstrate the ability to place an ultrasound-guided central venous catheter under direct supervision without intervention from the attending physician and free of complications. This fits the SMART criteria because it's specific, it's talking about line placement, it's measurable, there's a discrete task with measurable complications, it's assignable, meaning it's talking about the intern in this example, it's realistic, because it is a reasonable psychomotor skill that can be obtained with repetition, and it is time-based, meaning it's by the end of the rotation. Bloom's taxonomy on the hierarchy of learning is especially useful for conceptualizing and refining the learner's objectives when building them. Bloom described the domains of learning as cognitive, knowledge, effective, attitudinal, and psychomotor skills. Within the cognitive domain, which most of our curriculum are going to live within, Bloom described the ascending linear learning that occurs as knowledge or recall, then followed by comprehension, application, analysis, synthesis, and subsequent evaluation. Keeping Bloom's domains of learning and the hierarchy of learning in mind is important when writing specific SMART objectives. Process objectives refer to the implementation of the curriculum and the expected degree of participation from the learner. An example of this could be a minimum number of procedures performed or a minimum number of patients examined. Implementation outcome objectives relate to the effects of the curriculum on learner satisfaction, learner achievement, learner behaviors, and systemic outcomes. Defining these objectives well becomes important for Step 6, where the success of the curriculum will be evaluated to continuously improve the design. An example of this can be the following. As interest in critical care fellowship has waxed and waned over the years, a measurable outcome objective could be how many medical students who rotate through the ICU and experience a specific curriculum pursue critical care medicine in the future. By now we have gone through the early stages of curriculum design. Now we get into actually creating content. To reiterate, curriculum design is not linear. During the first three steps, you perhaps were thinking of what future content may look like. That is okay. But it is important to have a good foundation from the first three steps before really delving into content creation. There are big picture two parts of curriculum content, the content itself and the delivery method. We will go through each in turn. Depending on the type of curriculum you are creating, its content may come from a variety of sources. Take, for example, a curriculum for a critical care fellowship. The minimum content necessary to teach your fellows would come from a content outline from whatever board certifies your fellows. In my reality, the American Board of Anesthesiology creates a content outline of what will be tested. This list is fairly comprehensive and may also suit the entire needs of your curriculum. Or it may not. In order to add above and beyond the content outline is up to the curriculum designers. Similar content outline exists, for example, for TEE certification or bedside echo, the various residencies. Let's say you were creating a critical care curriculum for a physician assistant student rotating to an ICU. A popular approach would be to use the Delphi method or its many iterations. This method requires stakeholders for the curriculum, perhaps its educators, to identify important topics. Furthering the example as above, your curriculum stakeholders you have identified as a teaching faculty for the PA program. Let's say in the previous steps, you will have 10 weeks in which to do your curriculum. You can do one one-hour lecture a week. Say you want each lecture to have one main topic. Basically, you want to identify 10 critical care topics to teach. You plan a multi-round modified Delphi process to identify these 10 topics. Round 1 is idea generation, basically brainstorming. This is done individually and perhaps via an online questionnaire. Your teaching faculty identify these important topics. Round 1 is an assessment of those topics. Perhaps there were 30 total topics identified. Some were noted multiple times, and you note these. Perhaps after looking at the list, your educators identify a topic that they and the rest of the faculty missed. In this modified Delphi process, you then show this list to your educators and tell them to identify from the list the 10 most important topics. Now, not everyone is going to pick the same ones. What is likely going to happen is that some topics will be picked more often than others. In literature, a topic usually has to reach about 60 to 75 percent threshold to remain to the next round. Let's say after the first round of voting, you are now left with 18 topics. This voting process happens again in as many rounds as is necessary to reach your 10 topics. During those rounds, reassessment can happen when necessary. What you are left with is a group of topics identified by the stakeholders of your curriculum with the known objectives in mind. This list contains many important details when trying to determine what education methods you are going to use. This is an important step to stop and think, what is actually possible in the time we have to do it? Learning style is an important consideration in determining educational methods. One learner, who in their educational life primarily learned in a classroom environment from lectures, is unlikely to learn best from a TikTok video or YouTube short. However, a different learner may thrive from these. Making a curriculum primarily focus on hands-on situation without a simulation center would be a challenge. But you might be able to create simulation scenarios in other ways. Personnel may be teaching faculty and support staff and all play a critical role in your individual curriculum. What technology is available to you may allow or prohibit you from different educational methods. Time is another important factor. If you want to do PBLDs in your curriculum and your group has limited time to create lectures and has no experience in creating PBLDs, it may be best to create lecture-based content and slowly create PBLDs over time. Discussing learning models could be a multi-day event. A simplified approach to learning models are that learners in a curriculum are challenged by new experiences, have achievable goals within the curriculum, are appropriately supported in achieving those goals, and are given opportunities for feedback and reflection. Generally, active has a positive connotation and passive a negative one. This implies that active learning is superior to passive learning. As alluded to, active learning may not be best for all learners who have only known more passive teaching methods. A good curriculum likely has a mix of both active and passing teaching methods. Passive methods, particularly traditional lectures, have a bad rap when compared to some of the newer and more active learning methods, such as problem-based learning discussions or PBLDs. The more passive learning methods allow significant flexibility to busy learners and educators. A one-time PowerPoint, such as this one, can be made by an educator, recorded, and then watched by whomever whenever they wish. Podcasts can be listened to when learning otherwise wouldn't be happening, like during a walk or working out. Active learning has its advantages with group-based discussion or more hands-on things like simulation. However, these oftentimes require significantly more time and resources to create and maintain. Thus, mixing active and passive learning within a curriculum likely will maximize the impact of your content to as many learners as possible. In summary, for creating educational strategies, your curriculum's content should be considered in the context of its overall goals. Curriculum content may be given, such as by a governing body, or created, such as by using a process such as the Delphi process. A mix of active and passive learning will maximize the impact of your curriculum. The curriculum exists within the resources of your institution and its various constraints. Once content has been decided upon and educational materials are in the works or are already made, you can start implementing the curriculum. This is not a simple step and requires attention to several details. The process of implementing the curriculum includes resource identification, political stakeholder support, curriculum leadership, management and structure, and identifying potential barriers. The implementation of these details is critical to curriculum success and will be discussed in further detail. Resources go beyond having a sim center. Likely the most important resource are the educators themselves. Depending on the size and scope of your curriculum, you may need support staff to help record, store, and distribute lectures and to help coordinate schedules. Additionally, having a curriculum director is important. This person, who may have been the curriculum designer but does not necessarily have to be, takes the vision identified in the previous steps and implements and maintains the curriculum as being discussed now and later. Space is at a premium in most institutions. For example, sim center space or space for lectures. When possible, creative solutions such as video conferencing may mitigate this. Time is precious and we're pulled in a million different directions, from work to home and back. The educators in your critical care curriculum share these same time constraints as they often have clinical and non-clinical responsibilities. Preparation, maintenance, and delivery of content takes a significant amount of time. Learners themselves have time constraints and competing priorities. Coordinating in-person lectures at specific times and locations may be a challenge. Some learners may not even be in the same physical location where the lecture is taking place. If you are trying to have educators lectures from different departments within your curriculum, different specialties have widely different days. The 6 a.m. time lecture may not work for everyone. People, space, and time have costs associated with them. Knowing this aspect within your institution during curriculum planning and implementation is critical. Because the curriculum requires people, space, time, and money, there needs to be buy-in from leadership, be that a department chairman, a program director, whomever. Without their support in your endeavor, it will likely fail immediately. One of the best aspects of critical care, in my opinion, is its multidisciplinary aspect. There are many disciplines that participate in critical care, physicians and other members of the care team, all of whom are important to the outcome of our patients. Pooling these resources, people, space, time, and money from these groups may increase the reach and success of your curriculum. That being said, without proper forethought, by involving so many different departments in your curriculum, it may make it so complicated it fails. There are challenges to both of these approaches. I have already discussed the difference between the curriculum designer and director. The director is critical to the ultimate success of the curriculum. The director's role have numerous responsibilities. They continue the initial vision of the curriculum, assess whether or not goals are being met, and have a strategy in place to introduce new learning styles or topics. These roles are a few from an exhausted list of possibilities of the responsibilities of this person. Thus, as time is limited, depending on the scope of the curriculum, administrative support staff may also be important, especially if coordinating across multiple departments. As discussed, the resources of time, space, money, and people, and the challenges regarding their management is a constant theme in this step. Brainstorming with your curricular development team and trying to identify and plan for as many issues as possible as it relates to resources is critical. As we had previously discussed, an infectious disease specialist planning to talk to your group about an update on CLABSIs may or may not be able to make that 6 a.m. lecture. The surgery residents, because they're in the operating room, may not be able to make it to the daily new lecture series on critical care topics. Any number of these scenarios I'm sure you can think of as it relates to your own specialty. The importance of both active and passive teaching methods may make it easier to overcome the difference in the daily lives of our multidisciplinary milieu. There may be some topics, such as airway management, that one specialty may feel like they own. That can be both a potential problem and a learning opportunity if presented in the right context. Especially if running a multidisciplinary curriculum, some topics, such as TEE, may not be relevant to all learners. Unplanned absences happen, of course. Do you have a backup plan in place for your education day if the simulation center staff are sick? If your curriculum is changing from a 100% lecture-based system to something all-interactive, such as PVLDs, careful introduction of this approach may be warranted. This switch from all-passive to all-active learning may be too jarring for some. Consider the type of learning that was occurring before your new curriculum is implemented and develop strategies to prevent this culture shock. Consider small-group introduction and elicit feedback. Consider how you are going to elicit feedback for your curriculum. Include as many stakeholders as you feel is important for whatever goal you are trying to achieve. This will be discussed in greater detail in the next step. A great curriculum is a living entity, constantly reacting to feedback and adjusting as necessary. The curriculum director can guide this process. To summarize this step, implementing the curriculum requires time, space, people, and money. It for sure requires stakeholder and leadership buy-in. Come up with creative ways to leverage all your resources. Support the curriculum with a director and an appropriate support staff. Consider implementing major change slowly and constantly assess and reassess the curriculum. The last step should be the never-ending part of the curriculum. The last step should be the never-ending part of the curriculum development. Curriculum should be dynamic and responsive to feedback. If goals change, the curricula should change. There are three overarching concepts relating to curriculum evaluation. Individual evaluation for both learners and educators. The effectiveness of the curriculum in achieving its goals. And a great curriculum should grow itself, leveraging its successes. Both the learners and educators should get individual feedback, both formative and summative. Formative feedback should help learners and educators identify gaps on a more immediate basis. An example would be direct facilitative feedback on learners' performance during a PBLD that occurred that day. Or feedback on a taped recording of a mock patient encounter. Formative feedback allows for pivoting of either learner or educator mid-curricula to reach a different and better outcome. Summative feedback assesses whether a more longitudinal goal is being met by the learner or educator within the curriculum. Examples include ITE results, board exams, CCC meetings. This can be both learners and educators. If all learners on a particular topic fail a specific part of a board exam, then perhaps the educator content of delivery is ineffective for that learning group. When possible, formative and summative feedback should have an objective and measured component. For learners, this helps with promotion and possible graduation. For educators, objective and measured feedback can be used for their own promotion. The effectiveness of the curriculum should also be assessed. This is done often by the curriculum director or by a small committee. A bird's eye view from the original vision laid out by the curriculum designers should be done on a regular basis. Whether or not the goals of your original vision are being met should also have both formative and summative components. For example, if there was a goal to have a multidisciplinary team teach within your curriculum and you only have educators from one specialty, this is immediate feedback that can be rectified as a very simple example. Clinical competency committees, board scores, learner career success, post-graduation surveys are examples of more summative. As with individuals, metrics are helpful for both growth and promotion. Successful curriculums grow themselves. They attract both learners and educators to want to participate and help all parties achieve progress. The success of a curriculum achieving its goals in itself could be publishable, which can both positively affect leadership but also the institution itself. Occasionally, even the content itself can generate its own resources by becoming marketable and or used in new media, such as podcasts or YouTube channels. Individual evaluations, ideally with metrics, should be both formative and summative. Results can be assessed regarding the curriculum as a whole, and active adjustments can be made as needed. A successful curriculum will grow itself, creating interest in itself and leveraging resources for continued growth and progress. The six-step approach laid out by Thomas et al. is a pragmatic approach to medical curricular design. Once a problem is identified, a strategic vision is laid out. Next, the context is identified, such as the targeted learners, the key stakeholders to success, and the environment in which the curriculum is being introduced. The overall goal of the curriculum is clearly identified, as well as metrics to measure its success. Following these steps, the actual content is created, as well as its delivery methods. A thoughtful implementation of the curriculum follows, identifying resources, curricular leadership, and barriers, making a rolling introduction, and maintaining the curriculum. The final step entails critical examination and feedback. Acknowledging the curriculum development is an iterative process. Curriculum design is also a dynamic process. The steps to curriculum development are often not followed in an absolutely linear manner. Planning during one step may require critical examination at another. The framework outlined here is a starting point for development of any medical curriculum within critical care, encompassing the multidisciplinary nature of the specialty. Leveraging the different perspectives of each specialty during curriculum design is an important consideration in the increasingly complex world of critical care. These are our references. Thank you. www.cdc.gov

Dr. Nicholas Pisa

Dr. Connor McNamara

University Hospitals Cleveland Medical Center

Case Western Reserve University

SCCM Critical Care Congress

medical curricula

six-step approach