I just want to say thank you to SACM for this opportunity to present my research, and special thank you to everyone still in the crowd standing for the last presentation of the day. I truly appreciate you. Yeah, please. A round of applause for you guys. So my name is Chintan Davey. I'm a critical care internal medicine trained clinician scientist. I'm part of a research group called DeepBreathe over the past two years whose aim is to automate lung ultrasound in acute care settings. We'll move on. Today I'll be presenting our research title, Prospective Real-Time Validation of Lung Ultrasound Deep Learning Model Employed Within the ICU. The objectives for today would be to, one, describe the actual potential for employing automated lung ultrasound for direct patient care, and the second would be to discuss the results of a deep learning model capable of instantaneous bedside lung ultrasound interpretation. The disclosures are listed there. So I'm sure we've all, by this point, have heard about artificial intelligence, machine learning, deep learning, or a variant of that. In the last couple of decades, there's been a rapid proliferation of the number of models that are created for numerous medical tasks, but more commonly, it's been used for the imaging field. I think everyone would be familiar with chest x-rays or the ChexNet, which is quite popular and has been used quite widely. However, when you look at the recent systematic review that looked at all models that were created within the medical field, the number of models that are created compared to the models that actually make it to model validation, which includes validation on other center data, real-time testing, and workflow integration, and eventual clinical outcome evaluation are almost negligible. When we refer to the ICU particularly, there's only 2 percent of all created models within the ICU have been studied at the bedside. Lung ultrasound, in particular, boasts a very high diagnostic accuracy when compared to something like CT scan for many common clinical findings, such as ARDS or COVID-19. It does boast additional benefit that it's handheld and it's easy to employ. However, a lot of centers have not been able to fully utilize the strength of lung ultrasound because of a persistent lack of supply in lung ultrasound experts. So we thought, what better way to find a solution than to automate the part of lung ultrasound that is most dependent on an expert, which is the interpretation part. Just a brief overview of lung ultrasound. As part of your initial interpretation of lung ultrasound, there is a binary distinction between normal and abnormal lung parenchyma. On the left side, we see what a normal lung parenchyma would look like on ultrasound with a bright plural line in a horizontal reverberating standing pattern, commonly referred to as A-lines. This indicates a normal lung parenchyma. We see some ribbed shadows here as well, which we see on the right side. But here, we also see these vertical lines extending from the plural line all the way to the bottom, referred to as B-lines. These indicate a pathology in the parenchyma, could be from multiple different ideologies. So we thought, this is the most common and basic distinction in lung ultrasound and it forms the foundation of interpreting lung ultrasound. We decided to create a deep learning classifier that's capable of distinguishing between A-line and B-line as the primary purpose. We trained a customized convolutional neural network on over 270,000 lung ultrasound images. And we tested it on over 1,000 lung ultrasound clips coming from 480 patients from both our centers and multiple other centers across the country. When we tested our model against both local and external data, we found that it performed equally well on both. With local data, the AUC ranked at 96% and on external data was 92.4%, yielding a very robust model, even when tested against external data sets. Now this led us to the next part. So now we've created a model that's validated on both local and external data sets. The real question we had is, can it provide any real world clinical utility? Because realistically, a lot of these models are created. But if they don't find a way into the real world application, they remain just a good idea without real world impact. So the research question we asked ourselves was, can a lung ultrasound deep learning model demonstrate high accuracy when applied in real life ICU environment? And so the methods we did to conduct this was a prospective observational study to evaluate the safety and efficacy of the model deployed at the bedside. We enrolled 100 ICU patients receiving oxygen therapy from two mixed medical surgical ICUs at tertiary care centers where we work. And we excluded patients who were unstable and those who could not tolerate lung ultrasounds. This is the workflow. So thankfully, we're at an institution where ultrasound, both echocardiography, bedside focus is quite prevalent. And every single patient admitted to the ICU, I see a colleague in the crowd. So he's nodding vigorously. So thankfully, we're in a center where that has been a commonplace practice. So what we do is all patients routinely get an ultrasound exam anyway. So for the purpose of our study, we did a four-point anterior lung exam for the 100 patients that equaled 400 lung ultrasound clips. But instead of just the ultrasound machine, we actually had our AI model, our deep learning model that was housed in a portable GPU, which was connected to the ultrasound machine via an HDMI cable. And the image that's produced on the ultrasound machine was replicated onto our portable GPU. Here, the model made inferences or made predictions on a frame-based level based on the live feed that it received. From here, the frame-level predictions were then the video files and the frame-level predictions were then exported onto an external device. Their real-time predictions were masked. And then we got lung ultrasound experts to then look at these videos and label them either A lines or B lines. And the frame-level predictions were then taken. We applied thresholding methods to reach clip-level predictions. And then we compared clip-level predictions from our model to what our experts labeled the same clips as. This methodology has been employed before for many radiological studies in testing the accuracy of model within the imaging field. Here's an example of what our model frame-level model predictions look like. So this is a prototypical example of an A line. It's about a four-second clip, and it records as 30 frames per second, so it results in 120 frames. And as you can see on the top left, it's the prediction made by the model as well as the confidence the model has that it's making the right prediction. So thankfully, it's got a good high prediction here, 98.7% for A lines. On the right side, we see a prototypical example of a B line, and we see frame-by-frame prediction from the model as well as its confidence in each frame, whether it's an A line or a B line. Now our model generated frame-level predictions, but in real life, clinicians are not interpreting lung ultrasound in a static frame-based manner. So now our next task was to take these frame-level predictions and create a clip-level prediction generated by the model. And given the heterogeneity of a lung ultrasound clip, as we know, ICU environments not always optimal or conducive for lung ultrasound exams, the heterogeneity, we've created two unique hyperparameters to address this issue. So we called it the classification threshold and the contiguity threshold, titled as such. The classification threshold is basically the minimum prediction probability for B lines required to identify the frame as a B line, for example, 60%, 70%, so we can arbitrarily set what the model confidence should be to call a frame A or B. And contiguity threshold takes it to the next step, is what is the minimum number of consecutive frames for which the predicted class is B lines for the whole clip to be qualified as a B line. So now again, we can play with this number. We can say, if you see one frame with a B line, call the whole clip a B line. Or we can say, wait till there's 10 frames of continuous B lines, then you qualify the clip as a B line. So this was the way how we got frame-level predictions and reached clip-level predictions from the model. Here's a video just further iterating our bedside setup. You can see an ICU patient who's ventilated. Again, this video is taken with family consent. You can see a routine lung ultrasound going on at the bedside with the ultrasound and the same image in real time, second by second, is produced on a portable GPU and creating instantaneous predictions from our model, which are then recorded and further analyzed. The video is played twice, apparently. All right, so the results. So we created four different approaches to analyze our results in efforts to maximize the sensitivity and specificity of our model. Approach number one, the contiguity and classification thresholds were set as three frames of contiguous B lines at a minimum confidence of 70%. So we're saying the model should be 70% confident that there's a B line, and there should at least be three frames in a row with B lines to call it a B line frame. This was identified in our multicenter study as the best performing approach. So this was our first approach. Approach two and four employ essentially a clinical approach to interpretation of lung ultrasound. So as many are familiar, to qualify a B line as pathological B line, you need at least two or more, or sorry, three or more B lines. So if you have one or two B lines, it's not typically considered pathological. And so we did approach two and four to kind of take that clinical aspect into consideration. As our real emphasis is to make a model that closely aligns with the clinical approach of interpreting lung ultrasound. When we look at the results for all of our approaches, the approach two, which mimics again the clinical approach, performed the best with an accuracy of 95% when compared to clips labeled by experts, an AUC of 91, sensitivity of 93%, and a specificity of 96%. When we look at what our model performed on retrospective data from both local and external data sets, the performance is quite comparable. We did present ROC curves here as well, and we see the performance for approaches two and four, which again mimic the clinical approach for interpretation, performed much better at an AUC of 0.91. We also generated confusion matrices for all four approaches to truly get into the true and false positives and negatives, to then do a further analysis of where our model is not performing the best. For our best performing approach number two, this is the only result I present here, we see that nine out of 400 clips, the model falsely labeled as A line, even though they were true B line clips. So on the overall scheme, that's only 2.2% of the overall clips, so we weren't too worried. However, on a further qualitative analysis of these clips, we found a recurring theme. So clips that were flickering in and out throughout the clip, or clips that were not acquired effectively, either because the gain was too low, or if the lung window was not appropriate, were the things that caused our model to trip up. But again, we were reassured that the percentage is still quite negligible, to not question the overall accuracy and performance of our model. We put all that together. This is the first study to test the feasibility and performance of a deep learning and a convolutional network classification model for lung ultrasound in a dedicated ICU environment. And in fact, from our, if anyone was at the AI talk earlier today, it's probably one of the first ICU studies to take a model and apply it instantaneously at the bedside for interpretation. Our results justify that not only is it possible that we can incorporate these models at the bedside, but justify further inquiry into the impact of employing real-time automation of medical imaging into the care of the critically ill. Thank you so much. If you or your team are developing an AI model and are looking for ways to test this real-time clinical application, shoot me an email. I'd be happy to chat.

artificial intelligence

lung ultrasound interpretation

acute care settings

deep learning model

diagnostic accuracy