Development, Integration, Deployment, and Use of a Validated, Real-Time Prediction Model for Favorable Outcomes in Hospitalized COVID-19 Patients

AI in healthcare has significant promise for the quality of care delivered to our patients. In regards to Covid, multiple opportunities for leveraging machine learning ara available. In this work, we present an end to end development, integration, deployment, and use of a machine learning based model in hospitalized Covid-19 patients. We retrospectively built a model on 3,345 patients and prospectively validated on 474 patients to identify patients with favorable outcomes within 96 hours of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4–88.7] and 90.8% [90.8–90.8]) and discrimination (95.1% [95.1–95.2] and 86.8% [86.8–86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Results suggest clinicians are adopting these scores into their clinical workflows. The talk will explore the technical and social factors with prioritization, integration, and adoption of a point of care model at our institution. We will also discuss relevant lessons learned and the special settings and workflows in healthcare that must be addressed for success.