The COVID Scenario Pipeline: High Stakes Data Science

In March of 2020, DJ Patil assembled a team of data scientists and software engineers from Silicon Valley to assist with the data and modeling efforts for the State of California’s response to the COVID-19 pandemic. I was part of this group and worked for three months with a team of epidemiologists at Johns Hopkins and solutions architects from AWS to run large-scale forecasting models of infections and hospitalizations to aid decision makers at both the state and federal level with disaster response planning. I’ll share the story of kicking off the project during the critical 48 hours before California shut down, the work to operationalize and scale up our scenario generator to run as fast as possible on a single node, and the massively parallel, MCMC-enabled forecasting engine we built that ran on hundreds of machines and launched just two weeks before the second wave of infections began during the summer.