Back to Blog

How the Centers for Disease Control and Prevention overcame COVID-19 data challenges

When the COVID-19 pandemic first hit the United States in January 2020, the federal government struggled to collect accurate, comprehensive and timely disease data because of inadequate public health data surveillance and infrastructure systems throughout the country.

Test results and hospitalization records that reached federal agencies used differing data definitions and formats under the nation’s decentralized system. With the quick spread of the disease, many decisions had to be made based on incomplete, inconsistent and even outdated data.

To overcome these obstacles, a unit within the Centers for Disease Control and Prevention—the Situational Awareness team—worked around the clock to produce information that the CDC leadership and agency partners needed to make informed decisions.

One way the team navigated these challenges was by using “web-scraping” to collect COVID-19 cases and deaths directly from jurisdictional partner and school-district websites. Although this began manually, the team partnered with CDC leaders and Johns Hopkins University’s Applied Physics Lab to automate this process and aggregate case counts to track the spread of the disease within states and across the country in a timelier manner.

To ensure the data was accurate, the team coordinated daily with nongovernmental partners, including Johns Hopkins University’s Coronavirus Resource Center, to identify anomalies in data reporting and validate one another’s data.

When the team faced high turnover and burnout among skilled employees who had experience in emergency situations, CDC management worked with other agencies and universities to bring in additional personnel for a two- to six-month rotation and train these individuals on COVID-19 response operations.

The CDC team was resilient and met the data challenges, but noted some important lessons that all public health data collectors and users could better apply. Overall, “There needs to be a long-term strategy…to establish stronger enterprise systems and data standardization for availability, access and sharing to strengthen future operational responses,” said Roger Harlan, knowledge management team lead at the Situational Awareness team.

The Situational Awareness team also learned that public health data collectors and users need to transform its planning processes for public health emergencies. Preparedness and timely responses are only made possible when there is a long-term, strategic coordination and collaboration system in place.

If you are interested in more information about the Situational Awareness team and other case studies delving into management lessons in public health data during the pandemic, see the Partnership for Public Service’s report, Retracing Steps.

---