User survey

How was the Resurgence Map created and where does the data come from?

November 29, 2021

We calculate the surges in new cases of COVID-19 cases for 54 African countries from the beginning of pandemic onward using the Our World in Data (OWID) complete COVID-19 dataset. Peter van Heusden and Dirk Meerkotter tell us how the resurgence Map was created.

Background

The idea for this data dashboard originated with Peter van Heusden, a bioinformatician and Senior Systems Developer at the South African National Bioinformatics Institute (SANBI); he had the idea of tracking or assessing emerging and declining risk across the continent. In exploring the concept, we found several issues conceptualising this dashboard as tracking or accessing changes in risk. Firstly, we found ‘risk’ to be too loaded with meaning, and too prescriptive. Secondly, to measure risk, we need to measure more than the increase or decrease in cases (which is the data we had at the time).

The initial OWID COVID-19 dataset interrogation was done by Dirk Meerkotter and Peter van Heusden in a Jupyter Notebook using Python and Pandas. Dirk advises that “Jupyter is a great tool for data science as it allows on the fly interrogation, extraction and transformation of datasets.”  Dirk Meerkotter and Peter van Heusden began overlaying surges in new COVID-19 cases with different metrics to discern if there were any correlations between them. Ultimately, they decided that all the different metrics they had in the dataset should be available to the user to overlay with surges in new case data; this enables journalists to explore and find their own stories through the dashboard.  They did, however, decide on a formula to track the changes in new COVID-19 cases. They did this because the daily data had unexplained variability that obscured the meaningful data. Therefore, Dirk Meerkotter took a seven-day rolling window of the data to measure the percentage change in the number of new confirmed cases in that period relative to the previous seven days throughout the pandemic. This seven-day rolling view of the data is used in the resurgence dashboard - in the map and the leaderboard.

While this view of the data removes some of the noise associated with the daily data, percentage change of new COVID-19 cases, as we have calculated, can be problematic.

Dirk Meerkotter notes, “the problem I saw with the percentage approach is that it is relative to the week before - so even if cases are on the decrease - i.e. turning blue- that does not mean the situation is good or low in any way. That is why I added the Absolute view taking just the sum value - which more clearly shows the wave pattern even on the map.”

Where do we get our data from?

The data is sourced from Our World in Data (OWID), an online publication that provides research and data on global challenges. In addition, they source their COVID-19 vaccine data from the most recent official data released by governments and health ministries globally; see here.

Why is this a reliable data source?

Africa Data Hub does not collect the resurgence map data, and therefore we are dependent on OWID to collect and collate the data. We rely on their commitment and willingness to be transparent, as well as their credibility. We believe that they are credible based on their data collection from official sources, scientific methods, and level of transparency.

Is this sensitive data that requires approval before sharing?

OWID is a public resource, and their data is freely downloadable, and there is the option to do that on this dashboard. In addition, we download the data as CSV files daily. Therefore, visualisations and data can be shared with the corresponding attribution.

How current is the data used in the resurgence dashboard?

The data is updated daily; this enables the resurgence dashboard to be up-to-date with OWID datasets. We have noticed that there is a 1-2 days lag after governments and health ministries release official data.

How did we process the OWID data?

We created and employed a formula to track the changes in new COVID-19 cases. We did this because the daily data had unexplained variability that obscured the meaningful data. This statistical approach is called data smoothing and is used to eliminate outliers from datasets to make the patterns more transparent.  

What tools were used?

The initial OWID COVID-19 dataset interrogation was done in a Jupyter Notebook using Python and Pandas as this enables a quick interrogation, extraction and transformation of datasets. The web application of the Resurgence Dashboard is built with a React framework.

How were the indicators chosen?

We chose the indicators based on what data is most widely and commonly reported in Africa. The data selected has been highly curated and explained to avoid any misunderstanding and misuse of data.  

How did we decide on how to represent the data?

We selected data visualisations that lent themselves to the data and what story we wanted to tell with the data; for example, geographical data lends itself to be represented in a map.  

What should I look out for when using this dashboard?

Data is available at the country level for the continent because there are gaps in the sub-national level data. To ensure correct conclusions are drawn and accurate analyses are made we flag problematic data with the alert badge (!). At the time of writing this article, we are flagging data when there is no change in case numbers over a few days. These flags will, in the future, include watching for unrealistic percentage changes.

What combination of this data helps tell a compelling story?

The data is often surprising, and this dashboard makes looking for interesting trends simplified. Data smoothing can be defined as a statistical approach to eliminating outliers from datasets to make the patterns more transparent. However, looking at unsmoothed data can show how a country reports their COVID-19 statistics as surge data can indicate data dumping, gaps in data or just an increase in testing, for example. Showing the trends in COVID-19 data or looking into the quality of the data can both tell a compelling story.

Preferred citation: African Data Hub. (2021). Resurgence Dashboard. [online] https://resurgence.africadatahub.org

Sign up for our newsletter