Dr Claire Neill, Specialty Registrar in Public Health, PHA and Dr Lynsey Patterson, Head of Health Protection Surveillance, PHA
The year was 1858 and an area of London had been hit by an outbreak of cholera (an infection caused by bacteria). The cause of the outbreak was unknown, but one doctor, John Snow, was convinced that the source was contaminated water.
Dr Snow gathered information from hospital and public records, drew a map of where the cases occurred, noting clustering around a pump, and went door to door asking people if they had drank water from this pump on Broad Street. Snow used the data to convince officials that the pump was the likely cause, and after removing it, the outbreak was controlled! Snow is considered one of the founders of modern epidemiology, the study of health or disease in the population, and his methods underpin the approach that we still use today in response to COVID-19.
Fast forward 100 years and epidemiology as a discipline has continued to grow. Epidemiologists aim to describe a disease or health outcome by time (trends in the disease, has it changed?), place (where are cases occurring, is there a pattern?), and person (how does the disease vary across different age groups, or gender?) and to compare those with and without disease with a view to preventing ill-health, or promoting good health.
As the COVID-19 pandemic has progressed, we have seen a rise in what has been coined as the “armchair epidemiologist”. The term was used many years ago to refer to epidemiologists who were too old to walk from door to door, as Snow did in London. In 2020, the term has been used to describe anyone who has taken an interest in understanding the vast amounts of data we now have access to.
During the COVID-19 pandemic, many of the general public have followed the case numbers and trends with interest, and watched eagerly to determine if we have successfully “flattened the curve”. Given the amount of information and technical concepts available to us, there is also the potential for misunderstanding. In this blog we highlight key things to think about when trying to understand data.
Always know your source!
It is important to consider where the information has come from. While opinions can be valuable, we should always use a reputable source for data. In Northern Ireland, the Department of Health’s daily dashboard, provides a daily picture of the COVID-19 situation. The Public Health Agency’s epidemiological bulletin adds to the dashboard by providing monthly commentary about those affected by COVID-19 in Northern Ireland. Finally, the Northern Ireland Statistics Research Agency (NISRA) is the official source for death data.
Be clear on what the statistics do (or don’t) show
During a pandemic there is a need to have timely data to understand the rapidly progressing situation and react accordingly. For many indicators used to measure COVID-19, however, there can be a natural delay in reporting. For example, testing data may be delayed because of the time between a sample being taken and arriving in the laboratory, and from test result to transfer to the regional data warehouse for analysis. We have discussed delays in reporting of death data in a previous article, giving more detail. It is important when looking at data for the most recent days to ask if the reason for fewer cases or deaths could be due to a delay in reporting.
Comparing apples and oranges
It is natural instinct to make comparisons and indeed comparisons are fundamental to epidemiology. For COVID-19, however, some comparisons may not be appropriate because of differences in what is being measured. The most obvious example of this is the comparison of COVID-19 death data. Surely something like reporting of deaths should be universal? The short answer is no. Some countries only report laboratory confirmed COVID-19 related deaths, while others also include deaths when there is a strong suspicion that the death was related to COVID-19, as decided by the doctor. This latter measure, which is included in the NISRA figure for Northern Ireland, is more complete and will result in a higher number of deaths being recorded.
Is the difference real?
Differences in definitions plague comparisons across countries, but what about just comparing trends within our own country; surely that is OK? Well, yes, for the most part if the method is stable. The exception for COVID-19, however, is with changes to testing. At the start of 2020, a new test had to be introduced and rolled out across laboratories. This meant that testing capacity was limited. Over time this capacity has increased such that we now have an extensive testing programme, with everyone over the age of 5 with symptoms of COVID-19 now being eligible for a test in Northern Ireland.
There are also targeted testing programmes carried out in care homes, and as a result of local clusters, which will increase our testing numbers. When interpreting testing data the impact of these changes should also be considered.
The statistics remain vital going forwards
We believe that the increased public interest in epidemiology is a positive development. In response to COVID-19, people now want to see the numbers, and can understand how these numbers influence public health decisions. We hope this blog helps you look at the numbers with a critical eye, and influence you not only to ask “what does the data show”, but also “what does it not”. With this knowledge, we can all feel empowered to make decisions based on our own personal risk and our knowledge of local transmission in the population.