Pandemic Data and Analytics (PANDA) writes there should not be any manipulation of death statistics during the Covid-19 pandemic.
Statistics are easily manipulated.
For example, by testing more, you will find more cases, and by testing less - all things equal - you will find fewer positive cases.
Drawing strong conclusions from case counts is therefore impossible.
Death statistics, however, are harder to manipulate as you generally need bodies.
For this reason, researchers the world over have zoomed in on death statistics and used excess death measures to gauge the impact of the pandemic.
This is an imperfect measure - Covid-19 and lockdown-induced deaths will unfold over time.
For the period 5 May to 14 July, the South African Medical Research Council (SAMRC) expected 101 388 deaths from natural causes.
Covid-19 deaths for this period have been reported to be 4 207.
With the actual natural-cause death count being 109 833, the difference between actual and expected deaths, of 8 455, the reported Covid-19 deaths and the excess deaths are starting to diverge.
Paraphrasing Occam's Razor: "The simplest explanation is most often the right one."
According to the Centers for Disease Control and Prevention (CDC), estimates of excess deaths can provide information about the burden of mortality potentially related to the Covid-19 pandemic, including deaths that are directly or indirectly attributed to it.
Excess deaths
Excess deaths are typically defined as the difference between the observed numbers of deaths over a specific time period and the expected number of deaths over the same time period.
Most countries have a good idea of how many deaths they might expect over the coming year, and these estimates depend primarily on the deaths observed in their population over the previous years.
Under ordinary circumstances, these estimates, and the resulting deviation from those estimates, offer insight into mortality trends and things like the severity of seasonal viruses, for example.
When considering the period 5 May to 14 July, this graph below illustrates the total expected deaths in South Africa from natural causes.
The red line indicates the best estimate, and the blue lines represent a range around this best estimate, with few data points anticipated to be out of this range purely as the result of random variation.
When circumstances change, such as with the arrival of the Covid-19 pandemic, measuring the actual total deaths versus the modelled expected deaths can help to answer questions regarding the magnitude of the impact of the change, or in this case Covid-19.
Analysis can also shed light on the following types of question:
- Are there possibly more Covid-19 deaths than are being officially reported?
This is also a statistic that can be compared internationally, although adjustments need to be made to reflect the way that different countries report and record deaths, and Covid-19 deaths, and has been done so by the Financial Times, as an example.
We are fortunate in South Africa to have teams who are dedicated to producing these results. Given the severity of our lockdown, and specifically our curfews and alcohol bans, our academics have wisely separated out the death data by natural and non-natural causes.
There is little dispute that the lockdown has resulted in fewer non-natural deaths linked to alcohol consumption (e.g. road accidents).
For this reason, when trying to understand the mortality rate of Covid-19, one needs to look at the change in natural deaths, not all cause mortality. (As an aside, Covid-19 will affect mortality directly and indirectly, in the short term and over the long term. The reduction in non-natural deaths due to lockdown regulations will be noticeable, including alcohol-related accidents now. This is unfortunately a double edged sword, and while lockdown measures may have saved some lives today, they will also cost lives in future when the impact such as poverty, reduced vaccinations, reduced TB diagnoses and cancer screenings manifest.)
For the period 6 May to 14 July (a recent period for which data is available), reported deaths were approximately 8 500 more than the expected number of natural deaths (the best estimate). This is about double the reported Covid-19 deaths for the same period.
During the height of the lockdown (Level 5), reported deaths were lower than the anticipated natural deaths.
This is surprising, especially since the actual deaths before Covid-19 and lockdown hit South Africa, during the first quarter of 2020, were running 2.45% higher than expected - which is still reasonable and within the range of random volatility.
That reported deaths for natural causes had a meaningful reduction once lockdown started, and while this is interesting, it could be due to random variation, underreporting of deaths or some other reason, but it should not invalidate the definition: excess deaths are by definition the difference between what you expected versus what occurred.
Reporting delays, and underreporting in general, may have been exacerbated by travel restrictions and the closure of many government departments, for example the Department of Home Affairs.
When the official excess death report was released, we were astounded to see that excess deaths were reported to be 17 000, which was far higher than the 4 207 deaths attributed to Covid-19.
On closer inspection, we noted the SAMRC arrived at this higher excess death number by using the lower bound rather than the best estimate. It is also noticeable that in other parts of its report, it sometimes uses the lower bound, and other times correctly use the best estimate. In the case of Buffalo City, they actually use a number below the lower bound.
The leading causes of deaths from natural causes in South Africa (HIV/Aids, cardiovascular disease, respiratory diseases [including TB] and diabetes) are unlikely to be materially affected in the short term by Covid-19 or a Level 5 lockdown.
However, there is a plausible theory that during level 4 and 5 lockdowns, and in particular when schools were closed, that deaths from influenza were lower than expected.
This explanation could go some way to explaining what was observed during April and May, but it does not negate the other theory that there was underreporting of deaths during this period in general.
Manipulation
The SAMRC has taken the decision to set the observed lower level of deaths during April and May, and extrapolated that forward as its "new" baseline against which excess deaths are calculated.
By deliberately lowering the expected deaths from the actual best estimate, the SAMRC conjured up an additional 8 500 excess deaths that were not there before the manipulation.
Playing devil's advocate, let's assume that the difference is completely explained by less influenza, by implication, when we moved to Level 3 lockdown and schools largely reopened, we would expect that influenza would return and the related deaths would follow.
When these deaths are observed, using the lower baseline as proposed by the SAMRC, these influenza deaths would be considered excess deaths. This is entirely misleading since these deaths would be expected under ordinary circumstances and should not be considered excess.
The explanation provided by the SAMRC, in the report ending 7 July, includes an admission that the expected deaths were manipulated to avoid "understating the impact of the Covid-19 epidemic".
However, at the time this was a disingenuous statement, since had there been no manipulation for the reporting period concerned, the difference between the excess deaths and official Covid-19 deaths would have been in line.
By intentionally changing its methodology, it has manipulated the estimate of the impact of Covid-19, massively overstating its impact for the period under report.
This is the conclusion that Tamar Khan came to in an article titled "MRC finds Covid-19 has led to three times more deaths than official tally" published in Business Day on 16 July.
While we may never know why the natural reported deaths seemed to slow immediately as lockdown started, or whether it would pick up again from the introduction of Level 3 lockdown, for the purpose of measuring excess deaths according to the definition this is largely irrelevant unless there is a known and substantial reporting delay.
Changes to methodology should be treated with care. In accounting, if there is a material change, accounts may be produced with and without the particular change so that the reader has the appropriate information they need to do prior year analysis and unpick the impact of the change.
All changes in methodology lead to notes and explanation.
A statistical reporting entity should be responsible for preparing the data as simply as possible, with notes to detail their adjustments due to things like changes in reporting delay. Manipulation and interpretation should be separated from the report.
- written by Pandemic Data and Analytics (PANDA)
*Want to respond to the columnist? Send your letter or article to opinions@news24.com with your name, profile picture, contact details and location. We encourage a diversity of voices and views in our readers' submissions and reserve the right not to publish any and all submissions received.
Disclaimer: News24 encourages freedom of speech and the expression of diverse views. The views of columnists published on News24 are therefore their own and do not necessarily represent the views of News24.