Search and Hit Enter

SAGE Modeller: “We Got Everything Wrong”

Okay, that’s a slightly misleading headline because the SAGE modeller in question, Dr Mike Tildesley, who works as a sooth-sayer at the University of Warwick, didn’t actually say that.

In an interview with Freddie Sayers for UnHerd, he says he and his colleagues who’ve been producing models for SAGE under-estimated the efficacy of the vaccines and over-estimated the extent to which people would return to normal after restrictions were eased.

And for that reason, they almost certainly over-estimated the impact that unlocking on June 21st would have had on infections, hospitalisations and deaths.

Indeed, we know their models were wildly pessimistic because if you compare the number of hospital admissions the models were predicting for round about now they are about three times higher than actual hospital admissions – and that’s the number they were predicting if the Government didn’t unlock on June 21st.

Regular readers will recall that Glen Bishop pointed out in Lockdown Sceptics that the Government’s court astrologers had underestimated the efficacy of the vaccines when it published the models it was relying on when it postponed the unlocking a few weeks ago.

SPI-M (the “Scientific Pandemic Influenza Group on Modelling) is the government committee in charge of producing forecasts for the future direction of the pandemic in different circumstances.

It was their report in early June, combining mathematical models from Imperial, Warwick and LSHTM, than persuaded Boris Johnson to delay the planned re-opening of society on 21st June to its current scheduled date of 19th July.

In the weeks since that report, two things have become clear: the raw case numbers have been rising very rapidly, but the hospital admissions have been much lower than expected when the PM made his decision. As of today, 1st July, just over 250 people per day are being admitted into hospital with COVID, compared to over 600 as forecast by SPI-M.

Freddie Sayers spoke to Dr Mike Tildesley, an infectious disease modeller from the University of Warwick who sits on the committee and works the models himself, about how his forecasts have performed against reality, and whether the PM made the right decision.

Why were the SPI-M models overly pessimistic?

  1. Underestimated vaccine efficacy

“I think the vaccine efficacies throughout have been slightly underestimated, shall we say, by the modelling groups, we are actually find that the vaccines are much more effective than previously we thought they would be. Now when these models are parameterised, the vaccine efficacy data came through from Public Health England, so we’re not making up these values, we are using the best estimates of values that are coming through from those on the ground that have their estimates of them.”

– DR MIKE TILDESLEY, UNHERD

2. Overestimated behavioural change

“I suspect this is something else that perhaps some of these models have slightly overestimated as to what we might expect that we’ll do in terms of the R numbers. This is partly because of people’s behaviour. So just because controls have relaxed, it looks like looking at the data that actually people haven’t gone back to ‘normal’ in terms of what we might have expected prior to the pandemic. So people are still being a little bit more cautious. Maybe they’re not going to the pub in the way that they were, say, back in January 2020. And that, obviously has some implications upon these forecasts that when these models were done.”

– DR MIKE TILDESLEY, UNHERD

So, had we known then what we know now, should the PM have gone ahead with June 21st?

“In hindsight, possibly — we’re in a position that the vaccine efficacy is much more effective, but the delay has also enabled us to vaccinate a lot more people with slightly higher level of restrictions in place. This is where it’s always a bit of a challenge because in a way if you wait, you’re always going to have a smaller wave. But of course, if you wait, then that’s much more damaging for the economy and for people’s well being and mental health and so forth. So there’s always a little bit of a trade off. The epidemiologist in me would always say it’s slightly better to err on the side of caution, but I was always very adamant when I said this delay was in place that it is really important that we get back to normal. I think it’s really important that we do get back to normal on the 19th of July. And the delay was probably necessary to allow us to resolve that uncertainty.”

– DR MIKE TILDESLEY, UNHERD

Why 19th July should go ahead as planned

“Looking at the data, looking at possible admissions and deaths, there’s nothing at the moment that really worries me. And I think if we are going to get back to normal, we’ve really got to do it over the summer, when the virus is less likely to transmit anyway. Otherwise I think we’re going to be in a situation where it’s going to be really hard. So I’m hopeful 19th of July does go ahead as planned.”

– DR MIKE TILDESLEY, UNHERD

Is the Government placing too much emphasis on models?

“The model should only form part of the decision making process. You also need health experts, economists, people from all these social sciences with a huge range of different expertise to advise the government, because it’s not just what the models are predicting. There isn’t certainty in those models, I think our responsibility is to make sure that we communicate what we expect from the models, but also communicate the uncertainty in those models. That’s really, really crucial. I don’t ever think it’s helpful to go in the media and say, if we’d locked down three weeks earlier, we would have saved 100,000 lives or whatever that’s not productive. Because any one of those decisions, there’s uncertainty around what we might expect to see. So I do worry a little bit that too much of the responsibility has been put on modellers. And I also worry a little bit about some of the rhetoric and in among some members of the government, who have always used the mantra “we’re following the science”, which almost seems like it’s a little bit of a get out of jail free card.”

– DR MIKE TILDESLEY, UNHERD

On the forthcoming Delta wave in other countries

“We’re doing pretty well — getting back to domestic freedom, as it were, is looking really possible over the next few weeks. Internationally it’s a much more bleak picture…. In certain countries in Europe this is a real worry, because of course, if they start to relax, and vaccine uptake is really low, then they’re going to see a wave. And actually, their wave is likely to be not just a wave in cases that we’re seeing, but potentially a wave of hospital admissions and deaths as well, which is a big worry.”

– DR MIKE TILDESLEY, UNHERD

THE END GOAL

Should we stop publishing daily death numbers?

“I think, ultimately, we’ve got to do that. I think particularly with the deaths as I said, I’ve talked in the media before about how we don’t say how many deaths there are every day from, say, cancer, or from road accidents and all these other things. Actually, we’re at the stage at the moment where the number of deaths are sort of in the 10s, significantly lower than deaths from many other causes. And I think we need to put it into perspective. As I said, we don’t want hundreds or 1000s of deaths, but having a situation where for the foreseeable future, we report the number of deaths from COVID, even when it’s really, really low numbers, I don’t think is helpful in terms of enabling us to get back to normality.”

– DR MIKE TILDESLEY, UNHERD

On why we need to start treating COVID like flu

In the longer term, we have to get more into a kind of a flu type relationship. We don’t want 1000 deaths per day, clearly, that’s catastrophic. But if we get into the winter, and we have a rise in cases and a rise in hospital admissions, similar to what we’ve seen in previous years for flu, do we consider that to be acceptable? Anyway, everyone will have their own opinion as to what the answer to that question is. But clearly, in the longer term, we need to develop that more of that kind of flu relationship with COVID.

Source: Freddie Sayers – UnHerd