In the Ticknall Life newsletter sent last weekend to registered users and subscribers, I glibly questioned the validity of the argument leading to speculative plans for further restrictions to suppress Covid-19, saying
“As I write, the number of lab-confirmed new cases of Covid-19 is rising and there is speculation that a nationwide lockdown is to be re-imposed. Can this really be right? SAGE, the government advisory group, estimates that the tests we use have a false-positive rate of 1 per cent, so if 100 people are tested using our current methods, around one person will be diagnosed as having Covid-19 who definitely does not have the disease.
“The present testing rate is more than 200,000 per day according to the government, so that means that at least 2000 people per day are being added to the figures even though they do not have the virus. This must mean that the majority of verified daily new cases did not actually have Covid-19.
“As an ex-publican, I have heard a good deal of nonsense in my time, and I think I developed a nose for it. Surely, this government is in receipt of some seriously dodgy advice in relation to this pandemic. As they say on the TV ad: You do the math.”
The remarks prompted a number of people to write to me directly and I resorted to various forms of mail-merge in order to provide replies. Of course, I was delighted to be reminded how many people do actually read our occasional newsletters and pleased to receive so many responses, including from some who were eminently qualified to comment. It seems that many of our readers besides the ones who responded would be interested in the surprising results of the argument – and the impact on public policy – so I offer below some notes to help you get up to speed.
The interesting thing, from a mathematical point of view, is how the effect of false positives changes in relation to the prevalence of the disease, so that when prevalence is low (as at present) the number of genuine cases in a sample may be quite small but the number of false positives in a given sample will remain the same. Furthermore, as the rate of testing increases, inevitably, so too will the number of (false) positive results.
So, using the 1% false-positive figure, in today’s (22/9/2020) sample of 257,377 producing 4368 cases (1.7%), 2574 cases (1%) would be false. Nearly 60%! The prime minister was fielding questions about government policy in the House of Commons today and there were plans to ‘massively’ increase testing rates. Mention was made of over half a million tests per day by October. That would mean that there will be at least 5000 new lab-verified cases per day in the next couple of weeks – and that’s just the ones that don’t actually have Covid-19!
Edit, 30/9/2020. I just read a report in The Lancet dated yesterday. This snippet is scary (emphasis mine):
“The current rate of operational false-positive swab tests in the UK is unknown; preliminary estimates show it could be somewhere between 0·8% and 4·0%. This rate could translate into a significant proportion of false-positive results daily due to the current low prevalence of the virus in the UK population, adversely affecting the positive predictive value of the test.”
If the false-positive rate was approaching 4%, one wonders whether the test would even be worth doing. ‘Significant proportion’ should read ‘almost all’.
Of course, many questions remain. For a start, of those tested, most (though not all) will have had symptoms or at least will have been referred by the ‘test and trace’ process, so are a long way from being a ‘random’ sample, which begs the question why the number of cases found is as low as it is. However, if the sample was truly random, you would expect even less real positives; ONS prevalence study data suggests it might be about 1 in 1000 cases (so, 257 actual cases) – although their results vary a great deal from week to week.
So that you don’t have to, I have done a bit of research to find useful documents on the topic:
The earliest article I found was a piece by Prof. Carl Heneghan penned in July.
Dr Clare Craig seems to have a rough idea what it’s about and has written twice for The Spectator:-
Professor Cathie Sudlow also has taken an interest. She has put together a model and has made it available in the form of a useful tool, enabling users with spreadsheet software to play with the parameters. Yes, it works well and throws up some rather striking results. You can get it here.
After you have familiarised yourself with what is essentially an arithmetical exercise, you may wish to think about the different ways that data is presented, and the massively different impressions that can be given. Yesterday, Dr Chris Whitty and Sir Patrick Vallance gave a presentation intended to prepare us for announcements today about new restrictions. If you missed it, you can watch it here. But there are more ways to present the same data. You must form your own opinion about why Whitty and Vallance chose to present the info in the way that they did.
Professor David Paton thinks that Whitty and Vallance are playing a dangerous game.
Edit, 30/9/2020. For those of you still reeling from the Whitty/Vallance presentation, the chart below was posted by Prof. Carl Heneghan today.
I have enabled comments for this post, so if you have something you would like to contribute, please feel free.