Accuracy of tests

Covid-19 discussion, bring your own statistics
Post Reply
User avatar
Tessa K
Dorkwood
Posts: 1362
Joined: Tue Nov 12, 2019 5:07 pm
Location: Closer than you'd like

Accuracy of tests

Post by Tessa K » Wed Sep 30, 2020 11:36 am

Sorry if this has been covered but there are many threads I could have posted in and couldn't choose between them.

I had a flu jab today at Boots and the pharmacist told me the Covid test is giving around 40% false positives, the quick test even worse. Is this right?

bagpuss
Snowbonk
Posts: 455
Joined: Tue Nov 12, 2019 12:10 pm

Re: Accuracy of tests

Post by bagpuss » Wed Sep 30, 2020 12:13 pm

Tessa K wrote:
Wed Sep 30, 2020 11:36 am
Sorry if this has been covered but there are many threads I could have posted in and couldn't choose between them.

I had a flu jab today at Boots and the pharmacist told me the Covid test is giving around 40% false positives, the quick test even worse. Is this right?
The short answer is no.

I've been trying to find a definitive answer for the correct number but I don't think there is one. Most reliable sources say, however, that the PCR test has very high specificity and the only numbers I could find say <1%

https://fullfact.org/health/coronavirus ... -accuracy/ is worth a read

I suspect the 40% comes from the rate of false negatives where people are tested very soon after exposure and the infection isn't detectable yet.

User avatar
Gfamily
After Pie
Posts: 1759
Joined: Mon Nov 11, 2019 1:00 pm

Re: Accuracy of tests

Post by Gfamily » Wed Sep 30, 2020 12:35 pm

Tessa K wrote:
Wed Sep 30, 2020 11:36 am
Sorry if this has been covered but there are many threads I could have posted in and couldn't choose between them.
:roll:
I had a flu jab today at Boots and the pharmacist told me the Covid test is giving around 40% false positives, the quick test even worse. Is this right?
It's possible that if the incidence is sufficiently low and there is random testing, then 40% of the positive results could be false, but that would rely on a very low incidence rate.
As it is, the tests are being conducted on people who have some level of symptoms, so false positives will be a much lower percentage.

I think More or Less covered this recently
My avatar was a scientific result that was later found to be 'mistaken' - I rarely claim to be 100% correct
ETA 5/8/20: I've been advised that the result was correct, it was the initial interpretation that needed to be withdrawn
Meta? I'd say so!

User avatar
jdc
Hilda Ogden
Posts: 551
Joined: Wed Sep 25, 2019 4:31 pm

Re: Accuracy of tests

Post by jdc » Wed Sep 30, 2020 7:03 pm


AMS
Fuzzable
Posts: 357
Joined: Mon Nov 11, 2019 11:14 pm

Re: Accuracy of tests

Post by AMS » Wed Sep 30, 2020 8:13 pm

Gfamily wrote:
Wed Sep 30, 2020 12:35 pm
Tessa K wrote:
Wed Sep 30, 2020 11:36 am
Sorry if this has been covered but there are many threads I could have posted in and couldn't choose between them.
:roll:
I had a flu jab today at Boots and the pharmacist told me the Covid test is giving around 40% false positives, the quick test even worse. Is this right?
It's possible that if the incidence is sufficiently low and there is random testing, then 40% of the positive results could be false, but that would rely on a very low incidence rate.
As it is, the tests are being conducted on people who have some level of symptoms, so false positives will be a much lower percentage.

I think More or Less covered this recently
There's also the point that false positives should occur with a roughly constant probability, with an upper ceiling for this set by the observed rate for random asymptomatic testing during a period of low prevalence (ie not through contact tracing for example). Whatever the lowest % of positive tests has been over the whole pandemic, the false positive rate must be equal or less. Most importantly, false positives cannot explain an increase in the proportion of positive tests.

User avatar
jimbob
After Pie
Posts: 1623
Joined: Mon Nov 11, 2019 4:04 pm
Location: High Peak/Manchester

Re: Accuracy of tests

Post by jimbob » Wed Sep 30, 2020 9:30 pm

AMS wrote:
Wed Sep 30, 2020 8:13 pm
Gfamily wrote:
Wed Sep 30, 2020 12:35 pm
Tessa K wrote:
Wed Sep 30, 2020 11:36 am
Sorry if this has been covered but there are many threads I could have posted in and couldn't choose between them.
:roll:
I had a flu jab today at Boots and the pharmacist told me the Covid test is giving around 40% false positives, the quick test even worse. Is this right?
It's possible that if the incidence is sufficiently low and there is random testing, then 40% of the positive results could be false, but that would rely on a very low incidence rate.
As it is, the tests are being conducted on people who have some level of symptoms, so false positives will be a much lower percentage.

I think More or Less covered this recently
There's also the point that false positives should occur with a roughly constant probability, with an upper ceiling for this set by the observed rate for random asymptomatic testing during a period of low prevalence (ie not through contact tracing for example). Whatever the lowest % of positive tests has been over the whole pandemic, the false positive rate must be equal or less. Most importantly, false positives cannot explain an increase in the proportion of positive tests.
Exactly - the dashboard that @TravellingTabby on Twitter produces at

https://www.travellingtabby.com/uk-coronavirus-tracker/

Shows this positivity rate rising to about 4% overall and ICU occupancy highest for 3 months
Have you considered stupidity as an explanation

User avatar
Tessa K
Dorkwood
Posts: 1362
Joined: Tue Nov 12, 2019 5:07 pm
Location: Closer than you'd like

Re: Accuracy of tests

Post by Tessa K » Thu Oct 01, 2020 9:13 am

bagpuss wrote:
Wed Sep 30, 2020 12:13 pm
Tessa K wrote:
Wed Sep 30, 2020 11:36 am
Sorry if this has been covered but there are many threads I could have posted in and couldn't choose between them.

I had a flu jab today at Boots and the pharmacist told me the Covid test is giving around 40% false positives, the quick test even worse. Is this right?
The short answer is no.

I've been trying to find a definitive answer for the correct number but I don't think there is one. Most reliable sources say, however, that the PCR test has very high specificity and the only numbers I could find say <1%

https://fullfact.org/health/coronavirus ... -accuracy/ is worth a read

I suspect the 40% comes from the rate of false negatives where people are tested very soon after exposure and the infection isn't detectable yet.
I did ask him if that was false positives or negatives and he said definitely positives. It did seem an unlikely high percentage which is why I thought I'd check here before repeating it.

User avatar
Sciolus
Fuzzable
Posts: 338
Joined: Mon Nov 11, 2019 6:42 pm

Re: Accuracy of tests

Post by Sciolus » Thu Oct 01, 2020 11:05 am

Yeah, my understanding was that the false negative rate is quite high, but the false positive rate is low enough to be negligible for most purposes. For a layman like me, this makes sense if you consider that there are a bunch of reasons for a false negative (low viral load, poor swab, too early in infection so virus hasn't reached swabbed areas, degraded sample etc) but few reasons for a false positive (result assigned to wrong person, that's about all I can think of).

PeteB
Stargoon
Posts: 124
Joined: Wed Nov 13, 2019 1:02 pm

Re: Accuracy of tests

Post by PeteB » Thu Oct 01, 2020 12:01 pm

Yes, this is something that could be a possible issue, much loved by the covid sceptics, but isn't

https://www.bbc.co.uk/sounds/play/m000mr42 9 minutes - best explanation

User avatar
jdc
Hilda Ogden
Posts: 551
Joined: Wed Sep 25, 2019 4:31 pm

Re: Accuracy of tests

Post by jdc » Thu Oct 01, 2020 5:30 pm

Sciolus wrote:
Thu Oct 01, 2020 11:05 am
Yeah, my understanding was that the false negative rate is quite high, but the false positive rate is low enough to be negligible for most purposes. For a layman like me, this makes sense if you consider that there are a bunch of reasons for a false negative (low viral load, poor swab, too early in infection so virus hasn't reached swabbed areas, degraded sample etc) but few reasons for a false positive (result assigned to wrong person, that's about all I can think of).
I couldn't think of any but I googled to see if someone else could:
Technical problems including contamination during sampling (eg, a swab accidentally touches a contaminated glove or surface), contamination by PCR amplicons, contamination of reagents, sample cross-contamination, and cross-reactions with other viruses or genetic material could also be responsible for false-positive results.2 These problems are not only theoretical; the US Center for Disease Control and Prevention had to withdraw testing kits in March, 2020, when they were shown to have a high rate of false-positives due to reagent contamination.5
They go on to say:
The current rate of operational false-positive swab tests in the UK is unknown; preliminary estimates show it could be somewhere between 0·8% and 4·0%.
https://www.thelancet.com/journals/lanr ... 7/fulltext

User avatar
jimbob
After Pie
Posts: 1623
Joined: Mon Nov 11, 2019 4:04 pm
Location: High Peak/Manchester

Re: Accuracy of tests

Post by jimbob » Thu Oct 01, 2020 6:27 pm

jdc wrote:
Thu Oct 01, 2020 5:30 pm
Sciolus wrote:
Thu Oct 01, 2020 11:05 am
Yeah, my understanding was that the false negative rate is quite high, but the false positive rate is low enough to be negligible for most purposes. For a layman like me, this makes sense if you consider that there are a bunch of reasons for a false negative (low viral load, poor swab, too early in infection so virus hasn't reached swabbed areas, degraded sample etc) but few reasons for a false positive (result assigned to wrong person, that's about all I can think of).
I couldn't think of any but I googled to see if someone else could:
Technical problems including contamination during sampling (eg, a swab accidentally touches a contaminated glove or surface), contamination by PCR amplicons, contamination of reagents, sample cross-contamination, and cross-reactions with other viruses or genetic material could also be responsible for false-positive results.2 These problems are not only theoretical; the US Center for Disease Control and Prevention had to withdraw testing kits in March, 2020, when they were shown to have a high rate of false-positives due to reagent contamination.5
They go on to say:
The current rate of operational false-positive swab tests in the UK is unknown; preliminary estimates show it could be somewhere between 0·8% and 4·0%.
https://www.thelancet.com/journals/lanr ... 7/fulltext
We know from New Zealand that the false positive rate has to be very low - otherwise they'd have been chasing imaginary outbreaks.
Have you considered stupidity as an explanation

User avatar
basementer
Snowbonk
Posts: 547
Joined: Mon Nov 11, 2019 1:03 pm
Location: Aotearoa

Re: Accuracy of tests

Post by basementer » Thu Oct 01, 2020 10:26 pm

jimbob wrote:
Thu Oct 01, 2020 6:27 pm
We know from New Zealand that the false positive rate has to be very low - otherwise they'd have been chasing imaginary outbreaks.
Good point. At the time of writing, overall totals are
Total people tested 746,478; Test rate per 1000 head of population 151; Tests returning positive result 0.2% (1,492 out of the 746,478)

The current figures from the NZ Ministry of Health can be found here:
https://www.health.govt.nz/our-work/dis ... ty-and-dhb
Money is just a substitute for luck anyway. - Tom Siddell

Post Reply