Digital COVID-19 test - Kings/Zoe study

[bagpuss]

I've signed up to take part in the Kings/Zoe COVID-19 study into the accuracy of a digital test for diagnosis of COVID-19 based on symptoms.

I have little faith in it being accurate enough - I model stuff for a living (among lots of other things) and know first hand just how difficult it is to get a model that predicts anything well enough to be useful for something as important as this. But then, symptoms are probably more predictive of having an illness than a company's type and size are of its printer needs, or a person's age, home town and hobbies are of their investment approach*, so maybe I'm being unduly pessimistic.

And after the exam "algorithm" fiasco, even if it does prove highly accurate, will people have enough confidence in it to act in accordance with its predictions, I wonder.

Anyway, it will be interesting to see what, if anything, comes of it.



*Going on skiing holidays was the best predictor I could find of having an interest in higher risk investments. Shame there was no available data on BASE jumping or similar.

[bob sterman]

I've signed up to take part in the Kings/Zoe COVID-19 study into the accuracy of a digital test for diagnosis of COVID-19 based on symptoms.

I have little faith in it being accurate enough

The fundamental problem with this idea relates to the primary purpose of COVID-19 PCR antigen testing in the community in the current situation.

PCR antigen tests are not being used in the community to identify COVID-19 cases and refer them for treatment. They are mainly being used to rule out COVID-19 in people with symptoms suggestive of COVID-19 - so they can go back to work or school without causing further spread of the virus. Algorithms may work well for deciding who needs a referal to a cardiologist or oncologist. But not so well for this.

In the UK the current positivity rate is something like 3-4% for PCR tests right? So only 3-4% of the people who have sufficient symptoms (or other reasons) to warrant a PCR test are identified as having COVID-19.

So what could an algorithm based test add to this? Could it "rule out" COVID-19 in any helpful way? Almost certainly not. The sensitivity would need to be dialed up so high to avoid false negatives.

I guess it could be used to identify people with the highest pre-test probability of COVID-19 - i.e. the people who really need a PCR test.

(P.S. skiing, particularly in Austria, was a pretty good predictor of COVID risk earlier in the pandemic!)

[bagpuss]

I've signed up to take part in the Kings/Zoe COVID-19 study into the accuracy of a digital test for diagnosis of COVID-19 based on sympto
So what could an algorithm based test add to this? Could it "rule out" COVID-19 in any helpful way? Almost certainly not. The sensitivity would need to be dialed up so high to avoid false negatives.

This is my main concern with the idea, I think. If people get an all-clear from the so-called digital test, will they then assume they are truly negative and go about spreading the disease, despite all other evidence to the contrary?

I guess it could be used to identify people with the highest pre-test probability of COVID-19 - i.e. the people who really need a PCR test.

That, I would think, is the most useful aspect of the idea - identify those in most need of a real test, where there are not enough tests for everyone who might need one to have one.

But then, you're still left with the false negative issue.

[jimbob]

I've signed up to take part in the Kings/Zoe COVID-19 study into the accuracy of a digital test for diagnosis of COVID-19 based on symptoms.

I have little faith in it being accurate enough - I model stuff for a living (among lots of other things) and know first hand just how difficult it is to get a model that predicts anything well enough to be useful for something as important as this. But then, symptoms are probably more predictive of having an illness than a company's type and size are of its printer needs, or a person's age, home town and hobbies are of their investment approach*, so maybe I'm being unduly pessimistic.

And after the exam "algorithm" fiasco, even if it does prove highly accurate, will people have enough confidence in it to act in accordance with its predictions, I wonder.

Anyway, it will be interesting to see what, if anything, comes of it.



*Going on skiing holidays was the best predictor I could find of having an interest in higher risk investments. Shame there was no available data on BASE jumping or similar.

I thought it was for estimating the prevalence in the population from those reporting symptoms correlated to the test results.

If you're looking at population-level stats, then it possibly becomes far more useful. Literally spotting where the next hotspot will be.

It was too low in August where the prevelance was such that maybe one might expect one symptomatic app user in many constituencies but now, when it's tens, it's probably getting pretty good, and less worry about testing running out.

[bagpuss]

I've signed up to take part in the Kings/Zoe COVID-19 study into the accuracy of a digital test for diagnosis of COVID-19 based on symptoms.

I have little faith in it being accurate enough - I model stuff for a living (among lots of other things) and know first hand just how difficult it is to get a model that predicts anything well enough to be useful for something as important as this. But then, symptoms are probably more predictive of having an illness than a company's type and size are of its printer needs, or a person's age, home town and hobbies are of their investment approach*, so maybe I'm being unduly pessimistic.

And after the exam "algorithm" fiasco, even if it does prove highly accurate, will people have enough confidence in it to act in accordance with its predictions, I wonder.

Anyway, it will be interesting to see what, if anything, comes of it.



*Going on skiing holidays was the best predictor I could find of having an interest in higher risk investments. Shame there was no available data on BASE jumping or similar.

I thought it was for estimating the prevalence in the population from those reporting symptoms correlated to the test results.

If you're looking at population-level stats, then it possibly becomes far more useful. Literally spotting where the next hotspot will be.

It was too low in August where the prevelance was such that maybe one might expect one symptomatic app user in many constituencies but now, when it's tens, it's probably getting pretty good, and less worry about testing running out.

That would make more sense. It's not how I understood it though. This sentence from the information appears to suggest otherwise...

This study therefore aims to evaluate the use of this digital test as a diagnostic tool for Covid-19 infection.

ETA - for clarity, this isn't just the regular data modelling from the Zoe app but a specific new research project using data collected through it together with PCR tests, to attempt to create a digital test for diagnosis based on symptoms.