Scrutable

badger wrote: Sun May 03, 2020 3:02 pm Thanks for clarification above.

When we've been waving at stats from other countries about the number of tests (and lack of them in UK), are those numbers swabs or cases/people?

It seems it's becoming increasingly difficult to make comparisons internationally given the difference and opacity around the definitions and collection method.

I agree. AFAICT the only truly robust numbers we can get are excess deaths. Analyses should be using that as a basis, and explicitly including parameters to account for detection differences when trying to extrapolate to numbers of cases.

The enormous confusion about the numbers in every country makes this almost impossible to manage effectively. Fine to lockdown, but the return to normality is going to be a matter more of luck than judgement.

Bird on a Fire wrote: Sun May 03, 2020 4:33 pm I agree. AFAICT the only truly robust numbers we can get are excess deaths. Analyses should be using that as a basis, and explicitly including parameters to account for detection differences when trying to extrapolate to numbers of cases.

It takes a while for excess death data to come in though, something like 10-14 days I think or perhaps longer. So it's not super great for telling you what's happening right now.

raven wrote: Sun May 03, 2020 10:10 pm

Bird on a Fire wrote: Sun May 03, 2020 4:33 pm I agree. AFAICT the only truly robust numbers we can get are excess deaths. Analyses should be using that as a basis, and explicitly including parameters to account for detection differences when trying to extrapolate to numbers of cases.

It takes a while for excess death data to come in though, something like 10-14 days I think or perhaps longer. So it's not super great for telling you what's happening right now.

That's true. But using the months of data we already have would help to refine estimates of the relationship between what we can actually detect in near-real time and what's actually happening. Hopefully the discrepancy isn't so great that, say, an increase in numbers could be masked. I guess we'll find out soon enough.

badger wrote: Sun May 03, 2020 3:02 pm Thanks for clarification above.

When we've been waving at stats from other countries about the number of tests (and lack of them in UK), are those numbers swabs or cases/people?

It seems it's becoming increasingly difficult to make comparisons internationally given the difference and opacity around the definitions and collection method.

Our World In Data are doing quite a good job of at least explaining what metric they have for each nation that's providing test data - so far I've seen the following categories: cases tested, tests performed, samples tested, units unclear, and inconsistent units. All that can do is give you a mental flag for a particular nation but that's not nothing. They've been my favourite source for international comparisons on a bunch of metrics.

All things corona
Details of their testing database

RoMo wrote: Sun May 03, 2020 10:58 am

shpalman wrote: Sun May 03, 2020 10:49 am It's the difference between the number of swabs, and the number of people who have been swabbed.

I wonder how they know the difference, as the data we have to send to the government daily is total tests carried out, number of tests split my patients and staff and the number of positive samples in each category - no de-duplication involved.

Well, since the test results must be sent back to the individuals who were tested, there is presumably a big database somewhere recording all the tests, with a record for each test showing who was tested, when it happened, where it happened, and the result.

For you bingo card, please add boxes for "governement assures everyone their data is private" and "large chunk of this data is found unsecured on a website".

Millennie Al wrote: Mon May 04, 2020 2:16 am
Well, since the test results must be sent back to the individuals who were tested, there is presumably a big database somewhere recording all the tests, with a record for each test showing who was tested, when it happened, where it happened, and the result.

For you bingo card, please add boxes for "governement assures everyone their data is private" and "large chunk of this data is found unsecured on a website".

No there isn't. Each lab has it's own computer system which obviously records demographics and results for each test and basic aggregate data is sent by each lab to the government each day; there's no central database at individual test level, so I can't see that there's any way for the government to know how many cases were tested each day.

Don't think Spiegelhalter's article on the problems of comparisons made it here (or this thread at least). Whitty referenced it in the conference on Friday.

Coronavirus deaths: how does Britain compare with other countries? | David Spiegelhalter

https://www.theguardian.com/commentisfr ... -countries

With amusing correction proving the point.

badger wrote: Mon May 04, 2020 7:23 am Don't think Spiegelhalter's article on the problems of comparisons made it here (or this thread at least). Whitty referenced it in the conference on Friday.

Coronavirus deaths: how does Britain compare with other countries? | David Spiegelhalter

https://www.theguardian.com/commentisfr ... -countries

With amusing correction proving the point.

tl;dr: "It's hard so we can't really do it". Thanks David.

shpalman wrote: Mon May 04, 2020 7:58 am

badger wrote: Mon May 04, 2020 7:23 am Don't think Spiegelhalter's article on the problems of comparisons made it here (or this thread at least). Whitty referenced it in the conference on Friday.

Coronavirus deaths: how does Britain compare with other countries? | David Spiegelhalter

https://www.theguardian.com/commentisfr ... -countries

With amusing correction proving the point.

tl;dr: "It's hard so we can't really do it". Thanks David.

Indeed. But it's worth reminding ourselves from time to time? Especially as how much the press bandy figures around, and how much the government are "looking to other countries" to inform decisions on unlockdown.

Part of the planning in Jan/Feb should have been to address these metrics. If you're building a car, you don't just concentrate on the framework and the engine - you also put in a dashboard. It's not acceptable to now be racing along the motorway, saying how hard it is to know what speed we're going.

lpm wrote: Mon May 04, 2020 8:41 am Part of the planning in Jan/Feb should have been to address these metrics. If you're building a car, you don't just concentrate on the framework and the engine - you also put in a dashboard. It's not acceptable to now be racing along the motorway, saying how hard it is to know what speed we're going.

But as we all know the wrong time to start questioning why there are no seatbelts, whether the driver knows what they are doing, or is even sober enough to drive, is when you are hurtling down the wrong way of the motorway at an unknown speed with the engine possibly about to explode.

Your own experience should tell you that in those circumstances the best thing to do is constantly congratulate the driver on what a bl..dy marvellous job he did on ploughing through that lay-by full of the elderly and leave any postmortem (figuratively and literally) until you have fully exited the motorway, possibly on your roof in a ball of flames.

Its common sense really.

shpalman wrote: Mon May 04, 2020 7:58 am

badger wrote: Mon May 04, 2020 7:23 am Don't think Spiegelhalter's article on the problems of comparisons made it here (or this thread at least). Whitty referenced it in the conference on Friday.

Coronavirus deaths: how does Britain compare with other countries? | David Spiegelhalter

https://www.theguardian.com/commentisfr ... -countries

With amusing correction proving the point.

tl;dr: "It's hard so we can't really do it". Thanks David.

Actually, he concludes with recommending waiting until we can compare excess deaths. I agree with him on the measure (the others have too much noise). Similarly, we do need to wait. People are dissing Sweden now, but its too early to tell whether it has a better or worse long-term strategy (as its possible that populations elsewhere will demand a relaxation of controls).

PeteB wrote: Fri May 01, 2020 11:34 am Just had another look at the euromomo site, hadn't appreciated that the 15-64 was so much worse - England is roughly double Italy's Z-score for all ages but 5 times for 15-64.

Age-adjusted all-cause mortality is the only thing that will tell us the effect of COVID-19, but it's difficult to compare numbers across countries by age group at this point since relatively few people aged 15-64 die in a month anyway, and probably quite a lot of those are accidents. So a tough lockdown that closed building sites and took 90% of cars off the road might cause a more substantial drop in non-virus deaths among the working-age population in Italy versus the UK.

In fact for a single month's numbers for this age group I'd be more inclined to rely on the official COVID-19 cause of death statistics, as most of these people will probably be dying in hospital rather than at home.

RoMo wrote: Mon May 04, 2020 5:53 am

Millennie Al wrote: Mon May 04, 2020 2:16 am
Well, since the test results must be sent back to the individuals who were tested, there is presumably a big database somewhere recording all the tests, with a record for each test showing who was tested, when it happened, where it happened, and the result.

For you bingo card, please add boxes for "governement assures everyone their data is private" and "large chunk of this data is found unsecured on a website".

No there isn't. Each lab has it's own computer system which obviously records demographics and results for each test and basic aggregate data is sent by each lab to the government each day; there's no central database at individual test level, so I can't see that there's any way for the government to know how many cases were tested each day.

After some digging, I take it back, yes there is. Information on certain notifiable organisms is captured by PHE via the SGSS (second generation surveillance system) automatically from the lab system. Surveillance data for SARS-CoV-2 positive tests get sent as a notifiable organism, but negative test information is also sent; which I don't ever recall happening before.

From the FT, interactive chart to compare deaths and cases between countries:
https://ig.ft.com/coronavirus-chart/

This article in Italian uses ISTAT data to estimate a 64% increase in the death rate in the first three months of the year in the Como area.

Nationwide the number of official covid-19 deaths accounts for just over half of the estimated excess deaths.

It also reports the percentage increase for other areas: Bergamo (568%), Cremona (391%), Lodi (371%), Brescia (291%), Piacenza (264%), Parma (208%), Lecco (174%), Pavia (133%), Mantova (122%), Pesaro e Urbino (120%).

It already takes maybe a week or two for an intervention to start showing up in the infection rates, and the death rate lags maybe 1-2 weeks behind that, so then if you have to wait even longer to figure out what to do because your statistician keeps saying "it's more complicated than that" what are you actually supposed to do?

Yes, when this is all over, we can do the stats properly to figure out what really happened. But we need faster feedback than that to figure out if the unlockdown is about to cause a second peak for example.

shpalman wrote: Mon May 04, 2020 2:58 pm This article in Italian uses ISTAT data to estimate a 64% increase in the death rate in the first three months of the year in the Como area.

Nationwide the number of official covid-19 deaths accounts for just over half of the estimated excess deaths.

It also reports the percentage increase for other areas: Bergamo (568%), Cremona (391%), Lodi (371%), Brescia (291%), Piacenza (264%), Parma (208%), Lecco (174%), Pavia (133%), Mantova (122%), Pesaro e Urbino (120%).

It already takes maybe a week or two for an intervention to start showing up in the infection rates, and the death rate lags maybe 1-2 weeks behind that, so then if you have to wait even longer to figure out what to do because your statistician keeps saying "it's more complicated than that" what are you actually supposed to do?

Yes, when this is all over, we can do the stats properly to figure out what really happened. But we need faster feedback than that to figure out if the unlockdown is about to cause a second peak for example.

I think you have to proceed with caution. At the moment, we have an argument for lifting the lockdown based on death statistics that are very misleading. Historical analysis would at least give some indication as to how misleading and in which ways - e.g., are we simply underestimating numbers, or could we be missing trends too?

shpalman wrote: Mon May 04, 2020 2:58 pm Yes, when this is all over, we can do the stats properly to figure out what really happened. But we need faster feedback than that to figure out if the unlockdown is about to cause a second peak for example.

Of course. Hospitalizations provide quicker data.

Mass testing of a random sample of citizens would also work. Though you'd need big samples.

On the question of how we count tests and cases, this is rather worrying:

From this Guardian article:

The Department of Health and Social Care (DHSC) began setting up rapid testing centres across the UK to test hospital staff and key workers for Covid-19 in early April, but Scottish and Welsh ministers were given the data for the first time late last month

The DHSC, which hired the accountancy firm Deloitte to run the programme, decided at first to only give the results to individual patients, to the irritation of ministers in Scotland.

As a result, none of the positive test results detected in Scotland and Wales were included in their daily updates on Covid-19 cases in April, and could not be factored into those governments’ planning for the pandemic.

I suppose there may have been some data security/privacy issues, but on the face of it it sounds... terrible. We're in a pandemic, national emergency, all supposed to pull together, yada, yada. How are Scotland and Wales meant to keep on top of things if the UK government isn't sharing data with them?

And why the hell is it always Deloitte? What do they know about running lab tests?

Second, sort-of related point. Excess deaths run about 10 days behind, but even the daily cases run a little behind. I went back to that github page to grab the county figures for the last two days to get my graphs up to date. Noticed that generally the last 2-3, or sometimes 5-6 day's worth of figures had changed. And for one county, figures all the back to March had been adjusted. Only by one, but still.

If we want case numbers to be bang up to date, I don't think that can happen by relying on positive testing alone. Firstly because of the time it takes, and secondly the number of false negatives which last I looked seemed to be running at 30%.

Should we perhaps once we 'unlock' be relying on something like diagnosis by symptoms instead? How does flu surveillance work? I know there's a network of surgeries that test for flu strains, but most GPs don't so not every patient gets tested. Do we keep a tally of patient reporting with symptoms as well?

South Korea managed to do mass testing a month ago.

Having some sort of system for rapid, reliable diagnosis before starting to unlock is a no-brainer. It absolutely beggars belief that they haven't got their sh.t together yet.

Bird on a Fire wrote: Mon May 04, 2020 3:44 pm

shpalman wrote: Mon May 04, 2020 2:58 pm This article in Italian uses ISTAT data to estimate a 64% increase in the death rate in the first three months of the year in the Como area.

Nationwide the number of official covid-19 deaths accounts for just over half of the estimated excess deaths.

It also reports the percentage increase for other areas: Bergamo (568%), Cremona (391%), Lodi (371%), Brescia (291%), Piacenza (264%), Parma (208%), Lecco (174%), Pavia (133%), Mantova (122%), Pesaro e Urbino (120%).

It already takes maybe a week or two for an intervention to start showing up in the infection rates, and the death rate lags maybe 1-2 weeks behind that, so then if you have to wait even longer to figure out what to do because your statistician keeps saying "it's more complicated than that" what are you actually supposed to do?

Yes, when this is all over, we can do the stats properly to figure out what really happened. But we need faster feedback than that to figure out if the unlockdown is about to cause a second peak for example.

I think you have to proceed with caution. At the moment, we have an argument for lifting the lockdown based on death statistics that are very misleading. Historical analysis would at least give some indication as to how misleading and in which ways - e.g., are we simply underestimating numbers, or could we be missing trends too?

Do you mean historical analysis as in months ago, or do you mean waiting until what is happening now has become history?

I don't really think the UK has much of an argument for lifting the lockdown, by the way; the official death rate just isn't coming down very much. It would be an argument in favour of ending the lockdown, to say that actually there were way more deaths a month ago which didn't get registered as covid-related, but now we're registering all of them so the figures are more accurate. But I would still prefer that the UK kept its lockdown going another month. (Whether that's sustainable is somebody else's problem of course. And it is a big problem which I'm not trying to minimize, I'm just not getting into sociology/psychology/economics/politics here.)

I suspect something like this may have happened in Italy, since the numbers in hospital peaked about 20 days before the total numbers of infected cases peaked. (Either that or the virus is getting milder.) And there were all those excess deaths back when testing capacity hadn't been ramped up and the hospitals were full.

Sorry, yes - historical as in months ago, and using that to guide how we interpret the more responsive data types.

Bird on a Fire wrote: Mon May 04, 2020 4:50 pm South Korea managed to do mass testing a month ago.

Having some sort of system for rapid, reliable diagnosis before starting to unlock is a no-brainer. It absolutely beggars belief that they haven't got their sh.t together yet.

There will be threshold effects. The countries like New Zealand or Taiwan have tested around about 150 people for every positive case. That's what contact tracing entails.

On 4 May the UK announced 3 985 new cases and 62 956 tests, or 15 about tests per positive. Forget 100 000 tests per day, at the current level of infection the UK would need to do a million. That's going to be difficult to set up.

Interesting paper on targeted lockdowns
https://twitter.com/ivanwerning/status/ ... 95778?s=21