[petejh]

Question for Stu or others who understand probability and modelling.

Just reading this latest report from Imperial estimating the effects of the social distancing measures across Europe.
https://www.imperial.ac.uk/mrc-global-infectious-disease-analysis/COVID-19/

Noticed it used 'credible interval' instead of 'confidence interval' in its estimate that the European distancing measures:
'saved 59,000 deaths up to 31 March [95% credible interval 21,000-120,000]

..and wondered what the difference was between credible interval and confidence interval, so I looked it up. Credibility interval is used when you use bayesian probability to calculate a value. As explained here:

Classical approaches generally posit that the world is one way (e.g., a parameter has one particular true value), and try to conduct experiments whose resulting conclusion -- no matter the true value of the parameter -- will be correct with at least some minimum probability.

As a result, to express uncertainty in our knowledge after an experiment, the frequentist approach uses a "confidence interval" -- a range of values designed to include the true value of the parameter with some minimum probability, say 95%. A frequentist will design the experiment and 95% confidence interval procedure so that out of every 100 experiments run start to finish, at least 95 of the resulting confidence intervals will be expected to include the true value of the parameter. The other 5 might be slightly wrong, or they might be complete nonsense -- formally speaking that's ok as far as the approach is concerned, as long as 95 out of 100 inferences are correct. (Of course we would prefer them to be slightly wrong, not total nonsense.)

Bayesian approaches formulate the problem differently. Instead of saying the parameter simply has one (unknown) true value, a Bayesian method says the parameter's value is fixed but has been chosen from some probability distribution -- known as the prior probability distribution. (Another way to say that is that before taking any measurements, the Bayesian assigns a probability distribution, which they call a belief state, on what the true value of the parameter happens to be.) This "prior" might be known (imagine trying to estimate the size of a truck, if we know the overall distribution of truck sizes from the DMV) or it might be an assumption drawn out of thin air. The Bayesian inference is simpler -- we collect some data, and then calculate the probability of different values of the parameter GIVEN the data. This new probability distribution is called the "a posteriori probability" or simply the "posterior." Bayesian approaches can summarize their uncertainty by giving a range of values on the posterior probability distribution that includes 95% of the probability -- this is called a "95% credibility interval."

A Bayesian partisan might criticize the frequentist confidence interval like this: "So what if 95 out of 100 experiments yield a confidence interval that includes the true value? I don't care about 99 experiments I DIDN'T DO; I care about this experiment I DID DO. Your rule allows 5 out of the 100 to be complete nonsense [negative values, impossible values] as long as the other 95 are correct; that's ridiculous."

A frequentist die-hard might criticize the Bayesian credibility interval like this: "So what if 95% of the posterior probability is included in this range? What if the true value is, say, 0.37? If it is, then your method, run start to finish, will be WRONG 75% of the time. Your response is, 'Oh well, that's ok because according to the prior it's very rare that the value is 0.37,' and that may be so, but I want a method that works for ANY possible value of the parameter. I don't care about 99 values of the parameter that IT DOESN'T HAVE; I care about the one true value IT DOES HAVE. Oh also, by the way, your answers are only correct if the prior is correct. If you just pull it out of thin air because it feels right, you can be way off."

In a sense both of these partisans are correct in their criticisms of each others' methods, but I would urge you to think mathematically about the distinction -- as Srikant explains.

I was wondering, is the uncertainty in the estimate (21,000 to 120,000 lives saved) due to us not knowing the 'posterior' probabilities with high confidence?

 

[Murph]

Good questions TB. All I can think is that none of these questions came up in wuhan. Trains and buses cancelled. No one gets out or in. Leave the house once every three days for food. What we have going on isnt approaching what they did.

Gav - for what it's worth northumbria is different. You guys have so much space up there and can genuinely get out and not bump into people more power to you. (In case you cant tell I am not being sarcastic). The slightest relaxation of what we are allowed to do in Sheffield and the peak will be rammed again.

I think the police were right to run that shaming footage. I would fucking love to go out for a climb up at curbar right now. Or go to griffs.

[Murph]

Just realised I might have replied to the wrong thread. Oh well - apologies. CV and CV climbing both cover the same ground more or less!

[Stu Littlefair]

Noticed it used 'credible interval' instead of 'confidence interval' in its estimate that the European distancing measures:
'saved 59,000 deaths up to 31 March [95% credible interval 21,000-120,000]

...

I was wondering, is the uncertainty in the estimate (21,000 to 120,000 lives saved) due to us not knowing the 'posterior' probabilities with high confidence?

Basically yes. We know how many deaths there have been (ish) but must rely on the models to know how many they would have.

The model predictions can depend on very large numbers of input parameters, each of which is known only imprecisely. So there’s a really large spread in how many deaths their model predicts depending on what you assume the true parameter values are.

[dunnyg]

The output of a bayesian model is a distribution, so the size of the "credible interval" is one way of describing that posterior distribution, i.e. number of deaths.

If you imagine you draw two values from 2 (prior)distributions at random, and times them together, and save the output value.

Repeat this process a million times times and you will get a distribution of output values - your posterior distribution.

If you have broader distributions on your first 2 values (your prior), you will get a broader distribution of your posterior, and a larger credibility interval.

If you are modelling more complex things, you will have a more complex equation with more parameters, and the distribution size of some prior distributions have greater effects.

The general rule of models always stands though, if your prior assumptions are crap, the output is crap.

(not a comment on any linked models etc. etc.)

[tomtom]

From an environmental systems perspective - despite all the nonlinear / exponential responses of the models, the DATA shows the behaviour of CV19 in response to mitigation measures has been remarkably similar.

It’s clear that only very clear interventions (China) or massive testing and tracing efforts (Korea - and probably soon Germany) do anything. Most of Western Europe (Germany aside) is following the same trajectory with some further ahead than others. In other words looking at the larger picture it’s remarkably consistent in how it behaves.

I suspect if you were a pandemic modeller and you got something that looked like France when you where hoping for Italy (in your curve of cases/deaths) you’d be very pleased you were doing something right!

Part of me - therefore thinks that whilst the models and simulations are really good for showing how the dynamics of virus spread can change - or could be altered by social distancing - when you look over a large area where some areas will be doing it well and others not so - it makes little difference.

Either REALLY get with the program and sort your isolation/testing/tracing shot out or you’re just Shaving a little bit off the top of the curve (which may anyway be worthwhile)

My caveat is that the USA is looking like a train wreck happening in slow motion at the moment and could be an exemplar of not what to do (why are there still internal flights in the USA ffs!)

Edit. Not an expert view - but I do develop environmental models for a living.

[Stu Littlefair]

The new ICL paper that Pete refers to backs up your analysis. It’s a really nice bit of work actually.

Conceptually it’s really simple. Model the deaths the way you’d do it naively by modelling the infections the way Barrows tried - each person infects Ro~2-3 and the whole cycle repeats every ~5-6 days.

Then add the complexity needed because some people will infect more/less and might be infectious for more/less time. And allow Ro to be changed by interventions.

Finally, add a layer to a hierarchy that allows each country to be different, but makes sure they are all drawn from the same underlying distribution - eg assume all countries have the same Ro under each type of intervention unless the data says they don’t.

It’s a nice piece of work and their conclusions seem robust against their assumptions. It’s just the conclusions are so vague.

Basically almost all countries modelled might have reached the magic number needed to control the outbreak. Spain might, but probably hasn’t and Sweden urgently needs to sort itself out :-(

[Johnny Brown]

It’s clear that only very clear interventions (China) or massive testing and tracing efforts (Korea - and probably soon Germany) do anything.

So what's going on in Japan? The Americans I follow seem to think it's widespread mask wearing and the WHO are wrong in suggesting they are only useful for the infected and those caring for them.

[Oldmanmatt]

It’s clear that only very clear interventions (China) or massive testing and tracing efforts (Korea - and probably soon Germany) do anything.

So what's going on in Japan? The Americans I follow seem to think it's widespread mask wearing and the WHO are wrong in suggesting they are only useful for the infected and those caring for them.

Whilst this is true, masks are pretty “ normal” across East Asia. The Japanese have a cultural habit of avoiding contact etc etc.

There could be a panoply of seemingly minor differences that accumulate into massive outcomes for viral spread. Pinning it all on masks seems to oversimplify.

I mean, wear a mask, a lot of people are. We’re wearing gloves to go shopping and wiping down and using hospital grade Milton on stuff and...

No reason not to take excessive precautions if you want to.

Given the shortage of such for NHS staff etc, I can’t see the Gov. stumping up 68 million of them for everyone though.
 

[galpinos]

https://www.theguardian.com/us-news/2020/mar/30/las-vegas-parking-lot-homeless-shelter

Sickening. Surely there are empty hotels? Even abandoned ones would be better than this.

Currently over 100k empty hotel beds in Vegas.

[tomtom]

How’s Mrs Galpinos view on all the CV19 stuff Nick?

[galpinos]

How’s Mrs Galpinos view on all the CV19 stuff Nick?

She's pretty worried* but it's the calm before the storm in Manchester currently. It's playing havoc with what she can offer her patients and the reduction in the service offered will lead to poorer care/shortened life (no surgery or general medical support). They have movied to telephone consultations where they can and are trying to isolate in-patients when positive but that has only been so effective. She's waiting to be reassigned to the GMEX if we follow down the London path as there won't be enough work for the full lung cancer team. The approach to dealing with it has been very piecemeal and every hospital/trust seems to have its own plan.

*Her sister is a Renal Consulant in London who has been drafted into Covid support on a 7 day on 7 day off rota. Pretty full on. Also, our youngest collapsed a lobe in her lung pre Christmas, it took a lot of work to get it back up and just before this all kicked off we got her most recent immunology report back which was pretty poor so her potential vulnerability is an added stress.

[tomtom]

Christ Nick that’s enough to worry about. Sure you’ve got enough of a network to get you stuff but if you need anything let me know.

3/10 houses in our road have had it.. (now all recovering - all at home) so there’s a fair bit around here. Though what the hospitals are getting now is probably stuff transmitted 4 weeks ago etc... behind the London and Birmingham curve.

[petejh]

It’s clear that only very clear interventions (China) or massive testing and tracing efforts (Korea - and probably soon Germany) do anything.

So what's going on in Japan? The Americans I follow seem to think it's widespread mask wearing and the WHO are wrong in suggesting they are only useful for the infected and those caring for them.

I wondered about Japan when this was all starting to go global. They've among the lowest rates of testing, they haven't enforced much of a lock-down, and they have the world's oldest population (but it says in the article below the old in Japan are much more separated from the rest of society than in Europe).
Quite a good overview article here which looks at various theories, including that they just might not be reporting: https://www.vox.com/covid-19-coronavirus-explainers/2020/3/28/21196382/japan-coronavirus-cases-covid-19-deaths-quarantine
Don't read the 'elderly tragically separated' article it's just sad.

[Oldmanmatt]

It’s clear that only very clear interventions (China) or massive testing and tracing efforts (Korea - and probably soon Germany) do anything.

So what's going on in Japan? The Americans I follow seem to think it's widespread mask wearing and the WHO are wrong in suggesting they are only useful for the infected and those caring for them.

I wondered about Japan when this was all starting to go global. They've among the lowest rates of testing, they haven't enforced much of a lock-down, and they have the world's oldest population (but it says in the article below the old in Japan are much more separated from the rest of society than in Europe).
Quite a good overview article here which looks at various theories, including that they just might not be reporting: https://www.vox.com/covid-19-coronavirus-explainers/2020/3/28/21196382/japan-coronavirus-cases-covid-19-deaths-quarantine
Don't read the 'elderly tragically separated' article it's just sad.

Korea too.

There’s a huge difference in social/cultural attitudes. The Korean response seems as though fear had a large part to play in the effectiveness of their measures.
Rather than the denial that, at first (and still in some quarters) pervaded attitudes in the West?

[Coops_13]

Denver stay-at-home (lockdown) extended to April 30th. I really hope we can get through this in time for the Alpine season in summer but the chance is looking smaller and smaller...

[Muenchener]

Denver stay-at-home (lockdown) extended to April 30th. I really hope we can get through this in time for the Alpine season in summer but the chance is looking smaller and smaller...

Bavaria reviewing the situation after Easter; Austria already said it's unlikely schools will re-open this school year.

I have a mate who was keen to go the Écrins this summer, and another recruiting for a trip to Lofoten. Will be surprised if either actually happens, and thankful if I'm able to get out again locally.

[Offwidth]

I would fancy being in the shoes of the senior management morons who thought this was a good idea:

https://www.theguardian.com/society/2020/mar/31/nhs-staff-gagged-over-coronavirus-protective-equipment-shortages

[moose]

Simple explainer on the issues involved in modelling future Covid deaths:

https://fivethirtyeight.com/features/why-its-so-freaking-hard-to-make-a-good-covid-19-model/

[tomtom]

Simple explainer on the issues involved in modelling future Covid deaths:

https://fivethirtyeight.com/features/why-its-so-freaking-hard-to-make-a-good-covid-19-model/

I’m gonna be (importantly) pedantic here - as we don’t actually know what good is yet.

In other words we don’t have any complete data on what’s happening or happened.

And to go all Rumsfeld on y’all - there’s presumably a few unknown unknowns...

[Oldmanmatt]

563.

Today.

I know, stupid to post the number. Pointless, negative, unsurprising.

I just feel the need to acknowledge it, somehow.

[tomtom]

563.

Today.

I know, stupid to post the number. Pointless, negative, unsurprising.

I just feel the need to acknowledge it, somehow.

It’s fucking grim. And this without hospital beds being full (as far as I know). It implies that either this is a very bad day - or we’re just as riddled as Spain and Italy but haven’t got the testing penetration to show what would me a massively higher figure.

Makes 20k seem like a dream outcome. Very sorry to say that...

[Oldmanmatt]

Yeah, that’s the current thinking amongst some observers:

https://news.sky.com/story/coronavirus-the-data-suggests-the-uk-is-on-course-for-many-thousands-of-deaths-11966517

[petejh]

Makes 20k seem like a dream outcome. Very sorry to say that...

I'm fairly sure a couple of weeks ago when Chris Witty first started mentioning 20,000 deaths he used that figure in the context of the peak. I think this has been subtly altered to imply it will be 20,000 in total.

If you agree that cases (thus hospitalisations - thus deaths) double every 3-4 days currently, and that the doubling figure slows towards the peak, and the peak is approx 10-14 days in the future, then we're on a course to reach approx. 20,000 deaths at the peak.
But it should be clear there will be a down slope.
I just think they don't want the message to look too grim at one time. Drip feed the grimness.

[Ru]

I'm fairly sure a couple of weeks ago when Chris Witty first started mentioning 20,000 deaths he used that figure in the context of the peak. I think this has been subtly altered to imply it will be 20,000 in total.

No, he said social distancing could reduce the deaths to less than 20,000.