I was going to write up my thoughts on this but it would be easier to just comment here.
I agree with your assessments for almost all of these. I was most impressed by your understanding of the politics in Q9 & 11 (China and Hydroxychloroquine) and the predicting the lack of consensus for Q14 & 15.
A couple where I have a question:
1. On 6⁄7 (US highest toll official & unofficial) I had a bit more probability on Brazil (similar to India, more than China) – given large population (2/3rds US) and approach of the government.
Regarding official vs unofficial, you only mention deliberate lying but I had more expectation of insufficient / bad testing hiding true amounts than lying. According to WSJ Russia’s excess deaths are 4.8x higher than their official deaths (compared to 1.7x for US). This isn’t enough to overtake the US but I think this gives an idea of the scale of the potential problem. Mexico’s excess deaths are higher than Brazil’s despite having 35% fewer official cases. (India isn’t included in those numbers—excess deaths stats aren’t available I think).
Does that change your mind as to what a good prediction would have been?
2. On q17 (second wave) your prediction for p(17|16) is ~29%. Given that we are in a world where there is a general consensus that summer made things less bad, 29% seems low for a second wave even given the difficult operationalisation? My corresponding number was 50% which still seems better to me (although I messed up q16 so we actually predicted the same for 17 itself). In terms of which way it resolves, I think just numbers of deaths resolves this as clearly true (assuming by Autumn we mean 22 Sep – 21 Dec), both in terms of official result and intent:
Was there a second wave in Autumn? Yes, in late Autumn running into early Winter.
Russia’s higher death toll might or might not be mostly Covid, but I figured its population wasn’t high enough. Even if all of Russia gets it, they’d need a pretty high fatality rate to catch us given what was likely happening here. Brazil similarly I figured wouldn’t document all that well and had a smaller pop.
2. I still think that there’s enough different ways this can fail that 30% is reasonable, and I dunno where the 29% comes from here? Presumably it would be higher than the 30% baseline for p(17|16), what am I missing? (And the way it resolves to false is if we say it’s a third wave that happened rather than a second, not that the numbers don’t match, and I agree that this is wrong and it resolves to true).
Yes, I agree Russia was unlikely to be above US for population reasons, I mentioned them more as an example of how bad under-reporting can be—I can’t think of a way other than Covid to get 147k unaccounted for excess deaths but I could be missing something. I had concerns about this in all 3 of China, India and Brazil (although I guess there’s the chance that we wouldn’t get (accurate) excess deaths numbers anyway). 85% for 6 seems right but only dropping 5% for 17 seems low.
A commenter on Scott’s post has made a case for India deaths being higher than US (enough to convince Scott it seems).
Its possible / likely that I’m still missing how difficult it is to win a parlay but:
Given Covid is seen as seasonal by the end of the year, there was very likely some wave in Autumn—the main question is whether it meets the conditions set out in 17
At the time of prediction it seemed almost certain that we would get below the thresholds with the next month or two
I expected (but wasn’t certain) that a second wave would take us back above one of those thresholds.
There remains the question of having a wave in the middle (Autumn wave is therefore not second wave). This was somewhere that my model was expecting a profile in the US more like what happened in the UK/Europe where cases/deaths were at a very low level for most of the Summer. This is a common thread in a few of my other predictions about US numbers—I generally underpredicted slightly but noticeably and this was a significant cause for that. So yeah, definitely an oversight from me in that regards.
I was going to write up my thoughts on this but it would be easier to just comment here.
I agree with your assessments for almost all of these. I was most impressed by your understanding of the politics in Q9 & 11 (China and Hydroxychloroquine) and the predicting the lack of consensus for Q14 & 15.
A couple where I have a question:
1. On 6⁄7 (US highest toll official & unofficial) I had a bit more probability on Brazil (similar to India, more than China) – given large population (2/3rds US) and approach of the government.
Regarding official vs unofficial, you only mention deliberate lying but I had more expectation of insufficient / bad testing hiding true amounts than lying. According to WSJ Russia’s excess deaths are 4.8x higher than their official deaths (compared to 1.7x for US). This isn’t enough to overtake the US but I think this gives an idea of the scale of the potential problem. Mexico’s excess deaths are higher than Brazil’s despite having 35% fewer official cases. (India isn’t included in those numbers—excess deaths stats aren’t available I think).
Does that change your mind as to what a good prediction would have been?
2. On q17 (second wave) your prediction for p(17|16) is ~29%. Given that we are in a world where there is a general consensus that summer made things less bad, 29% seems low for a second wave even given the difficult operationalisation? My corresponding number was 50% which still seems better to me (although I messed up q16 so we actually predicted the same for 17 itself). In terms of which way it resolves, I think just numbers of deaths resolves this as clearly true (assuming by Autumn we mean 22 Sep – 21 Dec), both in terms of official result and intent:
Was there a second wave in Autumn? Yes, in late Autumn running into early Winter.
Russia’s higher death toll might or might not be mostly Covid, but I figured its population wasn’t high enough. Even if all of Russia gets it, they’d need a pretty high fatality rate to catch us given what was likely happening here. Brazil similarly I figured wouldn’t document all that well and had a smaller pop.
2. I still think that there’s enough different ways this can fail that 30% is reasonable, and I dunno where the 29% comes from here? Presumably it would be higher than the 30% baseline for p(17|16), what am I missing? (And the way it resolves to false is if we say it’s a third wave that happened rather than a second, not that the numbers don’t match, and I agree that this is wrong and it resolves to true).
Yes, I agree Russia was unlikely to be above US for population reasons, I mentioned them more as an example of how bad under-reporting can be—I can’t think of a way other than Covid to get 147k unaccounted for excess deaths but I could be missing something. I had concerns about this in all 3 of China, India and Brazil (although I guess there’s the chance that we wouldn’t get (accurate) excess deaths numbers anyway). 85% for 6 seems right but only dropping 5% for 17 seems low.
A commenter on Scott’s post has made a case for India deaths being higher than US (enough to convince Scott it seems).
p(17|16) = p(17) / p(16) = 0.2 / 0.7 ~ 0.29 (as p(17|¬16) = 0)
Its possible / likely that I’m still missing how difficult it is to win a parlay but:
Given Covid is seen as seasonal by the end of the year, there was very likely some wave in Autumn—the main question is whether it meets the conditions set out in 17
At the time of prediction it seemed almost certain that we would get below the thresholds with the next month or two
I expected (but wasn’t certain) that a second wave would take us back above one of those thresholds.
There remains the question of having a wave in the middle (Autumn wave is therefore not second wave). This was somewhere that my model was expecting a profile in the US more like what happened in the UK/Europe where cases/deaths were at a very low level for most of the Summer. This is a common thread in a few of my other predictions about US numbers—I generally underpredicted slightly but noticeably and this was a significant cause for that. So yeah, definitely an oversight from me in that regards.