Re hardware limit: flagging the implicit assumption here that network speeds are spotty/unreliable enough that you can’t or are unwilling to safely do hybrid on-device/cloud processing for the important parts of self-driving cars.
(FWIW I think the assumption is probably correct).
I think my base rate for basic comprehension failures is at a similar level!
Wow thank you! I found this really helpful.
Many of the thinker-heroes that we revere now, like Jeremy Bentham, Isaac Newton, Florence Nightingale, and Benjamin Franklin, among others, had ideas that were considered deeply weird within their time. Many of them were considered quite popular even within their own time.
Information request: How large are leads in military tech for wars historically? My naive impression was that there was a >20 year lead for, eg, US-Vietnam, USSR-Afghanistan, both the first and second Italy-Ethiopian wars, great emu war etc.
I’m also curious how much of a lead the US currently has over, e.g., other permanent members of the UN Security Council.
I agree with the following points:
That European countries very much appear to have this under control
That they did much better than the US and Latin America
Right-wing populist leaders did worse than I expected, in a non-coincidental way (Brazil’s Bolsonaro is another example to add to the list).
“trying to control the narrative over dealing with problems is a particularly dangerous approach with infectious diseases” very strongly agreed. I’m a big fan of this write-up by NunoSempere, and this historian’s touching reflection on the Spanish flu.
I think it’s likely our disagreements are somewhat about framing than actual empirical differences. For example, “they seems poised to have gotten it under control before it ended up everywhere, though they didn’t catch it enough to prevent spread at first, which would have been the goal” is a phrase I’d use to describe South Korea and Singapore, not Western Europe, where almost every locale had community transmission. I’d use “they caught it enough to prevent spread” to describe places like Mongolia with zero or close to zero community transmission, or contained community transmission to a single region.
I agree that Western European governments should get a lot of relative credit for managing to prevent more deaths, disability, and wanton economic destruction, despite being in an initially bad spot. But thousands of people nonetheless died, and those deaths appeared to be largely preventable (in a practical, humanly doable sense). So while I think we should also a) emphasize the relative successes (because in these dark times it’s good to both hold on to hope and be grateful for what we have), and b) be unequivocally clear that the other Western governments mostly did better than the US, I do want to not lose sight of the target and also be clear that the relative failings of the US under Trump does not excuse the lesser failings of other institutions and governments.
I know the conversation these days is (rightly) about preventing presymptomatic transmission from the wearer, but I’m personally still at ~80% that masks probably protect the wearer at least a little, though agree that the effect may not be huge.
people find it far easier to forgive others for being wrong than being right
Harry Potter and the Half-Blood Prince
First of all, I really appreciate this postmortem. Admitting times when you were wrong couldn’t have been an easy task, particularly if/when you staked a lot of your identity and reputation to being right. As EA and rationalist individuals and institutions become older and more professionalized, I’m guessing that institutional pressures will increasingly push us further and further away from wanting to admit mistakes; so I sincerely hope we get in the habit of publicly recognizing mistakes early on. (Unfinished list of my own mistakes, incidentally). I hope to digest your post further and offer more insightful thoughts, but here are some initial thoughts:
Addendum on masks:
Another consideration about masks is that masks turn out in practice to be very reusable, a fact we (or at least I) should have investigated a lot more in early March.
On hospital-based transmission:
I don’t know how much you believed in it, but as presented, this appears to be merely (ha!) a forecasting error rather than a strategic error. In the absence of a clear counterfactual, I don’t think you were obviously wrong here, since it’s quite plausible that if a lot of people like you ignored/downplayed the role of hospital-based transmission, it’d have gotten a lot worse.
On being a jerk re Jim and Elizabeth’s post:
For what it’s worth, I also (privately) asked them to take it down because I had similar considerations to you and thought the thing they wrote about masks was unilateralist-y and a bit of an infohazard. I think I was wrong there. But I think I mostly was object-level wrong about the relative tradeoffs and harms. To the extent I updated now, a) I updated object-level on how much I should cooperate or desire others to cooperate with specific institutions, and b) I updated broadly (but not completely) in general favor of openness and against censorship.
I continue to maintain that if I (and possibly you) had the same object-level beliefs as before, it was not incorrect to consider it an info-hazard (but not all object-level info-hazards are worth suppressing! Particularly if release promotes the relevant meta-level norms more than it harms), though of course not an existential one.
You said you think superforecasting is
materially worse than [you] hoped it would be at noticing rare events early.
I don’t know how high your hopes were, but for what it’s worth, I think this proves too much. I’m not sure about the exact aggregation algorithms that the Open Phil Good Judgement covid-19 project was running, but I feel like all I can realistically gather was that “of this specific set of part-time superforecasters that were on the Open Phil-funded project, more than 50% of them were way too optimistic.”
While it’s certainly some evidence against superforecasters being good at noticing rare events early, I don’t think it’s sufficient evidence against superforecasters being able to do this, and I definitely don’t think this is a lot of evidence against superforecasting as a process.
As you weakly allude to, if you were on the project and paying attention more, you would probably have done better. Likewise, I know other superforecasters who I think were much more pessimistic than the GJ median. I suspect superforecasters who regularly read LessWrong and the EA Forum would have done better; and if I were to design a better system for superforecasting on rare events, I’d a) prime people to pay attention to a lot of rare events first, and b) have people train and score on log-loss or some other scoring system that’s more punishing of overconfidence than Brier.
(All that said, I think Metaculus did okay but not great on covid-y questions relative to what someone with high hopes for prediction aggregation algorithms might reasonably expect).
On US Gov’t Institutions:
I think there was a bunch of insights that your policy research experience has colored. For example, you mention how you trusted the FDA to have done a lot better under Scott Goettlieb. This might be obvious to you, but it’s something I didn’t even really think about until you highlighted this point. You also highlight a lot of useful specific uncertainties about whether the issue was political directors under Trump or nonpolitical directors of specific institutions. I think all of these things are very useful to know from the perspective of a policy researcher like yourself (and for students of US policy), since how to reform institutions is very decision-relevant to you and many other EAs.
That said, at a very coarse level, I think I’m a lot more cynical than you are implying with regard to how well US institutions would have handled this pre-Trump. It’s possible we’re not actually disagreeing, so I’m curious on your counterfactual probabilities on things being an order of magnitude better (<20,000 Americans dead of COVID-19 by now, say) in the following two worlds:
a) Clinton administration continuing all of Obama’s policies?
b) Clinton administration continuing all of Obama’s policies except for US CDC in China being equally understaffed as they are in our timeline.
My reasoning for why I’m generally pretty cynical (at least conditional upon this pandemic spreading at all, maybe a larger international presence could have helped contained it early) in those counterfactual worlds:
1) There’s sort of an existing counterfactual for preparedness of governments with a broadly American/Western culture but as competent at governance as a typical European country. It’s called Europe. And I feel like every large geographically Western country was pretty bad at preparedness? People are praising Germany’s response, but when it comes down to it, Germany has 9000+ confirmed covid-19 deaths in a population of 83 million, or >100 deaths/million, despite taking a large economic hit to suppress the pandemic. Japan had <1000 confirmed deaths in a population of 126.5 million. Now Japan was bad at testing, so maybe Japan actually had ~4000 deaths. But even at those numbers (~31 deaths/million), Japan still had <1/3 the number of deaths per capita as Germany. And object-level, Japan seemed to have screwed up a bunch of important things, so there’s a simple transitivity argument where if a high-income country did worse than Japan, their policies/institutions couldn’t have been that great.
Maybe I’m harping on this too much, but I really don’t want us to succumb to the tyranny of low expectations here.
Now some culturally Western countries did fine (Australia, New Zealand). I’m not sure why they did well (maybe it’s because they’re islands, maybe seasonality is bigger than I think so Southern hemisphere had a huge initial advantage early on, maybe because they’re around 10-15% East Asian so people had enough ties to China to be worrying earlier, maybe low population density, maybe their institutions are newer and better, maybe just luck), but regardless, I’d counterfactually bet on the response of Hillary’s America looking more like a slightly less competent Europe or maybe Canada and less like Australia/NZ.
2) I didn’t look at it that much, but at the high-level, the US response to 2009 H1N1 looked more competent, but ultimately the response didn’t seem sufficient to have achieved containment if the mortality rates were as high as people thought it’d be? (Not sure of this, willing to be convinced otherwise on this one).
3) Some inside-view reasoning about specific actors.
Anyway, all these gripes aside, thank you again for your thoughtful (and well-written!) post. That couldn’t have been easy to write, and I really appreciate it.
 Your post actually me to thinking about how I should be more honest/introspective about my strategic and not just predictive mistakes, so thanks for that! I plan to update the list soon with some strategic mistakes as well. For example, I considered myself to be on the “right” side of masks epistemically but not strategically.
 I’m maybe 35% on a) and 30% on b). A lot of the probability mass is considerations on there being enough chaos/sensitivity to initial conditions that this pandemic maybe wouldn’t have happened at all, rather than Obama’s or Hillary’s response being an order of magnitude better conditional upon there being an epidemic.
First of all, I really appreciate this article! It helps me conceptualize cleanly some vocabulary that’s flying around in the rationalist community that I previously didn’t really understand.
To me, the most obvious missing archetype is The Truth-Giver or perhaps The Teacher.
The Teacher is concerned with only conveying truthful messages. She will usually tell the truth, but she may occasionally omit truthful things, or even (rarely) tell half-truths if she thinks it’s easier to convey truthful messages via half-truth. Importantly, she’s different from the Sage or the Pragmatist in that she’s not concerned with other object-level consequences, only in conveying truthful messages.
Consider the claim:
There’s a pandemic headed our way from China
Suppose that The Teacher believes that the following is more correct:
There’s a pandemic headed our way from Italy.
The Teacher will choose usually to clarify and say the full message, however if she only has one bit of response, she’ll say “yes” to “Is there a pandemic headed our way from China?” Importantly (unlike the Pragmatist) she’ll do this even if the perceived consequences are negative, as long as the subject gets more truthful information than they otherwise would have.
Against Level 1 and Level 2 players, The Teacher will never see a need to resort to Level 3. However, against a fully Level 3 player, she will (begrudgingly) issue utterances correctly conveying the ideological faction she’s on, as that’s the most relevant/only bit to transfer to fully Level 3 players.
I think Teacher roles are incredibly important in practical everyday communication, since all information is lossy, inferential gaps are common, attention and text is limited, etc. Indeed, I would go so far as argue that Teacher roles are often preferable to Oracle roles in murky situations if the purpose is to collectively seek truth.
Hi! First post here. You might be interested in knowing that not only is the broken radio example isomorphic to “Chicken,” but there’s a real-life solution to the Chicken game that is very close to “destroying your receiver.” That is, you can set up a “committment” that you will, in fact, not swerve. Of course, standard game theory tells us that this is not a credible threat (since dying is bad). Thus, you must make your commitment binding, eg., by ripping out the steering wheel.