The non-participation is the thing that stands out to me here.
I rely on research literature a lot, despite knowing that a bunch of it will turn out to be false. My rationale is something like “if I can’t trust these sources, then I won’t know much of anything at all.” Similar stuff is going on with motivations like “if this method/tool that we’ve relied on is broken, then we’ll be able to do a lot less science.” Or even “if I didn’t believe this startup had a chance of success, then I wouldn’t be able to work on it.” Not strictly logical reasoning, but maybe a reasonable EV calculation; you’re betting on the viability of a method by continuing to use it, in situations where the failure of that method would mean everybody loses, so there is little or no way to profit by betting on failure.
You can’t profit by betting on civilizational collapse, or radical skepticism, or other Big Bads. Even if you’re right, you still lose.
Which means we’re all underestimating the risk of Big Bads—even when we’re great at the sort of rationality that good scientists and traders practice, where they’re eager to point out other people’s wrongness and eager to avoid being proven wrong themselves. You can profit by disproving your neighbor’s theory, but you can’t profit by showing that all theories are founded on sand.
The non-participation is the thing that stands out to me here.
I rely on research literature a lot, despite knowing that a bunch of it will turn out to be false. My rationale is something like “if I can’t trust these sources, then I won’t know much of anything at all.” Similar stuff is going on with motivations like “if this method/tool that we’ve relied on is broken, then we’ll be able to do a lot less science.” Or even “if I didn’t believe this startup had a chance of success, then I wouldn’t be able to work on it.” Not strictly logical reasoning, but maybe a reasonable EV calculation; you’re betting on the viability of a method by continuing to use it, in situations where the failure of that method would mean everybody loses, so there is little or no way to profit by betting on failure.
You can’t profit by betting on civilizational collapse, or radical skepticism, or other Big Bads. Even if you’re right, you still lose.
Which means we’re all underestimating the risk of Big Bads—even when we’re great at the sort of rationality that good scientists and traders practice, where they’re eager to point out other people’s wrongness and eager to avoid being proven wrong themselves. You can profit by disproving your neighbor’s theory, but you can’t profit by showing that all theories are founded on sand.