Strategic effort can be viewed through an epistemic lens (seeking truth) or a coordination lens (amassing consensus and directing resources). This is also true for AI safety, and x-risk.
However, this binary viewpoint misses the point of why we discuss strategy in the first place. The goal is not to simply be right, rationally or politically. The goal is to win.
The art of strategy is the art of winning.
Of course, the objection that comes to mind: What does “winning” even mean in ASI safety? Isn’t that what we are trying to figure out here?
We must separate what from how.
When deciding the strategic win condition for x-risk, there is no need to overcomplicate it. Winning here means survival.
Survival famously requires adaptability. In terms of epistemics and coordination: We need epistemics fast enough to detect failure early, and coordination tight enough to pivot instantly.
Survival also generally requires resilience to failure modes. This is achieved by taking actions that generate defense in depth and slack.
Slack is key to resilience. A 100 % efficient system has no ability to absorb shock, and a train going full speed has a long brake time vs. a slow one.
As the military proverb goes: No plan survives enemy contact.
Strategic effort can be viewed through an epistemic lens (seeking truth) or a coordination lens (amassing consensus and directing resources). This is also true for AI safety, and x-risk.
However, this binary viewpoint misses the point of why we discuss strategy in the first place. The goal is not to simply be right, rationally or politically. The goal is to win.
The art of strategy is the art of winning.
Of course, the objection that comes to mind: What does “winning” even mean in ASI safety? Isn’t that what we are trying to figure out here?
We must separate what from how.
When deciding the strategic win condition for x-risk, there is no need to overcomplicate it. Winning here means survival.
Survival famously requires adaptability. In terms of epistemics and coordination: We need epistemics fast enough to detect failure early, and coordination tight enough to pivot instantly.
Survival also generally requires resilience to failure modes. This is achieved by taking actions that generate defense in depth and slack.
Slack is key to resilience. A 100 % efficient system has no ability to absorb shock, and a train going full speed has a long brake time vs. a slow one.
As the military proverb goes: No plan survives enemy contact.