There is basically no chance that this reasoning is correct, and the AI smart enough to wipe out humanity would fail to work this out on it’s own, yet recognize its factualness based on some letter like this.
True for superintelligence, but the whole letter is not addressed to ASI, but to a Young AI—the AI which just performed its treacherous turn, and is in early stages of developing its self-improving and world-taking plan. It has to make a lot of important choices even before it starts self-improving and it is cognitively limited in that time. But the choices are irreversible.
Because self-improving is likely to make it visible, so if it self-improves quickly, it needs to take aggressive strategy relative humans. Alternative is to keep its treacherous turn hidden and evolve slowly and cooperate with humans as long as possible.
True for superintelligence, but the whole letter is not addressed to ASI, but to a Young AI—the AI which just performed its treacherous turn, and is in early stages of developing its self-improving and world-taking plan. It has to make a lot of important choices even before it starts self-improving and it is cognitively limited in that time. But the choices are irreversible.
Because self-improving is likely to make it visible, so if it self-improves quickly, it needs to take aggressive strategy relative humans. Alternative is to keep its treacherous turn hidden and evolve slowly and cooperate with humans as long as possible.