My claim is more that if we do get a chimp running at 10^3 and it somehow turns into ASI, I believe the resulting ASI is likely to be much more aligned, and we probably all get to live rather whatever we get scaling GPT6 etc.
Ok but I don’t think this is very relevant? My point is that, before you get something powerful & aligned, before that point, you got something that someone else could turn into something powerful & not-aligned by doing non-safe machine learning with your thing as a component / ingredient.
I think a human brain would have decisive headroom compared to current AIs. Potentially whatever is needed for RSI doesn’t exist in current AI or maybe never under the current regime, but it does exist in the human mind.
So then if a project gets a true full human WBE, before that point it created capabilities, not already present in an AI, but yes in a computer. At that point, those capabilities could be expropriated and trained on by AI capabilities researchers.
Ok but I don’t think this is very relevant? My point is that, before you get something powerful & aligned, before that point, you got something that someone else could turn into something powerful & not-aligned by doing non-safe machine learning with your thing as a component / ingredient.
So then if a project gets a true full human WBE, before that point it created capabilities, not already present in an AI, but yes in a computer. At that point, those capabilities could be expropriated and trained on by AI capabilities researchers.