The key hypothesis is that it’s not uniformly harder to train AGIs in the safer regimes—rather, it’s primarily harder to get started in those regimes. Once an AI reaches a given level of intelligence, then transitioning to a safer regime might not slow down the rate at which it gains intelligence very much—but might still decrease the optimisation pressure in favour of that AI being highly agentic and pursuing large-scale goals.
Can’t choice of programming language (or coding platform) affect the optimization pressures? [if everyone ends up learning poorly-designed choices, it can cause a lot of weird behaviors long-run, so a safer regime would include, like, a decent programming language]. It’s like harder to get started on blockchains that aren’t as bloated as bitcoin or ethereum.
>
Can’t choice of programming language (or coding platform) affect the optimization pressures? [if everyone ends up learning poorly-designed choices, it can cause a lot of weird behaviors long-run, so a safer regime would include, like, a decent programming language]. It’s like harder to get started on blockchains that aren’t as bloated as bitcoin or ethereum.