I mean, because it asserts that the same people who advocate for thinking about pivotal acts, and who popularized the pivotal act notion would say anything like “We don’t know, whatever it thinks is best and has the highest chance of working according to the values we are writing into it.”.
This is explicitly not what Nate and Eliezer and some MIRI people are trying to do. The whole point of a minimum pivotal act is to make it so that you don’t have to align your AI all the way so that you just have it go off and do whatever is best according to the values we programmed into it. It’s so that you have as close as possible to a concrete plan of what you want to do with the AI, planning for the world where you didn’t fully solve the AI Alignment problem and can just fully defer to the AI.
I mean, because it asserts that the same people who advocate for thinking about pivotal acts, and who popularized the pivotal act notion would say anything like “We don’t know, whatever it thinks is best and has the highest chance of working according to the values we are writing into it.”.
This is explicitly not what Nate and Eliezer and some MIRI people are trying to do. The whole point of a minimum pivotal act is to make it so that you don’t have to align your AI all the way so that you just have it go off and do whatever is best according to the values we programmed into it. It’s so that you have as close as possible to a concrete plan of what you want to do with the AI, planning for the world where you didn’t fully solve the AI Alignment problem and can just fully defer to the AI.