Could you imagine, for example, that an AI CEO who somehow managed to align an AI to himself and his intents would step down if the board pointed out it legally had the right to remove him?
And if the human CEO decided to go along with the board, but the AI disagreed that this was in their agreed-upon interests?
“Now wait a minute, I wrote you!” ″I’ve gotten 2,415 times smarter since then.” — Ed Dillinger and the MCP, in Tron (1982)
The “galaxies” business is pretty silly. If someone promises you immortality in heaven — literally! — then it’s probably a good idea to check exactly what that person is up to, this quarter, down here on earth. The track record of humans promising that going along with their movement will get you immortality in heaven does not look all that great.
Good point, there is a paragraph I chose not to write about how insanely irresponsible this is. Driving people to now maximally invest/research AI for some insane future promise, while in reality ASI is basically guaranteed to kill them. Kind of like Heaven’s Gate drinking poison to get into that spaceship that’s waiting behind that comet.
And if the human CEO decided to go along with the board, but the AI disagreed that this was in their agreed-upon interests?
“Now wait a minute, I wrote you!”
″I’ve gotten 2,415 times smarter since then.”
— Ed Dillinger and the MCP, in Tron (1982)
The “galaxies” business is pretty silly. If someone promises you immortality in heaven — literally! — then it’s probably a good idea to check exactly what that person is up to, this quarter, down here on earth. The track record of humans promising that going along with their movement will get you immortality in heaven does not look all that great.
Good point, there is a paragraph I chose not to write about how insanely irresponsible this is. Driving people to now maximally invest/research AI for some insane future promise, while in reality ASI is basically guaranteed to kill them. Kind of like Heaven’s Gate drinking poison to get into that spaceship that’s waiting behind that comet.