Honestly, I’m more worried about this. See the Ethical Injunction mini-Sequence.
There are options now available to him that genuinely are more powerful, but… even Harry makes mistakes, and even Harry falls prey to overconfidence sometimes.
I’ve never been impressed by the Ethical Injunction business. It seems a “get out of unpleasant implications of my explicit moral claims free” card.
Not really. Ethical Injunctions really mean “design an AI to do this, but don’t do it yourself because you’re probably wrong.”
It’s entirely possible to create a problem involving an epistemic state a human can’t hold: see Pascal’s Muggle.
Lol
Pascal’s Muggle
It’s a real thing. Look it up. It’s basically the symmetric problem to Pascal’s Mugger.
In a slightly eccentric sense of the word “real”.
As in “well-known idea, not specific to myself.”
Honestly, I’m more worried about this. See the Ethical Injunction mini-Sequence.
There are options now available to him that genuinely are more powerful, but… even Harry makes mistakes, and even Harry falls prey to overconfidence sometimes.
I’ve never been impressed by the Ethical Injunction business. It seems a “get out of unpleasant implications of my explicit moral claims free” card.
Not really. Ethical Injunctions really mean “design an AI to do this, but don’t do it yourself because you’re probably wrong.”
It’s entirely possible to create a problem involving an epistemic state a human can’t hold: see Pascal’s Muggle.
Lol
It’s a real thing. Look it up. It’s basically the symmetric problem to Pascal’s Mugger.
In a slightly eccentric sense of the word “real”.
As in “well-known idea, not specific to myself.”