Eliezer.… This post terrifies me. How on earth can humans overcome this problem? Everyone is tainted. Every group is tainted. It seems almost fundamentally insurmountable...
And yet somehow, bridges stay up.
So you go in search of a mathematics of FAI that produces precision-grade predictions.
However, you’re correct that all vaguely optimistic arguments produced by various meddling dabblers about how friendly they expect their AI design to be, can be tossed out the window.
It’s not a challenge we have the option of avoiding, either.
And yet somehow, bridges stay up.
So you go in search of a mathematics of FAI that produces precision-grade predictions.
However, you’re correct that all vaguely optimistic arguments produced by various meddling dabblers about how friendly they expect their AI design to be, can be tossed out the window.
It’s not a challenge we have the option of avoiding, either.