And for how hard the problem is, I’d emphasize how likely we are to be wrong about determining or extrapolating exactly what we want, in each sense listed above.
An engaging way to open a talk is to challenge the audience, then give them this info as first steps. If I can suggest an example, I’d say something like “Asimov’s 3 Laws looked innocuous and even in our best interest, but they were terribly flawed. So can we find a real solution? One that will hold for powerful AGI? This is an open problem.”
If you don’t run out of time, connect it back to self-modifying code (if only to ground the problem for a group of programmers). We need software that will self-modify to satisfy dynamic criteria that may exist, but are obscured to us. And we may only get one shot.
That will make someone ask about take-off and cooperating agents, and you can use the Q&A time to mention FOOMing, negentropy etc.
And for how hard the problem is, I’d emphasize how likely we are to be wrong about determining or extrapolating exactly what we want, in each sense listed above.
An engaging way to open a talk is to challenge the audience, then give them this info as first steps. If I can suggest an example, I’d say something like “Asimov’s 3 Laws looked innocuous and even in our best interest, but they were terribly flawed. So can we find a real solution? One that will hold for powerful AGI? This is an open problem.”
If you don’t run out of time, connect it back to self-modifying code (if only to ground the problem for a group of programmers). We need software that will self-modify to satisfy dynamic criteria that may exist, but are obscured to us. And we may only get one shot.
That will make someone ask about take-off and cooperating agents, and you can use the Q&A time to mention FOOMing, negentropy etc.