I put “Friendliness” in quotes in the title, because I think what we really want, and what MIRI seems to be working towards, is closer to “optimality”
Meh. If we can get a safe AI, we’ve essentially done the whole of the work. Optimality can be tacked on easily at that point, bearing in mind that what may seem optimal to some may be an utter hellish disaster to others (see Repugnant Conclusion), so some sort of balanced view of optimality will be needed.
If we can get a safe AI, we’ve essentially done the whole of the work. Optimality can be tacked on easily at that point
I’m not seeing this. Suppose we’ve got a Oracle AI that’s been safely boxed, which we can use to help us solve various technical problems. How do we get to optimality from there, before others people take our Oracle AI technology and start doing unsafe things with it? I’ve argued, in this post, that getting to optimality requires solving many hard philosophical problems, and it doesn’t seem like having an AI that’s merely “safe” helps much with that.
so some sort of balanced view of optimality will be needed
Meh. If we can get a safe AI, we’ve essentially done the whole of the work. Optimality can be tacked on easily at that point, bearing in mind that what may seem optimal to some may be an utter hellish disaster to others (see Repugnant Conclusion), so some sort of balanced view of optimality will be needed.
I’m not seeing this. Suppose we’ve got a Oracle AI that’s been safely boxed, which we can use to help us solve various technical problems. How do we get to optimality from there, before others people take our Oracle AI technology and start doing unsafe things with it? I’ve argued, in this post, that getting to optimality requires solving many hard philosophical problems, and it doesn’t seem like having an AI that’s merely “safe” helps much with that.
Sure, no argument there.
To refine both of our ideas: I was thinking that safety for an autonomous or unleashed AI was practically the same thing as optimality.
But I agree that there may be systems of containments that could make certain AI designs safe, without needing optimality.