Correct, I should have had that in mind when I wrote my comment.
The alternative to runaway AI is an AI whose values are acceptable to us. I belive that this is a design problem. The perfect system should be something like what’s described in The Fun Theory Sequence and posts like When “yang” goes wrong describes two extremes as anarchy and tyranny, and I think it’s fair to assume that the perfect world must be a balance between these two extremes.
But this topic is not trivial at all, I will give you that. I also don’t think we agree very much about what the ideal future or AI looks like. I think some naive “Minimize suffering” optimization target is the most popular, but if you ask me that’s merely due to an old misunderstanding of Buddhism.
You could also say that any AI will become “Runaway”, and that we merely get to influence the direction. I’m pushing in my direction of choice, even if the force is tiny.
But I will stop here, as I have a lot of opinions on the subject, which probably don’t fit the consensus very well.
The context seems to be saving the world from runaway AI, which can’t be nontrivially described that way.
Correct, I should have had that in mind when I wrote my comment.
The alternative to runaway AI is an AI whose values are acceptable to us. I belive that this is a design problem. The perfect system should be something like what’s described in The Fun Theory Sequence and posts like When “yang” goes wrong describes two extremes as anarchy and tyranny, and I think it’s fair to assume that the perfect world must be a balance between these two extremes.
But this topic is not trivial at all, I will give you that. I also don’t think we agree very much about what the ideal future or AI looks like. I think some naive “Minimize suffering” optimization target is the most popular, but if you ask me that’s merely due to an old misunderstanding of Buddhism.
You could also say that any AI will become “Runaway”, and that we merely get to influence the direction. I’m pushing in my direction of choice, even if the force is tiny.
But I will stop here, as I have a lot of opinions on the subject, which probably don’t fit the consensus very well.