(nods) Yeah. I take it for granted that there are multiple ways to create the “smarter version” steven0461 was referring to, since the alternative seems implausibly neat, and that it’s therefore (hypothetically) up to the AI to figure out how to create a Dave2 whose utterances have the desired value.
Of course, if we live in a convenient universe where there’s only one possible “extrapolated Dave,” or at least an obviously superior candidate (which of course opens a whole infinite regress problem: how do I build a system I trust to decide which of the many possible simulations of my improved self it should use in order to determine what I would want if I were better? And if I somehow can trust it to do that much in an ethical fashion, haven’t I already solved the automated ethics problem? What work is left for CEV to do?)
In the less convenient worlds, the idea of averaging all the possible extrapolated mes into a weighted vector sum, along with all the possible extrapolated everyone elses, had not occurred to me, but is better than anything else I can think of.
Also, what exactly means “smarter version”? Is there only one way to make a smarter version, or are they many possible smarter versions?
What if the smarter versions have different utility functions—should AI take some weighted average of their functions?
(nods) Yeah. I take it for granted that there are multiple ways to create the “smarter version” steven0461 was referring to, since the alternative seems implausibly neat, and that it’s therefore (hypothetically) up to the AI to figure out how to create a Dave2 whose utterances have the desired value.
Of course, if we live in a convenient universe where there’s only one possible “extrapolated Dave,” or at least an obviously superior candidate (which of course opens a whole infinite regress problem: how do I build a system I trust to decide which of the many possible simulations of my improved self it should use in order to determine what I would want if I were better? And if I somehow can trust it to do that much in an ethical fashion, haven’t I already solved the automated ethics problem? What work is left for CEV to do?)
In the less convenient worlds, the idea of averaging all the possible extrapolated mes into a weighted vector sum, along with all the possible extrapolated everyone elses, had not occurred to me, but is better than anything else I can think of.