I wonder how much taking these facts into account helps. The error that gets people round up to simplistic goals such as “maximize pleasure” could just be replayed at a more sophisticated level, where they’d say “maximize neural correlates of wanting” or something like that, and move to the next simplest thing that their current understanding of neuroscience doesn’t authoritatively forbid.
Sure. And then I write a separate post to deal with that one. :)
There are also more general debunkings of all such ‘simple algorithm for friendly ai’ proposals, but I think it helps to give very concrete examples of how particular proposed solutions fail.
″Moore proposes an alternative theory in which an actual pleasure is already present in the desire for the object and that the desire is then for that object and only indirectly for any pleasure that results from attaining it. “In the first place, plainly, we are not always conscious of expecting pleasure, when we desire a thing. We may only be conscious of the thing which we desire, and may be impelled to make for it at once, without any calculation as to whether it will bring us pleasure or pain. In the second place, even when we do expect pleasure, it can certainly be very rarely pleasure only which we desire.″
I wonder how much taking these facts into account helps. The error that gets people round up to simplistic goals such as “maximize pleasure” could just be replayed at a more sophisticated level, where they’d say “maximize neural correlates of wanting” or something like that, and move to the next simplest thing that their current understanding of neuroscience doesn’t authoritatively forbid.
Sure. And then I write a separate post to deal with that one. :)
There are also more general debunkings of all such ‘simple algorithm for friendly ai’ proposals, but I think it helps to give very concrete examples of how particular proposed solutions fail.
It helps insofar as the person’s conscious mind lags behind in awareness of the object being maximized.
″Moore proposes an alternative theory in which an actual pleasure is already present in the desire for the object and that the desire is then for that object and only indirectly for any pleasure that results from attaining it. “In the first place, plainly, we are not always conscious of expecting pleasure, when we desire a thing. We may only be conscious of the thing which we desire, and may be impelled to make for it at once, without any calculation as to whether it will bring us pleasure or pain. In the second place, even when we do expect pleasure, it can certainly be very rarely pleasure only which we desire.″
Sounds like a decent methodology to me.