It seems to me that becoming more knowledgeable and smarter is gradually altering our utility functions.
That’s probably down to memes and memetic hijacking. Advanced civilisation results in a greater density of memes, and they have greater say in civilisation’s values. These values usually include reduced reproduction for humans—because of the resource competition between memes and genes—thus the demographic transition.
But what is it that we are approaching if the extrapolation of our volition becomes a purpose in and of itself?
As I say, the actual effect is probably down to more memes. If that continues, we’ll probably get this.
That’s probably down to memes and memetic hijacking. Advanced civilisation results in a greater density of memes, and they have greater say in civilisation’s values. These values usually include reduced reproduction for humans—because of the resource competition between memes and genes—thus the demographic transition.
As I say, the actual effect is probably down to more memes. If that continues, we’ll probably get this.