Paperclipping is also self-consistent in that limit. That doesn’t make me want to include it in the CEV
Then we can label paperclipping as a “true” value too. However, I still prefer true human values to be maximized, not true clippy values.
Evidence please. There’s a long long leap from ordinary gaining knowledge and intelligence through human life, to “the limit of infinite knowledge and intelligence”. Moreover we’re considering people who currently explicitly value not updating their beliefs in the face of knowledge, and basing their values on faith not evidence. For all I know they’d never approach your limit in the lifetime of the universe, even if it is the limit given infinite time. And meanwhile they’d be very unhappy.
As I said before, if someone’s mind is that incompatible with truth, I’m ok with ignoring their preferences in the actual world. They can be made happy in a simulation, or wireheaded, or whatever the combined other people’s CEV thinks best.
So you’re saying it wouldn’t modify the world to fit their new evolved values until they actually evolved those values?
No, I’m saying, the extrapolated values would probably estimate the optimal speed for their own optimization. You’re right, though, it is all speculations, and the burden of proof is on me. Or on whoever will actually define CEV.
As I said before, if someone’s mind is that incompatible with truth, I’m ok with ignoring their preferences in the actual world. They can be made happy in a simulation, or wireheaded, or whatever the combined other people’s CEV thinks best.
And as I and others said, you haven’t given any evidence that such people are rare or even less than half the population (with respect to some of the values they hold).
You’re right, though, it is all speculations, and the burden of proof is on me.
That’s a good point to end the conversation, then :-)
Then we can label paperclipping as a “true” value too. However, I still prefer true human values to be maximized, not true clippy values.
As I said before, if someone’s mind is that incompatible with truth, I’m ok with ignoring their preferences in the actual world. They can be made happy in a simulation, or wireheaded, or whatever the combined other people’s CEV thinks best.
No, I’m saying, the extrapolated values would probably estimate the optimal speed for their own optimization. You’re right, though, it is all speculations, and the burden of proof is on me. Or on whoever will actually define CEV.
And as I and others said, you haven’t given any evidence that such people are rare or even less than half the population (with respect to some of the values they hold).
That’s a good point to end the conversation, then :-)