Implementing CEV does not necessarily mean taking over the world or solving all our problems. Indeed, the whole point of CEV is that we don’t know what it’ll turn out to be; if we could know what its output would look like, we wouldn’t need a superintelligence to figure it out for us. There was (and still is, as far as I can tell) a strong expectation that CEV will significantly remake the world, but that is a conjectured probable implication (and debating it doesn’t matter too much anyway), not a definitional quality or a deductive implication.
Implementing CEV does not necessarily mean taking over the world or solving all our problems. Indeed, the whole point of CEV is that we don’t know what it’ll turn out to be; if we could know what its output would look like, we wouldn’t need a superintelligence to figure it out for us. There was (and still is, as far as I can tell) a strong expectation that CEV will significantly remake the world, but that is a conjectured probable implication (and debating it doesn’t matter too much anyway), not a definitional quality or a deductive implication.
That FAI will significantly change things is a pretty conclusive antiprediction. Status quo hath no moral power.
Agreed. We aren’t working on new technology with the intent of letting it gather dust while people continue to suffer and die.