When I see proposals that involve convincing everyone on the planet to do something, I write them off as loony-eyed idealism and move on. So, creating FAI would have to be hard enough that I considered it too “impossible” to be attempted (with this fact putatively being known to me given already-achieved knowledge), and then I would swap to human intelligence enhancement or something because, obviously, you’re not going to persuade everyone on the planet to agree with you.
But is it really necessary to persuade everyone, or 99.9% of the planet? If gwern’s analysis is correct (I have no idea if it is) then it might suffice to convince the policymakers of a few countries like USA and China.
I see. So you do have an upper bound in mind for the FAI problem difficulty, then, and it’s lower than other alternatives. It’s not simply “shut up and do the impossible”.
This calls for a calculation. How hard creating an FAI would have to be to have this inequality reversed?
When I see proposals that involve convincing everyone on the planet to do something, I write them off as loony-eyed idealism and move on. So, creating FAI would have to be hard enough that I considered it too “impossible” to be attempted (with this fact putatively being known to me given already-achieved knowledge), and then I would swap to human intelligence enhancement or something because, obviously, you’re not going to persuade everyone on the planet to agree with you.
But is it really necessary to persuade everyone, or 99.9% of the planet? If gwern’s analysis is correct (I have no idea if it is) then it might suffice to convince the policymakers of a few countries like USA and China.
I see. So you do have an upper bound in mind for the FAI problem difficulty, then, and it’s lower than other alternatives. It’s not simply “shut up and do the impossible”.