I always thought it was interesting that SIAI/MIRI seems to have a very libertarian bias in terms of present-day politics, but its endgame puts all power in a singleton, who divides resources according to CEV, weighting each person’s preferences equally, i.e. totalitarian communism, which is the diametrical opposite of libertarianism. Not that I’m saying this is a contradiction or hypocrisy, as different situations do require different politics—if anything, it shows a great deal of cognitive flexibility.
I don’t really see a contradiction here. The idea is that perfectly rational agents would choose to merge into a singleton that maximizes a combination of their utility functions, instead of wasting resources on competition. CEV is not the only mechanism that can achieve that, also see Carl’s paper on superorganisms. Humans can’t quite do that yet because we don’t have good technologies for cooperation, precommitment or self-modification, and also because we don’t have good enough math on bargaining which is necessary for merging.
I always thought it was interesting that SIAI/MIRI seems to have a very libertarian bias in terms of present-day politics, but its endgame puts all power in a singleton, who divides resources according to CEV, weighting each person’s preferences equally, i.e. totalitarian communism, which is the diametrical opposite of libertarianism. Not that I’m saying this is a contradiction or hypocrisy, as different situations do require different politics—if anything, it shows a great deal of cognitive flexibility.
I don’t really see a contradiction here. The idea is that perfectly rational agents would choose to merge into a singleton that maximizes a combination of their utility functions, instead of wasting resources on competition. CEV is not the only mechanism that can achieve that, also see Carl’s paper on superorganisms. Humans can’t quite do that yet because we don’t have good technologies for cooperation, precommitment or self-modification, and also because we don’t have good enough math on bargaining which is necessary for merging.