Needn’t be total—average would suggest creating one single extremely happy being—probably not human.
Needn’t only include hedonic pleasure—a preference utilitarian might support eliminating humans and replacing them with beings whose preferences are cheap to satisfy (hedonic pleasure being one cheap preference). Or you could want multiple kinds of pleasure, but see hedonic as always more efficient to deliver as proposed in the post.
You will be gone and something which does want to be a big integer will replace you and use your resources more effectively. Both hedonistic and preference utilitarianism demand it.
Utility as I care about it is probably the result of information processing. Not clear why information should only be able to be processed in that way by human type minds, let alone fleshy ones.
What are they then?
Silly to worry only about the preferences of your present self—you should also act to change your preferences to make them easier to satisfy. Your potential future self matters as much as your present self does.
Sorry, this may be a stupid question, but why is it a good for people to get cyonically frozen? Obviously if they don’t they won’t make it to the future—but other people will be born or duplicated in the future and the total number of people will be the same.
Why care more about people who live now than future potential people?
Probably the proportion of ‘kill all humans’ AIs that are friendly is low. But perhaps the proportion of FAIs that ‘kill all humans’ is large.
Why is it worse to die (and people cryonically frozen don’t avoid the pain of death anyway) than to never have been born? Assuming the process of dying isn’t painful, they seem the same to me.
OK, I’m new to this.
What if we could create a wirehead that made us feel as though we were doing 1 or 2? Would that be satisfactory to more people?
I can understand valuing oneself more than others (simple selfishness is unsurprising), but I think Eliezer is saying cryonics is a positive good, not just that it benefits some people at the equal expense of others.
If uploading gets up as is probably required for people to be woken up, the future will be able to make minds as diverse as they like.
I am using (total) preference utilitarianism to mean: “we should act so as to maximise the number of beings’ preferences that are satisfied anywhere at any time”.
“As for hedonistic utilitarians, why would any existing mind want to build something like that or grow into something like that?”
Because they are not selfish and they are concerned about the welfare of that being in proportion to its ability to have experiences?
“Further, why would something like that be better at seizing resources?”
That’s a weakness, but at some point we have to start switching from maximising resource capture to using those resources to generate good preference satisfaction (or good experiences if you’re a hedonist). At that point a single giant ‘utility monster’ seems most efficient.
That’s because chances for us to go extinct seem many. If we are necessarily held back from space for thousands of years, it’s very unlikely we would last that long just here on Earth.
I am skeptical of the evolutionary explanation he poses for inactivity.
I don’t believe large numbers of people were typically thrown out of hunter gatherer bands for incompetence, surely not more than inactive people (http://books.google.com.au/books?sitesec=reviews&id=ljxS8gUlgqgC). And in how many crisis situations is doing nothing really the best option? Hiding from a predator would surely be one of only a few.
Possibly doing nothing is a good idea for hunter gatherers in case of starvation, but that seems worth checking in the anthropology research. If starvation were a frequent risk, lethargy would surely been prompted by insufficient food intake, which is rare for humans today. We wouldn’t just be lazy for that reason all the time; during times of abundance you ought to gather and store as much food as possible.
Apparently hunter gatherer bands were egalitarian, so it’s unlikely people would have been beaten up by (non-existent) leaders just for hunting and gathering well, especially given food was shared. Again the conditions under which people would be picked on in bands are something we can find out by looking at existing anthropology research. Nonetheless it’s hard to imagine that hunter gatherer bands which would push out members merely for contributing to the food intake of the group would be the most successful. We don’t favour do-nothings over well-meaning incompetents today as far as I can tell.
I lurked til a few weeks back when I read something I really disagreed with.
I lurked until I read something I really disagreed with.
To Larks and Shminux—I am twisting the idea of arbitrage, to be more like ‘economic profit’ or ‘being sure to beat the market rate of return on investment/altruism’. Maybe I should stop using the term arbitrage.
“Isn’t your point basically just that consumer surplus can be unusually high for individuals with unusual demand functions because the supply (of chances to do good) is fixed so lower demand ⇒ lower price?”
Yes, though the supply surve just slopes upwards—it isn’t vertical.
I could re-write the principle as ‘when supply curves slope upwards the purchases that offer the highest consumer surplus to you will mostly be things that you value but others don’t.’ On financial markets that isn’t so important as most investors have very similar values, but in other areas it matters more.
I like your point about feedback loops in finance, but shouldn’t proven effective philanthropists attract more donations if people cared about efficacy?
Who cares about humans exactly? I care about utility. If the AI thinks humans aren’t an efficient way of generating utility, we should be eliminated.