I think democratic human control is extremely unlikely even with a US actor winning the race.
andrew sauer
Karma: 411
Yeah seriously. Omelas is better than our world in every possible way INCLUDING child abuse and exploitation
Unintended pregnancies don’t sound like a benefit to me.
Mark this, anyone who wishes to align AI by training it on human values. What might an intellectually superior AI conclude, is the appropriate way to treat intellectually inferior beings?
If you want a future that isn’t hell, reckoning conclusively with this issue, and all issues like it, is an absolute necessity.
dating apps where people could signal their wealth by buying the most expensive virtual good available.
This is a Molochian race to the bottom, similar to a dollar auction. The items have value to people, but only insofar as they have more of them than others. The people overall are therefore not better off for having these items, because the signalling game is zero-sum; there is only so much of the actual reward they are after.
The problem with arguments like this is that they are typically circular. At the end of the day you are using math to try to show why math is necessary for reasoning or whatever.
Best to just take a few unjustified axioms so that you’re honest about the uncertainty at the bottom of any worldview
Does the above chart assume all survival situations are better than non-survival? Because that is a DANGEROUS assumption to make.
Maybe hypocrisy in the sense that someone acts like they agree with the social consensus in order to avoid persecution, when in fact they don’t and are doing things which don’t conform to it. Legalized blackmail would encourage people to not mind their own business and become morality police or witch hunters even about things which don’t actually hurt them or anybody else.
Consider the effect legalized blackmail would have had on the gay community before widespread acceptance for a particularly brutal and relatively recent example
Keep in mind also, that humans often seem to just want to hurt each other, despite what they claim, and have more motivations and rationalizations for this than you can even count. Religious dogma, notions of “justice”, spitefulness, envy, hatred of any number of different human traits, deterrence, revenge, sadism, curiosity, reinforcement of hierarchy, preservation of traditions, ritual, “suffering adds meaning to life”, sexual desire, and more and more that I haven’t even mentioned. Sometimes it seems half of human philosophy is just devoted to finding ever more rationalizations to cause suffering, or to avoid caring about the suffering of others.
AI would likely not have all this endless baggage causing it to be cruel. Causing human suffering is not an instrumentally convergent goal. So, most AIs will not have it as a persistent instrumental or terminal goal. Not unless some humans manage to “align” it. Most humans DO have causing or upholding some manner of suffering as a persistent instrumental or terminal goal.
This is equivalent to the game Westley played with Vizzini. You know, if Westley didn’t cheat. I like to call it “Sicilian Chess” for that reason, though that’s just me.
Trump shot an arrow into the air; it fell to Earth, he knows not where...
Probably one of the best succinct summaries of every damn week that man is president lmao
LOL @ the AI-warped book in that guy’s hands
Now you can!
Gwern seems to think this would be used as a way to get rid of corrupt oligarchs, but… Wouldn’t this just immediately be co-opted by those oligarchs to solidify their power by legally paying for the assassinations of their opponents? Markets aren’t democratic, because a small percentage of the people have most of the money.
To be fair, my position is less described by that Quirrell quote and more by Harry’s quote when he’s talking to Hermione about moral peer pressure:
“The way people are built, Hermione, the way people are built to feel inside, is that they hurt when they see their friends hurting. Someone inside their circle of concern, a member of their own tribe. That feeling has an off-switch, an off-switch labelled ‘enemy’ or ‘foreigner’ or sometimes just ‘stranger’. That’s how people are, if they don’t learn otherwise.”
Unlike Quirrell I give people the credit for actually caring, rather than pretending to care, about people. I just don’t think that extends to very many people, for most people.
Fun fact for those reading this in the far future, when Eliezer said “effective altruist” in this piece, he most likely was using the literal meaning, not referring to the EA movement, as that name hadn’t been coined yet.
Wildbow (the author of Worm) is currently writing a story with a quite similar premise
In fact I think it’s safe to say that we’d collectively allocate much more than 1/millionth of our resources towards protecting the preferences of whatever weak agents happen to exist in the world (obviously the cows get only a small fraction of that).
Sure, but extrapolating this to unaligned AI is NOT an encouraging sign. We may allocate greater than 1/million of our resources to animal rights, but we allocate a whole lot more than that to goals which diametrically go against the preferences of those animals such as eating meat and cheese and eggs; we allocate MUCH more resources to “animal wrongs” than animal rights, so to speak.
So to show an AI will be “nice” to humans at all, it is not enough to suppose that it might have some 1/million “nice to humans” term. It requires showing that that term won’t be outweighed handily by the rest of its utility function.
100%. Social contract gives no consideration to the powerless, and this fact is the source of much of the horrible opinions in the world.
If you’re averaging over time as well as space, that isn’t an option. All the people you kill will just drag down your average, and the one person who is really happy at the end of it all will barely register in the grand scheme of lives across time. In practice average utilitarianism just reduces to regular old utilitarianism, just with the zero point set at the average utility for a life across a history much vaster than you can affect, instead of set at nonexistence or whatever
Perhaps average utilitarianism would consider a world which only ever had one super happy person in it, as better than our world. But that seems less obviously false to me than the idea we should kill everyone to achieve that, which average utilitarianism wouldn’t recommend when properly considered.
I agree that many worlds has little bearing on this question though. Unless it’s to claim that you should expect the effective zero point to be different, because for whatever reason you think that our branch is particularly good or particularly bad.