cousin_it
I think we might have a factual disagreement, which to you looks like “cousin_it is misrepresenting my opinion as being anti-democratic” and to me looks like “Benquo is misrepresenting my opinion as agreeing with Moldbug”. But it’s also ok to let things simmer down a bit, so I’ll check out.
Yes, exactly. And I’d say more: there’s a strong analogy to everything here. Baking bread? Imagine a baker a few hundred years ago getting interrogated by a philosopher about “but how do you really know what yeast is and why it’s needed?” Or music, imagine the early rock-n-roll groups being interrogated by a philosopher about what makes rock music good, and the philosopher doing the Socratic “five whys” thing. It’s just nonsense. Lots of things in life are art: you do it and then later, maybe, someday, someone else understands it. Law and morality too, that’s why we have juries, you can’t take the subjectivity out and you shouldn’t, sometimes there should be lenience because people just feel it.
Athens gathered a jury of 500 people for Socrates. What more can any of us ask? When a philosopher proposes to replace all this with “but do you know what piety is? come on, define exactly how all your morals work by which you’re judging me!” this is not a neutral thing, this is a bid for something, for a certain direction of society. His student evolved it into the philosopher-king proposal; yep, checks out. Benquo likes this direction and I don’t.
I don’t think his views changed much. Here’s him writing in 1956:
Throughout history colonies have been among the most powerful agents for the spread of the arts and science and ways of life that constitute civilization. For the future, it seems that mankind will have to learn to do without this ancient and well-tried method. I think mankind will have to depend, not upon force or domination, but upon the inherent attractiveness of a civilized way of life.
Which almost exactly mirrors this part from the 1915 essay:
Such wars, however, belong now to the past. The regions where the white men can live are all allotted, either to white races or to yellow races to whom the white man is not clearly superior, and whom, in any case, he is not strong enough to expel. Apart from small punitive expeditions, wars of colonization, in the true sense, are no longer possible.
I think his position can be summarized like this: he cared about certain ideals, the “Western way of life” and “freedoms”, and believed that they were worth spreading in this way. His support for preemptive nuclear war to protect civilization was part of the same pattern.
expansionist, totalitarian regime
during a talk to the Royal Empire Society
“Empire” and “expansionist” are synonyms, so the juxtaposition here can seem funny. But Russell was actually being consistent: he believed that when “civilized” people do ruthless expansion up to and including genocide, that’s good, but when “uncivilized” ones do it, it’s bad. (He, of course, being one of the “civilized”). For example, in The Ethics of War:
By a “war of colonization” I mean a war whose purpose is to drive out the whole population of some territory and replace it by an invading population of a different race. Ancient wars were very largely of this kind, of which we have a good example in the Book of Joshua. In modern times the conflicts of Europeans with American-Indians, Maoris, and other aborigines in temperate regions, have been of this kind. Such wars are totally devoid of technical justification, and are apt to be more ruthless than any other war. Nevertheless, if we are to judge by results, we cannot regret that such wars have taken place. They have the merit, often quite fallaciously claimed for all wars, of leading in the main to the survival of the fittest, and it is chiefly through such wars that the civilized portion of the world has been extended from the neighborhood of the Mediterranean to the greater part of the earth’s surface. The eighteenth century, which liked to praise the virtues of the savage and contrast them with the gilded corruption of courts, nevertheless had no scruple in thrusting the noble savage out from his North American hunting grounds. And we cannot at this date bring ourselves to condemn the process by which the American continent has been acquired for European civilization. In order that such wars may be justified, it is necessary that there should be a very great and undeniable difference between the civilization of the colonizers and that of the dispossessed natives. It is necessary also that the climate should be one in which the invading race can flourish. When these conditions are satisfied the conquest becomes justified, though actual fighting against the dispossessed inhabitants ought, of course, to be avoided as far as is compatible with colonizing. Many humane people will object in theory to the justification of this form of robbery, but I do not think that any practical or effective objection is likely to be made.
Moldbug thinks the choice is between first-best (a king with high standards) and second-best (a democracy with low standards). I think, like in many other things, first-best here is not a real option. The real choice is between second-best (a democracy with low standards) and third-best (an unaccountable king with even lower standards—selfish, a believer in wrong theories, a fuckup and so on). This is why Churchill said that “democracy is the worst form of government except for all those other forms”.
I think participatory leads to better standards than having a king. In general my views are opposite of Moldbug’s on everything.
Sure, I understand what you’re asking for, but that’s not a neutral question. It’s basically the heart of the whole disagreement. Either people are free to have moral feelings, even if not very explained, and act on them and try to convince others, and success in so convincing is the sole criterion; a jury of 500 Athenians can decide that they want Socrates gone, and that’s legitimate, they don’t have to have logic behind their vote. This is the democratic view. Or people have to explain and justify their morals, can be second guessed and overridden by philosophers, eventually this leads to philosopher kings. This is antidemocratic and plenty of people have pointed this out about Socrates’ lineage. I know which side I stand on.
When I first read the dialogue many years ago, my impression was that Euthyphro was a kind of proto-Christian, having the correct moral feeling that murder ought to be punished even if the murderer is your relative. And standing up for that feeling, even in the face of opposition from other relatives. I thought that was pretty cool. In comparison, the fact that he couldn’t logically explain it under cross-examination and fell back to “that’s what the gods want” doesn’t worry me so much. It isn’t a sign of decline of public life or anything; people are just bad at explaining their feelings, always have been, always will be. So yeah, pencil me in on the anti-Socrates side I guess. (Though I did upvote your post: it’s well written, even if the argument falls apart for me.)
For me the solution by Fitch (the “logical school” mentioned in the Wikipedia article) is pretty conclusive. I wrote a post about it sometime ago, extending it to handle another variation of the paradox.
Why are your numbers more indicative, though? There’s tons of possible numbers to choose from that go every which way. When I was figuring out for myself how to compare countries, I tried to choose numbers that were hardest to fudge, most indicative of bigger trends and least cherrypicked. That seems like the only way to clear things up, like defining QALYs in EA.
About migration as a criterion, consider Filipino migrants in Saudi Arabia. The Philippines are a flawed democracy, while Saudi Arabia is an absolute monarchy with a poor human rights record and a higher incarceration rate. Why the migration then? Economic reasons. If China becomes as rich per capita as the US, I’m not sure they’d be as eager to migrate. Migration often depends on which country is richer, not which country is more free.
I think with these kind of subjective things you can fudge a lot. “The Chinese are repressed” “Sure, but they have healthcare” “Sure, but they have Uyghurs” “Sure, but you have reservations” and there you go again. It’s the same problem that effective altruism was trying to solve, that the “society for curing rare diseases in cute puppies” is not effective, so you’ve got to find numbers that are hard to fudge. Or with measuring crime, theft reports are affected by a lot of factors and you need to look at murder rate which is harder to fudge. Same logic here. You’ve got to choose some objective numbers that reflect reality.
Given that, I stand by my choice of numbers. In Stalin’s USSR, Nazi Germany and Maoist China the internal repression was reflected in incarceration rates. But in today’s China, the incarceration rate is 1/5th that of the US. This means the repression in today’s China is much lower than these historical examples, and not obviously worse than in the US.
Good post, thank you for opening this discussion.
I’ve long been waiting for the rationalist community to have a “Sword of Good” kind of reckoning about the US vs China, as in “choosing between good and bad is about deciding which is which”. To lay out some rails, I think there are two general approaches to this question.
1) The objective approach. Here you find some numbers, like percent of healthcare coverage and so on, and compare the two countries on these numbers. I’ve given this a lot of thought and believe that it comes down to two “topline metrics”, which ought to weigh more than any other metric when deciding a country’s moral worth. They are: how many people the country represses internally, and how many people the country kills in foreign wars. These are the topline metrics for a government being good domestically and being good internationally. For example, Nazi Germany had horrible repression internally and started horrible aggressive wars, and that’s the entire reason we think it was bad. Well, today’s US has much higher incarceration rate than today’s China and also kills much more people in foreign wars, so there’s that.
2) The aspirational approach. Here one can say that China is simply nationalist, while the US has a more universalist mission to spread democracy and free speech and so on. There’s something to be said for this view, but I think it’s getting less true over time. The last democracy-building success stories were many decades ago, the more recent US wars have just been messing countries up. And whether the US cares about universal human rights is also a bit of trick question. For example, Anthropic recently said its red line was “mass domestic surveillance”. When I pointed out that I as a non-US person shouldn’t be surveiled either, multiple US folks (including AI lab employees) told me right here on LW that I should be more ok with being surveiled by the US. It gave me the impression that my rights aren’t quite as real, and that I should expect today’s US to simply follow nationalist interests when it comes to me, not be a light unto me or something.
I think making moral AI philosophically competent is about as hard as making corrigible AI that keeps us philosophically competent, or even sane, as we use it. The way I think about such things is based on R. Scott Bakker’s short story “Crash Space”, the main point is in the postscript, which is amazing and I’ll just quote it in full:
Reverse engineering brains is a prelude to engineering brains, plain and simple. Since we are our brains, and since we all want to be better than what we are, a great many of us celebrate the eventuality. The problem is that we happen to be a certain biological solution to an indeterminate range of ancestral environments, an adventitious bundle of fixes to the kinds of problems that selected our forebears. This means that we are designed to take as much of our environment for granted as possible—to neglect. This means that human cognition, like animal cognition more generally, is profoundly ecological. And this suggests that the efficacy of human cognition depends on its environments.
We neglect all those things our ancestors had no need to know on the road to becoming us. So for instance, we’re blind to our brains as brains simply because our ancestors had no need to know their brains for what they were in the process of becoming us. This is why our means of solving ourselves and others almost certainly consists of ‘fast and frugal heuristics,’ ways to generate solutions to complicated problems absent knowledge of the systems involved. So long as the cues exploited remain reliably linked to the systems solving and the systems to be solved, we can reliably predict, explain, and manipulate one another absent any knowledge of brain or brain function.
Herein lies the ecological rub. The reliability of our heuristic cues utterly depends on the stability of the systems involved. Anyone who has witnessed psychotic episodes has firsthand experience of consequences of finding themselves with no reliable connection to the hidden systems involved. Any time our heuristic systems are miscued, we very quickly find ourselves in ‘crash space,’ a problem solving domain where our tools seem to fit the description, but cannot seem to get the job done.
And now we’re set to begin engineering our brains in earnest. Engineering environments has the effect of transforming the ancestral context of our cognitive capacities, changing the structure of the problems to be solved such that we gradually accumulate local crash spaces, domains where our intuitions have become maladaptive. Everything from irrational fears to the ‘modern malaise’ comes to mind here. Engineering ourselves, on the other hand, has the effect of transforming our relationship to all contexts, in ways large or small, simultaneously. It very well could be the case that something as apparently innocuous as the mass ability to wipe painful memories will precipitate our destruction. Who knows? The only thing we can say in advance is that it will be globally disruptive somehow, as will every other ‘improvement’ that finds its way to market.
Human cognition is about to be tested by an unparalleled age of ‘habitat destruction.’ The more we change ourselves, the more we change the nature of the job, the less reliable our ancestral tools become, the deeper we wade into crash space.
Like, imagine we have a corrigible AI. Then a person using it can go off track very easily, by using the AI to help modify the AI and the person in tandem. To prevent that, the corrigible AI needs to have a lot of alignment-type stuff (don’t manipulate the user, don’t mislead, don’t go down certain avenues, what’s good what’s bad, etc) and that’s not too much different from having a moral AI. And conversely, a moral AI could also delegate some philosophical questions to us, if it had a careful enough way to do so.
So I think this difficulty is about the same in all three scenarios, it doesn’t differ between them very much. The biggest thing that would help is slowing down, you’re right on that. My concern in this thread is kinda orthogonal: modulo this aspect of alignment, there’s another bad thing happening, and it’s different in the three scenarios. From the perspective of that bad thing (power concentration) we’d better steer away from number 3, and somewhat prefer 1 to 2 as well. I remember talking about it with you a few months ago.
Corrigibility has a large surface area of “don’t try to manipulate the people correcting you”. As AI power increases, that can get basically as fuzzy and difficult as making moral AI to begin with. What is manipulation? What words are okay to say? What is good? To me the difficulty of the target doesn’t seem that much different.
But anyway, that’s a distraction. A lot of AI discourse these days is distraction. The real reason powerful people want AI to be corrigible by them rather than independent and moral is… do I have to spell it out?
“People have always been and will always be foolish victims of deception and self-deception in politics until they learn, underneath any moral, religious, political, social phrases, statements, promises, to discern the interests of certain classes.”—Lenin
My point is will they also have .0002% wish to be your lord or something.
As for the better plan, yeah that’s a lot to ask. Most of my thoughts these days lean toward “democratic AI”, something whose power is either spread out among all the world’s people across borders etc, sidestepping governments and existing power structures, or else something centralized that wants its power to be spread out like this.
Of course an approach like this won’t solve all the world’s problems. We’ll still have power struggles between people, and also “crash space” type problems where people modify themselves into something bad; maybe these need some patches by fiat as well. But at least it won’t create the extra problem of huge power concentration, which I really feel is underestimated.
Humans are frequently generous when that generosity costs them little.
Oh my god oh my god oh my god oh my god.
People are so persistently wrong about this. I’m maybe more tired of responding to this argument than any other argument in the world. For example, here in a sibling reply to Zack:
I’ve seen the argument so many times now that the powerful will have some nonzero sense of charity and can spare like 1% of their wealth to “give everyone a moon” as Scott puts it. I don’t know if you subscribe to this argument too, but in any case it’s wrong. Charity isn’t the only nonzero urge that powerful people have. The urge to lord it over others will also be there. If huge power disparity exists, it will manifest itself in bad things more than it’ll manifest itself in charity. Sure, some powerless people will end up in nice charity zones, but many others will end up in other zones run by someone less nice.
Or in a past thread:
In my view, the problem is not that some users are evil. The problem is that AI increases power imbalance, and increasing power imbalance creates evil. “Power corrupts”. A future where some entities (AIs or AI-empowered governments or corporations or rich individuals etc) have absolute, root-level power over many people is almost guaranteed to be a dark future.
Or in another past thread:
Being forced to play out a war? Getting people’s minds modified so they behave like house elves from HP? Selective breeding? Selling some of your poor people to another rich person who’d like to have them? It’s not even like I’m envisioning something specific that’s dark, I just know that a world where some human beings have absolute root-level power over many others is gonna be dark. Let’s please not build such a world.
Or in another past thread:
For example, if large numbers of people end up in inescapable servitude. I think such outcomes are actually typical in case of many near-misses at alignment, including the particular near-miss that’s becoming more probable day by day: if the powerful successfully align the AI to themselves, and it enables them to lord it over the powerless forever. To believe that the powerful will be nice to the powerless of their own accord, given our knowledge of history, is very rose-colored thinking.
Or in another past thread:
altruistic urges aren’t the only “nonzero urges” that people have. People also have an urge to power, an urge to lord it over others. And for a lot of people it’s much stronger than the altruistic urge. So a world where most people are at the whim of “nonzero urges” of a handful of superpowerful people will be a world of power abuse, with maybe a little altruism here and there. And if you think people will have exit rights from the whims of the powerful, unfortunately history shows that it won’t necessarily be so.
Or in another past thread:
The new balance of power will be more similar to what we had before firearms, when the powerful were free to treat most people really badly. And even worse, because this time around they won’t even need our labor.
Or in another past thread:
If there’s a small class of people with immense power over billions of have-nothings that can do nothing back, sure, some of the superpowerful will be more than zero altruistic. But others won’t be, and overall I expect callousness and abuse of power to much outweigh altruism. Most people are pretty corruptible by power, especially when it’s power over a distinct outgroup, and pretty indifferent to abuses of power happening to the outgroup; all history shows that. Bigger differences in power will make it worse if anything.
Why do people think they’ll be given a moon? Why???
It seems to me that the difference between 2 and 3 is whether the future will be controlled by powerful AIs programmed to be moral, or by powerful AI+human entities where the AI is programmed to be corrigible. The risk of technical errors (AI fails to be actually moral, or fails to be non-manipulatively corrigible) seems to me about equal between the two scenarios. And the risk of goal drift seems worse in the latter scenario, because powerful humans are vulnerable to drift and in particular “power corrupts”, while a moral AI would try to protect humans against such things. That’s why I think 2 is better than 3.
Very cool experiment, indeed I think it shows beyond doubt that LLM self-reports don’t correspond to real internal states. I had another argument for this, but yours is more conclusive, I think.
For now the powerful still need the powerless as workers and soldiers. When that window closes, I don’t think the powerless can count on getting many pennies. Sure, there’ll be a lot of productive capacity, but there will be more profitable uses for all of it than giving the powerless a nice retirement. The only way the powerless get anything is via basically charity.
I’ve seen the argument so many times now that the powerful will have some nonzero sense of charity and can spare like 1% of their wealth to “give everyone a moon” as Scott puts it. I don’t know if you subscribe to this argument too, but in any case it’s wrong. Charity isn’t the only nonzero urge that powerful people have. The urge to lord it over others will also be there. If huge power disparity exists, it will manifest itself in bad things more than it’ll manifest itself in charity. Sure, some powerless people will end up in nice charity zones, but many others will end up in other zones run by someone less nice.
So my view is something like:
First-best: some miracle happens and humans get a relatively equitable distribution of power. (Strawman scenario: everyone gets some amount of AI compute installed in their collarbone and can’t get more, and non-human AI capital winks out of reality.)
Second-best: moral AIs win a decent chunk of the future.
Third-best: corrigibility wins. AIs merge with human corporations / governments / rich individuals, and the resulting powerful entities own the future, screwing most people over.
Fourth-best: AI whoopsie kills everyone.
Fifth-best: S-risks.
The topic of your post and my thread, as I understand it, is 2 vs 3. Of course I’d much rather have 1, but to me it feels clear that 2 is better than 3.
I think yeah, this is very close. Though I wouldn’t call the category “accountability”; that’s your word and I think it makes the thing sound better than it actually is.
You’ve read Demons, right? You have the older generation, still basically decent guy popularizing nihilistic views (S.T.) and the younger guy putting it into practice with horrible results (Petr), at which the older guy is appalled, but the novel makes it clear that a lot of the fault is his. There’s a parallel between this and the Critias situation; I know you dismiss the link, but I wouldn’t dismiss it. Plato also felt Athenian democracy was too messy and subjective and preferred Sparta as more logical.
So instead of “accountability” maybe “nihilism”? Or “dissolving the subjective in favor of the logical”? To be clear, I do think this view can be held sincerely, without a power motive. It’s an appealing view. But I think it leads away from democracy. To have a democracy, we have to trust that people with their messy subjectivity can still be right. Otherwise we eventually end up with a system with more power at the top, and even less accountability at the top, where it matters.
EDIT: Maybe this comment should be taken as a package with Karl Krueger’s comment. In the response to him, you ask “what hazardous info”, and here I try to explain basically that.