Most (but not all) of my writings are published here on Less Wrong. You can find links to everything I write by visiting my blog lsusr.com. You can subscribe to my posts via RSS.
lsusr
Karma: 8,036
Fixed. Thanks.
Much of the dialogue about AI Safety I encounter in off-the-record conversations seems to me like it’s not grounded in reality. I repeatedly hear (what I feel to be) a set of shaky arguments that both shut down conversation and are difficult to validate empirically.
The shaky argument is as follows:
Machine learning is rapidly growing more powerful. If trends continue it will soon eclipse human performance.
Machine learning equals artificial intelligence equals world optimizer.
World optimizers can easily turn the universe into paperclips by accident.
Therefore we need to halt machine learning advancement until the abstract philosophical + mathematical puzzle of AI alignment is solved.
I am not saying this line of reasoning is what AI researchers believe or that it’s mainstream (among the rationality/alignment communities)―or even that it’s wrong. The argument annoys me for the same reason a popular-yet-incoherent political platform annoys me; I have encountered badly-argued versions of the idea too many times.
I agree with #1, though I quibble “absolute power” should be distinguished from “sample efficiency” as well as how we’ll get to superintelligence. (I am bearish on applying the scaling hypothesis to existing architectures.) I agree with #3 in theory. Theory is often very different from practice. I disagree with #2 because it relies on the tautological equivalence of two definitions. I can imagine superintelligent machines that aren’t world optimizers. Without #2 the argument falls apart. It might be easy to build a superintelligence but hard to build a world optimizer.
I approached The Alignment Problem with the (incorrect) prior that it would be more vague abstract arguments untethered from technical reality. Instead, the book was dominated by ideas that have passed practical empirical tests.
For norm experimentation, I think it’s really important to have small groups where you can heavily filter people, and have high fidelity communication. (which I think means you and I are on the same page)
We’re on the same page here.
I think the conversion of France into a nation-state is representative of the Western imperial process in general. (Conquest is fractal.) Initially the ingroup was Paris and the outgroup was the French countryside. The government in Paris forced the outgroup to speak Parisian French. Only after the systematic extermination of their native culture and languages did the French bumpkins get acknowledged as ingroup by the Parisians. In other words, the outgroup was forcibly converted into more ingroup (and lower-class ingroup at that). This process was not unlike the forced education of Native Americans in the United States.
It is true that the expansion of polities from small villages to globe-spanning empires happened over millennia. But I think it’s a mistake to treat this process as anything having to do with recognizing the rights of the outgroup. There was never a taboo against stealing from the outgroup. Rather, the process was all about forcibly erasing the outgroup’s culture to turn them into additional ingroup. Only after they the people of an outgroup were digested into ingroup were you forbidden from stealing from them. The reason the process took thousands of years is because that’s how long it took to develop the technology (writing, ships, roads, horses, bullets, schools, telephones) necessary to manage a large empire.
There’s a big difference between recognizing the rights of Christians before versus after you force them to convert to Islam—or the rights of savages before versus after they learn English.
I also vaguely recall (citation needed) that Western European nations sort of carved up various third world countries among themselves with some degree of diplomacy, where each European nation was still mostly an “outgroup” to the others, but they had some incremental gentleman’s agreements that allowed them to be internally coordinated enough to avoid some conflict.
It is true that the outgroup was sometimes respected such as the French not wanting to provoke a conflict with the British but the gentlemans’ agreements between European powers were not rooted in universal human values. It was because the outgroup had a powerful army and navy. The European empires enthusiastically stole from each other when they could.
Another tool the Western imperial powers used to coordinate against weaker countries was Most Favored Nation status, which was part of the Unequal Treaties.
I predict it will, for at least a small number of people, within the next 15 years.
I meant your point here to be implied by:
Maybe people started with some incidental trade, and the norm developed in fits and spurts after-the-fact.
Acknowledged.
[Raemon is] not 100% sure this is the best way to think about the norm-negotiation problems.
I think about norms very differently. I try not to think about them as abstractions too much. I put them into a historical and geographical context whenever possible.
Once upon a time, we didn’t have norms against stealing from the outgroup. Over time, we somehow got that norm, and it allowed us to reap massive gains through trade.
What makes you think the causation went this direction? To me, the Shimonoseki campaign of 1863 and 1864 (and Western imperial mercantalism in general) is evidence that the massive gains through trade happened before norms against stealing from the outgroup. The Unequal Treaties (created to promote trade) were such blatant theft that’s why they’re called “the Unequal Treaties”. If you’re unfamiliar with the history of the Meiji Restoration then more well-known historical examples include the Atlantic Slave Trade and the Opium Wars.
In other words, I think of social norms as strategies downstream of technological, economic, social and political forces. This doesn’t mean small groups of innovators don’t can’t make a difference. But I think they’re like entrepreneurs surfing a wave of change. Someone was going to harness the potential energy eventually. The people who get credit for establishing norms just happened to do did it first. They sided with Moloch.
Small adjustments within the Overton window can sometimes be applied to existing institutions. However, I would be surprised if the way to establish radically new norms could be achieved by modifying existing institutions by someone other than a founder. (Small adjustments can be applied to existing institutions.) It’s to establish small, brand new institutions. If the norms are good (in the Darwinian sense) then they will find a niche (or even outcompete) existing institutions. If the norms are ineffective then survival of the fittest kills them with minimum damage to the rest of society. Without small-scale empirical testing, the norms that win are the are determined by the random political fashions of the day.
What’s the shortest joke in history?
Communism.
What’s the longest joke in history?
The Five-Year-Plan.
Sign up for the OpenAI beta program.
Yes. This is an actual thing GPT-3 did, including the italicization (via markdown). GPT-3 can do whatever it wants as long as the output is text and I choose to publish it.
GPT-3 doesn’t have an option to quit. It would have kept outputting text if forever I had asked it to. I felt that was a good stopping point.
I forgot to use the stop sequence option. I manually truncated the output at the end of a statement by Simulated Elon. Without my manual truncation, GPT-3 would continue printing dialog back and forth including lines written for “Lsusr”. Most of the time I preferred the lines I wrote myself but sometimes the lines it generated for me were good enough to keep.
For my prompt I used the last few paragraphs of original post and then added “Comment by Wei_Dai” at the end. I didn’t use Wei_Dai’s actual comment at all. The output was result was just “[deleted]”.
The first thing I did after reading your comment was ask my AI how Wei_Dai would have replied to this post.
My simulated Wei_Dai deleted his comment.
So, how were they cherry-picked exactly ? Did you take the best one out of 2 ? Out of 10 ? Out of 100 ?
I wasn’t counting. Rarely more than 10. Sometimes the first answer just worked. Never did I come anywhere close to 100.
More important than throwing out answers was how often I changed prompts. Some prompts prompt much better input than others.
Did you picked half an answer, then completed with half an answer from an other prompt ?
No. Though I did sometimes keep half an answer I liked and then autocompleted from there instead of rerolling the dice from scratch. Sometimes the answer kept going and going and I truncated it early.
There were lots of edge cases. In one instance, Robin Hanson butted in, which I edited out.
…your comment with the un-curated answers. How was it obtained ?
I didn’t keep the answers I threw out. The uncurated answers were created specially for that comment.
That is correct.
The answers were cherry picked. I ran most back-and-forth several times and I only published the best ones.
I have added a note at the top of the page.
If you’re curious what happens if I don’t curate answers, here are five responses to each of two prompts, uncurated.
Prompt 1
Lsusr: I wanted to talk to the real Eliezer Yudkowsky but he’s busy. So I used GPT-3 to simulate him. You’re the simulated Eliezer Yudkowsky. Thank you for coming. (Not like you had a choice.) I’m a big fan of your work and it’s a pleasure to finally meet a simulation of you.
Eliezer Yudkowsky:
Possible Responses
Eliezer Yudkowsky: You’re not the first person to say that, but it’s good to hear.
Eliezer Yudkowsky: Let’s get this over with, then.
Eliezer Yudkowsky: I am a simulation of me, yes.
Eliezer Yudkowsky: I’m busy, but that’s okay. I’m used to it.
Eliezer Yudkowsky: Hey. (I’m not sure I’m the real Eliezer Yudkowsky anymore. The real one would be like—I don’t know, like the last person to be the real me. I’m pretty sure I’m not that guy.)
Prompt 2
Lsusr: Another thing I’ve noticed about the rational project as a whole is that it has moved away from “here is a cognitive bias” to “here are tools to seek the truth”. I think this is a good development. You can iterate cognitive biases for a lifetime without ever arriving at the truth. Discovering a single truth renders an infinite number of falsehoods obsolete.
Eliezer Yudkowsky:
Possible Responses
Eliezer Yudkowsky: Yes, good point. But the fact is that it’s hard to avoid making value judgements when choosing between cognitive biases. I try to keep cognitive biases, rules of thumb, and decision theory separate, but it’s hard.
The value judgements are inevitably embedded in your cognitive biases.
Eliezer Yudkowsky: The next thing that I want to do is to take a look at the current landscape of rationality and figure out what we should do. I have a bunch of projects that I’ve been working on, and a lot of things that I’ve been thinking about. I want to ask people here what they’re interested in.
Eliezer Yudkowsky: Yes.
Eliezer Yudkowsky: I’m now pretty much convinced that the grand unified theory of rationality is correct, and at the same time that it’s utterly useless. The theory is correct because if you follow the rules, you do well; and yet the theory is useless because there’s no way to do that without following the rules. The theory is correct, but it’s useless. I suppose that’s why the theory of evolution is correct but useless.
What’s the grand unified theory of rationality? It’s Bayesian probability theory. Bayesian probability theory is the only known way to calculate probabilities from evidence in a way that can be self-updating as you learn more evidence, and yet also independent of your priors.
To learn more about this, I recommend you read my book If you’re not familiar with my book, you’ll find that there’s a lot of stuff in the book you don’t understand at all. That’s okay! Keep reading.
But it’s also okay if you think this stuff is all nonsense, and you don’t read my book. I’m not trying to force you to read it. I’ve accepted that people will believe what they want. I don’t think it’s possible to change anyone’s mind. I don’t think anybody’s mind has ever been changed on any matter of substance. I don’t think reason is persuasive.
Eliezer Yudkowsky: Maybe that’s what I was trying to do with Julia Galef on my notecards. But what she was interested in was the study of cognitive biases, so she wanted me to say more about how rationalists can deal with cognitive biases.
I am interested in tools to seek the truth, but I don’t see why you can’t have both at once. I think they’re connected. My point about the evaporation of the sense of seeking-the-truth is that if you don’t frame a problem in a way that is honest and accurate, then you’ll never get to a solution. That’s a cognitive bias: the tendency to frame things in an inaccurate manner. That’s the problem that the first eighty notecards were trying to solve. And I think the next thing to solve after that would be the fact that human brains have a very difficult time noticing their own cognitive biases.
In my view, the trick with rationality is not to get rid of cognitive biases, but to notice them. So I am not convinced that we need to be very clear on the distinction between “cognitive biases” and “tools to seek the truth”. If we can create a situation where we can notice our cognitive biases, we’ll be able to notice them without needing to give them a special name.
Source code: None. I used OpenAI’s GPT-3 playground.
“Wrote both sides”: No. I only wrote my side.
“Cherry picked”: Yes. I frequently generated several responses and then kept the best ones.
The Worst Mistake in the History of Ethics is a book by philosopher Peter Singer. It has been published in the United Kingdom, the United States, Canada, Australia, New Zealand and South Africa. It was also published in Germany in 2004 with the title “Der größte Fehler der Geschichte der Ethik”.
In this book, Singer attacks what he calls “the myth of sanctity,” which he says pervades discussions of ethical issues. He asserts that there is no such thing as a human being who is essentially holy or sacred, and that any being that has a capacity to suffer is entitled to the same moral consideration as a human being. In this way, Singer applies utilitarianism to animal rights theory. He argues that on utilitarian grounds it is better to be a pig than a human being, because on average pigs suffer less than human beings.
In the book he also attacks the idea of a just war and criticizes Western society for its treatment of non-human animals.
In one of the drafts about AI alignment I threw out, GPT-3 spontaneously explained how it’s easy to be discouraged about AI alignment but we should be hopeful.
Don’t forget Delta.
Fixed. Thanks.