Let me first clarify something. Are you asking because you want to understand MIRI’s specific model of AI risk; or do you just want a simple argument that AI risk is real, and it doesn’t matter who makes the argument?
You’re writing as if the reality of AI risk depends on whether or not there’s an up-to-date FAQ about it, on this website. But you know that Less Wrong does not have a monopoly on AI doom, right? Everyone from the founders of deep learning to the officials of the deep state are worried about AI now, because it has become so powerful. This issue is somewhere in the media every day now; and it’s just common sense, given the way of the world, that entities which are not human and smarter than human potentially pose a risk to the human race.
I want neither. I observe that Raemon cannot find an up to date introduction that he’s happy with, and I point out that this is really weird. What I want is an explanation to this bizarre situation.
Is your position that Raemon is blind, and good, convincing explanations are actually abundant? If so, I’d like to see them, it doesn’t matter where from.
Expositions of AI risk are certainly abundant. there have been numerous books and papers. Or just go to Youtube and type in “AI risk”. As for whether any given exposition is convincing, I am no connoisseur. For a long time, I have taken it for granted that AI can be both smarter than humans and dangerous to humans. I’m more interested in details, like risk taxonomies and alignment theories.
But whether a given exposition is convincing, depends on the audience as well as on the author. Some people have highly specific objections. In our discussion, you questioned whether adversarial relations between AI and human are likely to occur, and with Raemon you bring up the topic of agency, so maybe you specifically need an argument that AIs would ever end up acting against human interests?
As for Raemon, I suspect he would like a superintelligence FAQ that that acknowledges the way things are in 2023 - e.g. the rise of a particular AI paradigm to dominate discussion (deep learning and large language models), and the existence of a public debate about AI safety, all the way up to the UN Security Council.
I don’t know if you know, but after being focused for 20 years on rather theoretical issues of AI, MIRI has just announced it will be changing focus to “broad public communication”. If you look back at their website, in the 2000s their introductory materials were mostly aimed at arguing that smarter-than-human AI is possible and important. Then in the 2010s (which is the era of Less Wrong), the MIRI homepage was more about their technical papers and workshops and so on, and didn’t try to be accessible to a general audience. Now in the mid-2020s, they really will be aiming at a broader audience.
OK, there are many people writing explanations, but if all of them are rehashing the same points from Superintelligence book, then there is not much value in that (and I’m tired of reading the same things over and over). Of course you don’t need new arguments or new evidence, but it’s still strange if there aren’t any.
Anyone who has read this FAQ and others, but isn’t a believer yet, will have some specific objections. But I don’t think everyone’s objections are unique, a better FAQ should be able to cover them, if their refutations exist to begin with.
Also, are you yourself working on AI risk? If not, why not? Is this not the most important problem of our time? Would EY not say that you should work on it? Could it be that you and him actually have wildly different estimates of P(AI doom), despite agreeing on the arguments?
As for Raemon, you’re right, I probably misunderstood why he’s unhappy with newer explanations.
are you yourself working on AI risk? If not, why not?
etc.
I presume you have no idea how enraging these questions are, because you know less than nothing about my life.
I will leave it to you to decide whether this “Average Redditor” style of behavior (look it up, it’s a Youtube character) is something you should avoid in future.
If you actually do want to work on AI risk, but something is preventing you, you can just say “personal reasons”, I’m not going to ask for details.
I understand that my style is annoying to some. Unfortunately, I have not observed polite and friendly people getting interesting answers, so I’ll have to remain like that.
Your questions opened multiple wounds, but I’ll get over it.
I “work on” AI risk, in the sense that I think about it when I can. Under better circumstances, I suspect I could make important contributions. I have not yet found a path to better circumstances.
Let me first clarify something. Are you asking because you want to understand MIRI’s specific model of AI risk; or do you just want a simple argument that AI risk is real, and it doesn’t matter who makes the argument?
You’re writing as if the reality of AI risk depends on whether or not there’s an up-to-date FAQ about it, on this website. But you know that Less Wrong does not have a monopoly on AI doom, right? Everyone from the founders of deep learning to the officials of the deep state are worried about AI now, because it has become so powerful. This issue is somewhere in the media every day now; and it’s just common sense, given the way of the world, that entities which are not human and smarter than human potentially pose a risk to the human race.
I want neither. I observe that Raemon cannot find an up to date introduction that he’s happy with, and I point out that this is really weird. What I want is an explanation to this bizarre situation.
Is your position that Raemon is blind, and good, convincing explanations are actually abundant? If so, I’d like to see them, it doesn’t matter where from.
Expositions of AI risk are certainly abundant. there have been numerous books and papers. Or just go to Youtube and type in “AI risk”. As for whether any given exposition is convincing, I am no connoisseur. For a long time, I have taken it for granted that AI can be both smarter than humans and dangerous to humans. I’m more interested in details, like risk taxonomies and alignment theories.
But whether a given exposition is convincing, depends on the audience as well as on the author. Some people have highly specific objections. In our discussion, you questioned whether adversarial relations between AI and human are likely to occur, and with Raemon you bring up the topic of agency, so maybe you specifically need an argument that AIs would ever end up acting against human interests?
As for Raemon, I suspect he would like a superintelligence FAQ that that acknowledges the way things are in 2023 - e.g. the rise of a particular AI paradigm to dominate discussion (deep learning and large language models), and the existence of a public debate about AI safety, all the way up to the UN Security Council.
I don’t know if you know, but after being focused for 20 years on rather theoretical issues of AI, MIRI has just announced it will be changing focus to “broad public communication”. If you look back at their website, in the 2000s their introductory materials were mostly aimed at arguing that smarter-than-human AI is possible and important. Then in the 2010s (which is the era of Less Wrong), the MIRI homepage was more about their technical papers and workshops and so on, and didn’t try to be accessible to a general audience. Now in the mid-2020s, they really will be aiming at a broader audience.
OK, there are many people writing explanations, but if all of them are rehashing the same points from Superintelligence book, then there is not much value in that (and I’m tired of reading the same things over and over). Of course you don’t need new arguments or new evidence, but it’s still strange if there aren’t any.
Anyone who has read this FAQ and others, but isn’t a believer yet, will have some specific objections. But I don’t think everyone’s objections are unique, a better FAQ should be able to cover them, if their refutations exist to begin with.
Also, are you yourself working on AI risk? If not, why not? Is this not the most important problem of our time? Would EY not say that you should work on it? Could it be that you and him actually have wildly different estimates of P(AI doom), despite agreeing on the arguments?
As for Raemon, you’re right, I probably misunderstood why he’s unhappy with newer explanations.
etc.
I presume you have no idea how enraging these questions are, because you know less than nothing about my life.
I will leave it to you to decide whether this “Average Redditor” style of behavior (look it up, it’s a Youtube character) is something you should avoid in future.
If you actually do want to work on AI risk, but something is preventing you, you can just say “personal reasons”, I’m not going to ask for details.
I understand that my style is annoying to some. Unfortunately, I have not observed polite and friendly people getting interesting answers, so I’ll have to remain like that.
Your questions opened multiple wounds, but I’ll get over it.
I “work on” AI risk, in the sense that I think about it when I can. Under better circumstances, I suspect I could make important contributions. I have not yet found a path to better circumstances.