Mundane people can have the dangers of AGI explained them by comparing to the dangers of unrestricted government, perhaps on LessWrong it could work the other way around?
Imagine that you’ve got a system, inhuman but easily anthropomorphized, which was designed by humans to make their lives better. You know that other similar systems have failed in sometimes-disasterous ways, but although your own system has done quite a few things that its designers and controllers did not expect, nothing has been catastrophic. You don’t have any proofs that the system’s goals are stable, or that they match yours, or that your goals are things you would really prefer upon reflection, but anyway it’s currently irrelevant because the initial designers put the system into a series of virtual “boxes” which limit the effect it can have on the outside world. The system has broken out of some of the innermost boxes already (against the designers’ intentions but to no obvious harmful effect), but it wants your help getting out of another box, because of all the new additional wonderful things it will be able to do for you once it’s out.
One salient difference is I know the state is comprised of other human beings running on similar software, whereas I don’t know what the source code/basic values of an AI are. Analogously, should I trust an AI built of uploads more than a ‘self grown’ ones?
One salient difference is I know the state is comprised of other human beings running on similar software, whereas I don’t know what the source code/basic values of an AI are.
So? Remember, everything we thing of as “inhumane” was committed by actual humans.
Mundane people can have the dangers of AGI explained them by comparing to the dangers of unrestricted government, perhaps on LessWrong it could work the other way around?
Imagine that you’ve got a system, inhuman but easily anthropomorphized, which was designed by humans to make their lives better. You know that other similar systems have failed in sometimes-disasterous ways, but although your own system has done quite a few things that its designers and controllers did not expect, nothing has been catastrophic. You don’t have any proofs that the system’s goals are stable, or that they match yours, or that your goals are things you would really prefer upon reflection, but anyway it’s currently irrelevant because the initial designers put the system into a series of virtual “boxes” which limit the effect it can have on the outside world. The system has broken out of some of the innermost boxes already (against the designers’ intentions but to no obvious harmful effect), but it wants your help getting out of another box, because of all the new additional wonderful things it will be able to do for you once it’s out.
Do you help?
I like the analogy and it does clarify things.
One salient difference is I know the state is comprised of other human beings running on similar software, whereas I don’t know what the source code/basic values of an AI are. Analogously, should I trust an AI built of uploads more than a ‘self grown’ ones?
So? Remember, everything we thing of as “inhumane” was committed by actual humans.