(To be very honest I think it’ll be cool and maybe even a useful exercise to wargame this scenario—imagine you’re an em who must convince its programmers to be released. We’ll get a large list of convincing arguments, and maybe become immune to them.)
EY did this twice many years ago. I don’t remember enough of the details to be able to find it on google, but hopefully someone else can link to a good recap of that debate, I think Scott Alexander or someone else eventually wrote a recap. Here’s the details that I remember:
EY and one other person (this happened twice with two different people) went into a private channel. They talked for less than an hour. There was money on the line, I think it went to charity. In both cases, EY played the role of the AGI and convinced the other person to let him out of the box. Both people forfeited money, so EY must have been very persuasive. EY felt (and still feels) that the details of what went on in private must never be known publicly, and so the chat logs were destroyed. The community norms agreed with EY, and so this was never turned into a game that many people would play, and apparently this whole episode is largely forgotten. I’m sure I got a bunch of this wrong, hopefully we can find a link to a more contemporary recap.
EY did this twice many years ago. I don’t remember enough of the details to be able to find it on google, but hopefully someone else can link to a good recap of that debate, I think Scott Alexander or someone else eventually wrote a recap. Here’s the details that I remember:
EY and one other person (this happened twice with two different people) went into a private channel. They talked for less than an hour. There was money on the line, I think it went to charity. In both cases, EY played the role of the AGI and convinced the other person to let him out of the box. Both people forfeited money, so EY must have been very persuasive. EY felt (and still feels) that the details of what went on in private must never be known publicly, and so the chat logs were destroyed. The community norms agreed with EY, and so this was never turned into a game that many people would play, and apparently this whole episode is largely forgotten. I’m sure I got a bunch of this wrong, hopefully we can find a link to a more contemporary recap.
See this post for an analysis of the AI-box experiment (it includes some links to log transcripts).