Did you find something you were happy with for this?
Was it The Problem?
If it wasn’t The Problem, why not? (like ‘in what ways did you find it lacking for your purposes?’ because it’s pretty important to me that it be suitable/useful for these kinds of purposes and we’re updating it right now).
Thanks for the recommendation btw. I take it from your comment that you’re involved in writing this?
I ended up using a version of this from the Center for AI Safety that I edited down for length (basically taking the intro, the front end of each section, and then most of the “Rogue AIs” section).
My context here is assigning this to undergraduates in a survey course where AI is being discussed among other serious future threats. Some things I didn’t like about it for that purpose (that may or may not have anything to do with your own target audience):
The argumentation style is very heavy on quoting other people’s arguments. A reaction I want a student to have to that is something along the lines of, “Why am I reading this, rather than those underlying sources, then?”
The general use of quotations and sources aligns more with op-ed norms than academic norms. That is, isolated quotations from various people or reports are used without an attempt to explain their independent perspectives and the extent to which they do or don’t share your conclusions.
There’s also a lot of line blurring between quite distinct positions (especially as to quoting people who are advancing much weaker conclusions as evidence for your stronger ones). We would be compelled to have a long discussion about whether the sources are used in a responsible way and whether they really support the argument, and whether something like a remark made by an expert as a minor point in an interview should really be used as evidence, and so on. That’s just not what I’m looking to do.
There’s a real tension in the purpose of the essay. There’s an oscillation between the 90% risk of extinction idea and the “the risks are high enough we need an off switch” which are radically different things. A major problem in undergraduate writing is to make really over the top claims that are much bolder than necessary to prove the overall point. To the extent the point here is “we need an off switch,” this essay is very guilty of that and so represents a bad example in the war on claiming more than necessary when building arguments.
I think the Stockfish argument is really unhelpful to you. I can understand what you’re trying to say there after thinking about it, but the immediate intuition when you bring this up is the opposite. That is, Stockfish seems to be a well-designed system that was given a clear goal by its designers, pursues that goal as intended, and causes no trouble for anyone. It’s not winning at chess by cheating or blackmailing you or anything like that. If you want to shut it down, you just do that. And statements like the one that Stockfish doesn’t capture your pieces because it hates them are just weird because they’re arguing against nothing. The point that it will “wipe the floor with you” also, I think, triggers nothing for someone who is young and not especially into chess and who probably thinks of computers being really good at chess as a really ordinary and uninteresting state of affairs.
For my purposes in particular, I’d like to see a bit more on the geopolitics.
Following up to ask:
Did you find something you were happy with for this?
Was it The Problem?
If it wasn’t The Problem, why not? (like ‘in what ways did you find it lacking for your purposes?’ because it’s pretty important to me that it be suitable/useful for these kinds of purposes and we’re updating it right now).
Thanks for the recommendation btw. I take it from your comment that you’re involved in writing this?
I ended up using a version of this from the Center for AI Safety that I edited down for length (basically taking the intro, the front end of each section, and then most of the “Rogue AIs” section).
My context here is assigning this to undergraduates in a survey course where AI is being discussed among other serious future threats. Some things I didn’t like about it for that purpose (that may or may not have anything to do with your own target audience):
The argumentation style is very heavy on quoting other people’s arguments. A reaction I want a student to have to that is something along the lines of, “Why am I reading this, rather than those underlying sources, then?”
The general use of quotations and sources aligns more with op-ed norms than academic norms. That is, isolated quotations from various people or reports are used without an attempt to explain their independent perspectives and the extent to which they do or don’t share your conclusions.
There’s also a lot of line blurring between quite distinct positions (especially as to quoting people who are advancing much weaker conclusions as evidence for your stronger ones). We would be compelled to have a long discussion about whether the sources are used in a responsible way and whether they really support the argument, and whether something like a remark made by an expert as a minor point in an interview should really be used as evidence, and so on. That’s just not what I’m looking to do.
There’s a real tension in the purpose of the essay. There’s an oscillation between the 90% risk of extinction idea and the “the risks are high enough we need an off switch” which are radically different things. A major problem in undergraduate writing is to make really over the top claims that are much bolder than necessary to prove the overall point. To the extent the point here is “we need an off switch,” this essay is very guilty of that and so represents a bad example in the war on claiming more than necessary when building arguments.
I think the Stockfish argument is really unhelpful to you. I can understand what you’re trying to say there after thinking about it, but the immediate intuition when you bring this up is the opposite. That is, Stockfish seems to be a well-designed system that was given a clear goal by its designers, pursues that goal as intended, and causes no trouble for anyone. It’s not winning at chess by cheating or blackmailing you or anything like that. If you want to shut it down, you just do that. And statements like the one that Stockfish doesn’t capture your pieces because it hates them are just weird because they’re arguing against nothing. The point that it will “wipe the floor with you” also, I think, triggers nothing for someone who is young and not especially into chess and who probably thinks of computers being really good at chess as a really ordinary and uninteresting state of affairs.
For my purposes in particular, I’d like to see a bit more on the geopolitics.