I’m worried about notkilleveryonism as a meme. Years ago, Tyler Cowen wrote a post about why more econ professors didn’t blog, and his conclusion was that it’s too easy to make yourself look like an idiot relative to the payoffs. And that he had observed this actually play out in a bunch of cases where econ professors started blogs, put their foot in their mouth, and quietly stopped. Since earnest discussion of notkilleveryonism tends to make everyone, including the high status, look dumb within ten minutes of starting, it seems like there will be a strong inclination towards attribute substitution. People will tend towards ‘nuanced’ takes that give them more opportunity to signal with less chance of looking stupid.
Worry about looking like an idiot is a VERY fine balance to find. If you get desensitized to it, that makes it too easy to BE an idiot. If you are over-concrerned about it, you fail to find correct contrarian takes.
‘notkilleveryoneism’ IMO is a dumb meme. Intentionally, I presume. If you wanted to appear smart, you’d use more words and accept some of the nuance, right? It feels like a countersignal-attempt, or a really bad model of someone who’s not accepting the normal arguments.
I dunno, the problem with “alignment” is that it doesn’t unambiguously refer to the urgent problem, but “notkilleveryoneism” does. Alignment used to mean same-values, but then got both relaxed into compatible-values (that boundary-respecting norms allow to notkilleveryone) and strengthened with various AI safety features like corrigibility and soft optimization. Then there is prosaic alignment, which redefines it into bad-word-censure and reliable compliance with requests, neither being about values. Also, “existential catastrophe” inconveniently includes disempowerment that doesn’t killeveryone. And people keep bringing up (as an AI safety concern) merely large lethal disasters that don’t literally killeveryone, which is importantly different because second chances.
So on one hand it sounds silly, but on the other hand it’s harder to redefine away from the main concern. As a compromise between these, I’m currently experimenting with use of the term “killeveryone” as replacement for “existential catastrophe in the sense of extinction rather than disempowerment”. It has less syllables, a verb rather than a noun, might be slightly less silly, but retains the reference to the core concern.
It sounds non-silly to discuss “a balance between AI capabilities and alignment”. But try “a balance between restriction of AI capabilities and killing everyone”. It’s useful to make it noticeable that the usual non-silly framing is hiding an underlying omnicidal silliness, something people wouldn’t endorse as readily if it was more apparent.
When you say you’re worried about “nonkilleveryoneism” as a meme, you mean that this meme (compared to other descriptions of “existential risk from AI is important to think about”) is usually likely to cause this foot-in-mouth-quietly-stop reaction, or that the nature of the foot-in-mouth-quietly-stop dynamic just makes it hard to talk about at all?
I mean that I think why AI ethics had to be split as a term with notkilleveryonism in the first place will simply happen again, rather than notkilleveryonism solving the problem.
I’m worried about notkilleveryonism as a meme. Years ago, Tyler Cowen wrote a post about why more econ professors didn’t blog, and his conclusion was that it’s too easy to make yourself look like an idiot relative to the payoffs. And that he had observed this actually play out in a bunch of cases where econ professors started blogs, put their foot in their mouth, and quietly stopped. Since earnest discussion of notkilleveryonism tends to make everyone, including the high status, look dumb within ten minutes of starting, it seems like there will be a strong inclination towards attribute substitution. People will tend towards ‘nuanced’ takes that give them more opportunity to signal with less chance of looking stupid.
Worry about looking like an idiot is a VERY fine balance to find. If you get desensitized to it, that makes it too easy to BE an idiot. If you are over-concrerned about it, you fail to find correct contrarian takes.
‘notkilleveryoneism’ IMO is a dumb meme. Intentionally, I presume. If you wanted to appear smart, you’d use more words and accept some of the nuance, right? It feels like a countersignal-attempt, or a really bad model of someone who’s not accepting the normal arguments.
I dunno, the problem with “alignment” is that it doesn’t unambiguously refer to the urgent problem, but “notkilleveryoneism” does. Alignment used to mean same-values, but then got both relaxed into compatible-values (that boundary-respecting norms allow to notkilleveryone) and strengthened with various AI safety features like corrigibility and soft optimization. Then there is prosaic alignment, which redefines it into bad-word-censure and reliable compliance with requests, neither being about values. Also, “existential catastrophe” inconveniently includes disempowerment that doesn’t killeveryone. And people keep bringing up (as an AI safety concern) merely large lethal disasters that don’t literally killeveryone, which is importantly different because second chances.
So on one hand it sounds silly, but on the other hand it’s harder to redefine away from the main concern. As a compromise between these, I’m currently experimenting with use of the term “killeveryone” as replacement for “existential catastrophe in the sense of extinction rather than disempowerment”. It has less syllables, a verb rather than a noun, might be slightly less silly, but retains the reference to the core concern.
It sounds non-silly to discuss “a balance between AI capabilities and alignment”. But try “a balance between restriction of AI capabilities and killing everyone”. It’s useful to make it noticeable that the usual non-silly framing is hiding an underlying omnicidal silliness, something people wouldn’t endorse as readily if it was more apparent.
When you say you’re worried about “nonkilleveryoneism” as a meme, you mean that this meme (compared to other descriptions of “existential risk from AI is important to think about”) is usually likely to cause this foot-in-mouth-quietly-stop reaction, or that the nature of the foot-in-mouth-quietly-stop dynamic just makes it hard to talk about at all?
I mean that I think why AI ethics had to be split as a term with notkilleveryonism in the first place will simply happen again, rather than notkilleveryonism solving the problem.
What do you think will actually happen with the term notkilleveryonism?
Attempts to deploy the meme to move the conversation in a more productive direction will stop working I guess.