I dunno, the problem with “alignment” is that it doesn’t unambiguously refer to the urgent problem, but “notkilleveryoneism” does. Alignment used to mean same-values, but then got both relaxed into compatible-values (that boundary-respecting norms allow to notkilleveryone) and strengthened with various AI safety features like corrigibility and soft optimization. Then there is prosaic alignment, which redefines it into bad-word-censure and reliable compliance with requests, neither being about values. Also, “existential catastrophe” inconveniently includes disempowerment that doesn’t killeveryone. And people keep bringing up (as an AI safety concern) merely large lethal disasters that don’t literally killeveryone, which is importantly different because second chances.
So on one hand it sounds silly, but on the other hand it’s harder to redefine away from the main concern. As a compromise between these, I’m currently experimenting with use of the term “killeveryone” as replacement for “existential catastrophe in the sense of extinction rather than disempowerment”. It has less syllables, a verb rather than a noun, might be slightly less silly, but retains the reference to the core concern.
It sounds non-silly to discuss “a balance between AI capabilities and alignment”. But try “a balance between restriction of AI capabilities and killing everyone”. It’s useful to make it noticeable that the usual non-silly framing is hiding an underlying omnicidal silliness, something people wouldn’t endorse as readily if it was more apparent.
I dunno, the problem with “alignment” is that it doesn’t unambiguously refer to the urgent problem, but “notkilleveryoneism” does. Alignment used to mean same-values, but then got both relaxed into compatible-values (that boundary-respecting norms allow to notkilleveryone) and strengthened with various AI safety features like corrigibility and soft optimization. Then there is prosaic alignment, which redefines it into bad-word-censure and reliable compliance with requests, neither being about values. Also, “existential catastrophe” inconveniently includes disempowerment that doesn’t killeveryone. And people keep bringing up (as an AI safety concern) merely large lethal disasters that don’t literally killeveryone, which is importantly different because second chances.
So on one hand it sounds silly, but on the other hand it’s harder to redefine away from the main concern. As a compromise between these, I’m currently experimenting with use of the term “killeveryone” as replacement for “existential catastrophe in the sense of extinction rather than disempowerment”. It has less syllables, a verb rather than a noun, might be slightly less silly, but retains the reference to the core concern.
It sounds non-silly to discuss “a balance between AI capabilities and alignment”. But try “a balance between restriction of AI capabilities and killing everyone”. It’s useful to make it noticeable that the usual non-silly framing is hiding an underlying omnicidal silliness, something people wouldn’t endorse as readily if it was more apparent.