This suggestion is too much defensive writing for my taste. Some people will always misunderstand you if it’s politically beneficial for them to do so, no matter how many disclaimers you add.
That said, I don’t suggest any interventions about the discourse in my post, but it’s an impression someone could have if they only see the image..? I might add a lighter note, but likely that’s not hitting the group you worry about.
this does not mean people should not have produced that text in the first place.
That’s an empirical question. Normal sociohazard rules apply. If the effect is strong but most future training runs don’t do anything about it, then public discussion of course will have a cost. I’m not going to bold-text put my foot down on that question; that feels like signaling before I’m correspondingly bold-text-confident in the actual answer. Though yes, I would guess that AI risk worth talking about.[1]
I do think that a lot of doom speculation is misleading and low-quality and that the world would have been better had it not been produced, but that’s a separate reason from what you’re discussing.
> [!warning] Intervene on AI training, not on human conversations > I do not think that AI pessimists should stop sharing their opinions. I also don’t think that self-censorship would be large enough to make a difference, amongst the trillions of other tokens in the training corpus.
yay, thanks! It means a lot to me because I expect some people to use your ideas as a sort of cheap rhetorical cudgel “Oh those silly doomers, speculating about AIs being evil. You know what the real problem is? Their silly speculations!”
I would argue that we do have a responsibility to prevent this data on misaligned AIs being scraped by LLM scrapers as much as possible. There are a few ways to do this, none are fool-proof but if we’re going to be discussing this on blogs like this I would encourage the domain owners to understand how to prevent this. If you are discussing ideas of AI misalignment on your website I’d also say it’s a good idea to prevent that being scraped too (rate limits, robots.txt, etc)
It makes sense that you don’t want this article to opine on the question of whether people should not have created “misalignment data”, but I’m glad you concluded that it wasn’t a mistake in the comments. I find it hard to even tell a story where this genre of writing was a mistake. Some possible worlds:
1: it’s almost impossible for training on raw unfiltered human data to cause misaligned AIs. In this case there was negligible risk from polluting the data by talking about misaligned AIs, it was just a waste of time.
2: training on raw unfiltered human data can cause misaligned AIs. Since there is a risk of misaligned AIs, it is important to know that there’s a risk, and therefore to not train on raw unfiltered human data. We can’t do that without talking about misaligned AIs. So there’s a benefit from talking about misaligned AIs.
3: training on raw unfiltered human data is very safe, except that training on any misalignment data is very unsafe. The safest thing is to train on raw unfiltered human data that naturally contains no misalignment data.
Only world 3 implies that people should not have produced the text in the first place. And even there, once “2001: A Space Odyssey” (for example) is published the option to have no misalignment data in the corpus is blocked, and we’re in world 2.
This suggestion is too much defensive writing for my taste. Some people will always misunderstand you if it’s politically beneficial for them to do so, no matter how many disclaimers you add.
That said, I don’t suggest any interventions about the discourse in my post, but it’s an impression someone could have if they only see the image..? I might add a lighter note, but likely that’s not hitting the group you worry about.
That’s an empirical question. Normal sociohazard rules apply. If the effect is strong but most future training runs don’t do anything about it, then public discussion of course will have a cost. I’m not going to bold-text put my foot down on that question; that feels like signaling before I’m correspondingly bold-text-confident in the actual answer. Though yes, I would guess that AI risk worth talking about.[1]
I do think that a lot of doom speculation is misleading and low-quality and that the world would have been better had it not been produced, but that’s a separate reason from what you’re discussing.
I’m adding the following disclaimer:
> [!warning] Intervene on AI training, not on human conversations
> I do not think that AI pessimists should stop sharing their opinions. I also don’t think that self-censorship would be large enough to make a difference, amongst the trillions of other tokens in the training corpus.
yay, thanks! It means a lot to me because I expect some people to use your ideas as a sort of cheap rhetorical cudgel “Oh those silly doomers, speculating about AIs being evil. You know what the real problem is? Their silly speculations!”
I would argue that we do have a responsibility to prevent this data on misaligned AIs being scraped by LLM scrapers as much as possible. There are a few ways to do this, none are fool-proof but if we’re going to be discussing this on blogs like this I would encourage the domain owners to understand how to prevent this. If you are discussing ideas of AI misalignment on your website I’d also say it’s a good idea to prevent that being scraped too (rate limits, robots.txt, etc)
It makes sense that you don’t want this article to opine on the question of whether people should not have created “misalignment data”, but I’m glad you concluded that it wasn’t a mistake in the comments. I find it hard to even tell a story where this genre of writing was a mistake. Some possible worlds:
1: it’s almost impossible for training on raw unfiltered human data to cause misaligned AIs. In this case there was negligible risk from polluting the data by talking about misaligned AIs, it was just a waste of time.
2: training on raw unfiltered human data can cause misaligned AIs. Since there is a risk of misaligned AIs, it is important to know that there’s a risk, and therefore to not train on raw unfiltered human data. We can’t do that without talking about misaligned AIs. So there’s a benefit from talking about misaligned AIs.
3: training on raw unfiltered human data is very safe, except that training on any misalignment data is very unsafe. The safest thing is to train on raw unfiltered human data that naturally contains no misalignment data.
Only world 3 implies that people should not have produced the text in the first place. And even there, once “2001: A Space Odyssey” (for example) is published the option to have no misalignment data in the corpus is blocked, and we’re in world 2.