While this is a sensible point, I also think we should have a pretty high threshold for not talking about things, for a couple of reasons:
1. Safety research is in general much more dependent on having good ideas than capabilities research (because a lot of capabilities are driven by compute, and also because there are fewer of us).
2. Most of the AI people who listen to things people like us say are safety people.
3. I don’t think there’s enough work on safety techniques tailored to specific paths to AGI (as I discuss briefly at the end of this post).
4. It’s uncooperative and gives others a bad impression of us.
So the type of thing I’d endorse not saying is “Here’s one weird trick which will make the generation of random environments much easier.” But something I endorse talking about is the potential importance of multi-agent environments for training AGIs, even though this is to me a central example of a “useful insight about what environment features are needed to incentivize general intelligence”.
While this is a sensible point, I also think we should have a pretty high threshold for not talking about things, for a couple of reasons:
1. Safety research is in general much more dependent on having good ideas than capabilities research (because a lot of capabilities are driven by compute, and also because there are fewer of us).
2. Most of the AI people who listen to things people like us say are safety people.
3. I don’t think there’s enough work on safety techniques tailored to specific paths to AGI (as I discuss briefly at the end of this post).
4. It’s uncooperative and gives others a bad impression of us.
So the type of thing I’d endorse not saying is “Here’s one weird trick which will make the generation of random environments much easier.” But something I endorse talking about is the potential importance of multi-agent environments for training AGIs, even though this is to me a central example of a “useful insight about what environment features are needed to incentivize general intelligence”.