My recommendation was very concretely “try to build an internal model of what really needs to happen for AI-risk to go well”
I’m not sure anyone knows what really needs to happen for AI-risk to go well, including people who have been thinking about this question for many years. Do you really mean for RAISE to solve this problem, or just to think about this question for more than they already have, or to try to learn the best available model from someone else (if so who)?
Mostly think more about this question than they already have, which likely includes learning the best available models from others.
The critique here was more one of intention than one of epistemic state. It seems to me like there is a mental motion of being curious about how to make progress on something, even if one is still confused, which I contrast with a mental motion of “trying to look like you are working on the problem”.
Ah ok. Given that, it seems like you need to explain your critique more, or try to figure out the root cause of the wrong intention and address that, otherwise wouldn’t they just switch to “trying to look like you’re trying to build models of what needs to be done to solve AI risk”?
Another problem is that it seems even harder to distinguish between people who are really trying to build such models, and people who are just trying to look like they’re doing that, because there’s no short-term feedback from reality to tell you whether someone’s model is any good. It seems like suggesting people to do that when you’re not sure of their intention is really dangerous, as it could mess up the epistemic situation with AI risk models (even more than it already is). Maybe it would be better to just suggest some concrete short-term projects for them to do instead?
I’m not sure anyone knows what really needs to happen for AI-risk to go well, including people who have been thinking about this question for many years. Do you really mean for RAISE to solve this problem, or just to think about this question for more than they already have, or to try to learn the best available model from someone else (if so who)?
Mostly think more about this question than they already have, which likely includes learning the best available models from others.
The critique here was more one of intention than one of epistemic state. It seems to me like there is a mental motion of being curious about how to make progress on something, even if one is still confused, which I contrast with a mental motion of “trying to look like you are working on the problem”.
Ah ok. Given that, it seems like you need to explain your critique more, or try to figure out the root cause of the wrong intention and address that, otherwise wouldn’t they just switch to “trying to look like you’re trying to build models of what needs to be done to solve AI risk”?
Another problem is that it seems even harder to distinguish between people who are really trying to build such models, and people who are just trying to look like they’re doing that, because there’s no short-term feedback from reality to tell you whether someone’s model is any good. It seems like suggesting people to do that when you’re not sure of their intention is really dangerous, as it could mess up the epistemic situation with AI risk models (even more than it already is). Maybe it would be better to just suggest some concrete short-term projects for them to do instead?