One concept which I think is strongly related is common knowledge (if we extend it to not only refer to beliefs but also to norms). I think that a good part of the difference between the scenarios for answering Alec is captured by the difference between sharing personal advice and knowledge compared to sharing common norms and knowledge. The latter will give Alec a lot of information about how to work together and cooperate with “the typical member of” the AI safety community, which is important information independently of whether it would be best for Alec’s thinking to adopt the beliefs or follow the advice. At least he will be able to navigate working together with others in the community even if he disagrees with their opinions.
Of course there still can be the failure mode of just aiming to fit in into the group and neglecting the group’s intended purpose. I think one important aspect here is that mentally treating a group’s norms, behaviours and beliefs as being simple and uniform has huge difficulty advantages for establishing common knowledge and makes it much simpler for an individual to navigate the group. “The Intelligent Social Web” nicely describes this and also the pull that people feel towards acting out well trodden social roles.
Of course, AI safety might well be one of the topics where we really want people not to just adopt the patterns of thought provided by the group without seriously thinking them through.
One concept which I think is strongly related is common knowledge (if we extend it to not only refer to beliefs but also to norms).
I think that a good part of the difference between the scenarios for answering Alec is captured by the difference between sharing personal advice and knowledge compared to sharing common norms and knowledge. The latter will give Alec a lot of information about how to work together and cooperate with “the typical member of” the AI safety community, which is important information independently of whether it would be best for Alec’s thinking to adopt the beliefs or follow the advice. At least he will be able to navigate working together with others in the community even if he disagrees with their opinions.
Of course there still can be the failure mode of just aiming to fit in into the group and neglecting the group’s intended purpose.
I think one important aspect here is that mentally treating a group’s norms, behaviours and beliefs as being simple and uniform has huge difficulty advantages for establishing common knowledge and makes it much simpler for an individual to navigate the group. “The Intelligent Social Web” nicely describes this and also the pull that people feel towards acting out well trodden social roles.
Of course, AI safety might well be one of the topics where we really want people not to just adopt the patterns of thought provided by the group without seriously thinking them through.