The first thing I would note is that stakeholders need to be involved in making any guidelines, and that pushing for guidelines from the outside is unhelpful, if not harmful, since it pushes participants to be defensive about their work. There are also an extensive literature discussing the general issue of information dissemination hazards and the issues of regulation in other domains, such as nuclear weapons technology, biological and chemical weapons, and similar.
The first thing I would note is that stakeholders need to be involved in making any guidelines, and that pushing for guidelines from the outside is unhelpful, if not harmful, since it pushes participants to be defensive about their work.
Hmm, maybe I was unclear. When I said that “we need to have a public debate on the topic inside the community” I meant, the community of AI alignment researchers. So, not from the outside.
As to the links, thank you. They do seem like potentially valuable inputs into the debate, although (from skimming) they don’t seem to reach the point of proposing concrete guidelines and procedures.
I think there needs to be individual decisionmaking (on the part of both organizations and individual researchers, especially in light of the unilateralists’ curse,) alongside a much broader discussion about how the world should handle unsafe machine learning, and more advanced AI.
I very much don’t think that the AI safety community debating and coming up with shared, semi-public guidelines for, essentially, what to withhold from the broader public, done without input from the wider ML / AI and research community who are impacted and whose work is a big part of what we are discussing, would be wise. That community needs to be engaged in any such discussions.
I’m not talking about guidelines for the wider AI community. I’m talking about guidelines for my own research (and presumably other alignment researchers would be interested in the same). The wider AI community doesn’t share my assumptions about AI risk. In particular, I believe that most of what they’re doing is actively harmful. Therefore, I don’t expect them to accept these guidelines, and I’m also mostly uninterested in their input. Moreover, it’s not the broader public that worries me, but precisely the broader AI community. It is from them that I want to withhold things.
Creating any sort of guidelines that the wider community would also accept is a different sort of challenge altogether. It’s also a job for other people. Personally, I have enough on my plate as it is, and politics is not my comparative advantage by any margin.
I think a page titled “here are some tools and resources for thinking about AI-related infohazards” would be helpful and uncontroversial and feasible… That could include things like a list of trusted people in the community who have an open offer to discuss and offer feedback in confidence, and links to various articles and guidelines on the topic (without necessarily “officially” endorsing any particular approach), etc.
I agree that your proposal is well worth doing, it just sounds a lot more ambitious and long-term.
The first thing I would note is that stakeholders need to be involved in making any guidelines, and that pushing for guidelines from the outside is unhelpful, if not harmful, since it pushes participants to be defensive about their work. There are also an extensive literature discussing the general issue of information dissemination hazards and the issues of regulation in other domains, such as nuclear weapons technology, biological and chemical weapons, and similar.
There is also a fair amount of ongoing work on synthesizing this literature and the implications for AI. Some of it is even on this site. For example, see: https://www.lesswrong.com/posts/RY9XYoqPeMc8W8zbH/mapping-downside-risks-and-information-hazards and https://www.lesswrong.com/posts/6ur8vDX6ApAXrRN3t/information-hazards-why-you-should-care-and-what-you-can-do
So there is tons of discussion about this already, and there is plenty you should read on the topic—I suspect you can start with the paper that provided the name for your post, and continuing with sections of GovAI’s research agenda.
Hmm, maybe I was unclear. When I said that “we need to have a public debate on the topic inside the community” I meant, the community of AI alignment researchers. So, not from the outside.
As to the links, thank you. They do seem like potentially valuable inputs into the debate, although (from skimming) they don’t seem to reach the point of proposing concrete guidelines and procedures.
I think there needs to be individual decisionmaking (on the part of both organizations and individual researchers, especially in light of the unilateralists’ curse,) alongside a much broader discussion about how the world should handle unsafe machine learning, and more advanced AI.
I very much don’t think that the AI safety community debating and coming up with shared, semi-public guidelines for, essentially, what to withhold from the broader public, done without input from the wider ML / AI and research community who are impacted and whose work is a big part of what we are discussing, would be wise. That community needs to be engaged in any such discussions.
I’m not talking about guidelines for the wider AI community. I’m talking about guidelines for my own research (and presumably other alignment researchers would be interested in the same). The wider AI community doesn’t share my assumptions about AI risk. In particular, I believe that most of what they’re doing is actively harmful. Therefore, I don’t expect them to accept these guidelines, and I’m also mostly uninterested in their input. Moreover, it’s not the broader public that worries me, but precisely the broader AI community. It is from them that I want to withhold things.
Creating any sort of guidelines that the wider community would also accept is a different sort of challenge altogether. It’s also a job for other people. Personally, I have enough on my plate as it is, and politics is not my comparative advantage by any margin.
I think a page titled “here are some tools and resources for thinking about AI-related infohazards” would be helpful and uncontroversial and feasible… That could include things like a list of trusted people in the community who have an open offer to discuss and offer feedback in confidence, and links to various articles and guidelines on the topic (without necessarily “officially” endorsing any particular approach), etc.
I agree that your proposal is well worth doing, it just sounds a lot more ambitious and long-term.