Suppose that an AI PotRogue-1 is released to the public. Then OpenBrain leaders or researchers, who are already concerned with the ways to check alignment, might also ask their more powerful Agent-2 to check whether PotRogue-1 is misaligned and/or is finetunable by terrorists. If Agent-2 finds that the answer for any of the two questions is “yes”, then the researchers ask the leaders to lobby the laws AGAINST open-source AIs (and, potentially, to have the compute confiscated and sold to OpenBrain). Otherwise PotRogue-1 is actually harmless...
Suppose that an AI PotRogue-1 is released to the public. Then OpenBrain leaders or researchers, who are already concerned with the ways to check alignment, might also ask their more powerful Agent-2 to check whether PotRogue-1 is misaligned and/or is finetunable by terrorists. If Agent-2 finds that the answer for any of the two questions is “yes”, then the researchers ask the leaders to lobby the laws AGAINST open-source AIs (and, potentially, to have the compute confiscated and sold to OpenBrain). Otherwise PotRogue-1 is actually harmless...