To rephrase the primary disagreement: “Yes, AIs are incredibly, world-threateningly dangerous, but there’s nothing you can do about it.”
Is that really a disagreement? If the current SingInst can’t make direct contributions, AGI researchers can, by not pushing AGI capability progress. This issue is not addressed, the heuristic of endorsing technological progress has too much support in researchers’ minds to take seriously the possible consequences of following it in this instance.
In other words, there are separate questions of whether current SingInst is irrelevant and whether AI safety planning is irrelevant. If the status quo is to try out various things and see what happens, there is probably room for improvement over this process, even if particular actions of SingInst are deemed inadequate. Pointing out possible issues with SingInst doesn’t address the relevance of AI safety planning.
Is that really a disagreement? If the current SingInst can’t make direct contributions, AGI researchers can, by not pushing AGI capability progress. This issue is not addressed, the heuristic of endorsing technological progress has too much support in researchers’ minds to take seriously the possible consequences of following it in this instance.
In other words, there are separate questions of whether current SingInst is irrelevant and whether AI safety planning is irrelevant. If the status quo is to try out various things and see what happens, there is probably room for improvement over this process, even if particular actions of SingInst are deemed inadequate. Pointing out possible issues with SingInst doesn’t address the relevance of AI safety planning.
Agreed. But it does mean SI “loses the argument”. Yahtzee!