“modeling would help the AI safety community feel confident enough to strongly advocate for a ban on RSI”.
Another positive impact of the modeling is it could convince politicians of the need to implement good AI regulations soon, because I currently model elites/politicians as being AI pilled in the sense that AI is doing real things and matters for lots of things, but they currently aren’t bought onto the idea of a software-only/software + hardware singularity, and if the AI safety community is correct on this point, modeling can help make it legible to other people.
Also, to a first approximation, the question of a software-only singularity is one of the most important cruxes, and explains many, many deep disagreements between factions.
Another positive impact of the modeling is it could convince politicians of the need to implement good AI regulations soon
This feels like exactly the same mistake I was critiquing Thomas for making. You can rationalize many things as helpful for convincing politicians, but if you sat down and actually tried to model what things would make politicians update in good directions, this would end up far from the top of your list.
(I do want to flag that it’s worth avoiding “optimizing over politicians” in an overly adversarial way, but I think my claim holds even given the constraint of cooperativeness.)
Another positive impact of the modeling is it could convince politicians of the need to implement good AI regulations soon, because I currently model elites/politicians as being AI pilled in the sense that AI is doing real things and matters for lots of things, but they currently aren’t bought onto the idea of a software-only/software + hardware singularity, and if the AI safety community is correct on this point, modeling can help make it legible to other people.
Also, to a first approximation, the question of a software-only singularity is one of the most important cruxes, and explains many, many deep disagreements between factions.
This feels like exactly the same mistake I was critiquing Thomas for making. You can rationalize many things as helpful for convincing politicians, but if you sat down and actually tried to model what things would make politicians update in good directions, this would end up far from the top of your list.
(I do want to flag that it’s worth avoiding “optimizing over politicians” in an overly adversarial way, but I think my claim holds even given the constraint of cooperativeness.)