I don’t strongly disagree re architectures, but I do think we are uncertain about this. Depending on AGI architecture, different forms of regulation may or may not work. Work should be carried out to determine which regulation works for how many flops needed for takeover-level AI.
That it’s not happening yet is 1) no reason it won’t (xrisk awareness is just too low, but slowly rising) and 2) equally applicable to the alternative you propose, universal surveillance.
If we treat universal surveillance seriously, we should consider its downsides as well. First, there’s no proof it would work: I’m not sure an AI, even a future one, would necessarily catch all actions towards building AGI. I have no idea what these actions are, and no idea which actions a surveillance AI with some real-world sensors can catch (or could be blocked etc.). I think we should not be more than 70% confident this would technically work. Second, currently we have power vacuums in the world, such as failed states, revolutions, criminal groups, or just instances were those in power are unable to project their power effectively. How would we apply universal surveillance to those power vacuums? Or do we assume they won’t exist anymore, and if so, why is that assumption justified? Third, universal surveillance is arguably the world’s least popular policy. It seems outright impossible to implement this in any democratic way. Perhaps the plan is to implement it by force through an AGI, then I would file it as a form of pivotal act. If we’re anyway in pivotal act territory, I’d strongly prefer Yudkowsky’s “subtly modifying all GPUs such that they can no longer train an AGI” (kind of hardware regulation, really) over universal surveillance.
I think research is urgently required into how to implement a pause effectively. We have one report almost finished on the topic that mostly focuses on hardware regulation. PauseAI is working on a Building a pause button-project that is a bit similar. Other orgs should do work on this as well, and compare options such as hardware regulation, universal surveillance, data regulation, etc. and conclude in which AGI regime (how many flops, how much hardware required) these options are valid.
True, I guess we’re not in significant disagreement here.
I don’t strongly disagree re architectures, but I do think we are uncertain about this. Depending on AGI architecture, different forms of regulation may or may not work. Work should be carried out to determine which regulation works for how many flops needed for takeover-level AI.
That it’s not happening yet is 1) no reason it won’t (xrisk awareness is just too low, but slowly rising) and 2) equally applicable to the alternative you propose, universal surveillance.
If we treat universal surveillance seriously, we should consider its downsides as well. First, there’s no proof it would work: I’m not sure an AI, even a future one, would necessarily catch all actions towards building AGI. I have no idea what these actions are, and no idea which actions a surveillance AI with some real-world sensors can catch (or could be blocked etc.). I think we should not be more than 70% confident this would technically work. Second, currently we have power vacuums in the world, such as failed states, revolutions, criminal groups, or just instances were those in power are unable to project their power effectively. How would we apply universal surveillance to those power vacuums? Or do we assume they won’t exist anymore, and if so, why is that assumption justified? Third, universal surveillance is arguably the world’s least popular policy. It seems outright impossible to implement this in any democratic way. Perhaps the plan is to implement it by force through an AGI, then I would file it as a form of pivotal act. If we’re anyway in pivotal act territory, I’d strongly prefer Yudkowsky’s “subtly modifying all GPUs such that they can no longer train an AGI” (kind of hardware regulation, really) over universal surveillance.
I think research is urgently required into how to implement a pause effectively. We have one report almost finished on the topic that mostly focuses on hardware regulation. PauseAI is working on a Building a pause button-project that is a bit similar. Other orgs should do work on this as well, and compare options such as hardware regulation, universal surveillance, data regulation, etc. and conclude in which AGI regime (how many flops, how much hardware required) these options are valid.
True, I guess we’re not in significant disagreement here.