Lets say we manage to implement a world wide ban/pause on large training runs, what happens next?
Well obviously smaller training runs, up to whatever limit has been imposed, or no training runs for some time.[1]
The next obvious thing that happens, and btw is already happening in the open source community, would be optimizing algorithms. You have a limit on compute? Well then you OBVIOUSLY will try and make the most of the compute you have.
Pour tons of money into research, first order of business is to make the formal case for x-risk is a thing and must actively be mitigated. Or said another way, we need humans aligned on “alignment does not happen by default”[3]
Next order of business, assuming alignment does not happen by default, is to formally produce and verify plans for how to build safe / aligned cognitive architectures.
And all the while there cannot be any training runs, and no work on algorithmic optimization or cognitive architectures in general.[4]
The problem is we can’t do that, its too late, the cat is out of the bag, there is too much money to be made in the short term, open source is plowing ahead and the amount of people who actually looked at the entire edifice for long enough to realize “yeah you know what I think we have a problem, we really must look into if that is real or not, and if it is we need to figure out what it takes to do this risk free” is miniscule compared to the amount of people who go “bah, it will be fine, don’t be such a drama queen”
And that’s why I think a pause at best extends timelines ever so slightly, and at worst they shorten them considerably, and either way the outcomes remains unchanged.
Contingent on how hard the problem is—if we need 100 years to solve the problem, we would destroy the world many time over if we plowed ahead with capabilities research.
How does pausing change much of anything?
Lets say we manage to implement a world wide ban/pause on large training runs, what happens next?
Well obviously smaller training runs, up to whatever limit has been imposed, or no training runs for some time.[1]
The next obvious thing that happens, and btw is already happening in the open source community, would be optimizing algorithms. You have a limit on compute? Well then you OBVIOUSLY will try and make the most of the compute you have.
Non of that fixes anything.
What we should do:[2]
Pour tons of money into research, first order of business is to make the formal case for x-risk is a thing and must actively be mitigated. Or said another way, we need humans aligned on “alignment does not happen by default” [3]
Next order of business, assuming alignment does not happen by default, is to formally produce and verify plans for how to build safe / aligned cognitive architectures.
And all the while there cannot be any training runs, and no work on algorithmic optimization or cognitive architectures in general.[4]
The problem is we can’t do that, its too late, the cat is out of the bag, there is too much money to be made in the short term, open source is plowing ahead and the amount of people who actually looked at the entire edifice for long enough to realize “yeah you know what I think we have a problem, we really must look into if that is real or not, and if it is we need to figure out what it takes to do this risk free” is miniscule compared to the amount of people who go “bah, it will be fine, don’t be such a drama queen”
And that’s why I think a pause at best extends timelines ever so slightly, and at worst they shorten them considerably, and either way the outcomes remains unchanged.
Except people will do runs no matter what, the draconian measures needed will not happen, cannot happen.
Actually its what we should have done.
Unless of course it does, and a formal proof of this can be produced.
Contingent on how hard the problem is—if we need 100 years to solve the problem, we would destroy the world many time over if we plowed ahead with capabilities research.