I like this! Here’s a specific concrete version of it just to have some numbers on the table:
Companies are legally required to allocate their compute as follows: 1. 60% on serving customers. 2. 15% on internal safety research that’s fully transparent to the public. 3. 10% on external safety research that’s fully transparent to the public & has access to the latest internal models. 4. 15% on whatever else they want (so, presumably, core AI R&D and big training runs)
Suppose this were a US law, that applies to all the major AI companies but not to e.g. Chinese companies or Mistral for example.
15% is less than the roughly 50% they seem to be spending now, so overall the pace of AI progress would slow down to maybe something like half the speed it’s going now. But the amount of compute devoted to capabilities progress would still be higher at each US company than any Chinese company for example, so US companies would stay in the lead.
Meanwhile the amount of internal compute going to safety research would double or quadruple, and then the effect size of a ton of external researchers being able to look at the work, critique it, run their own replications and ablations, run their own experiments, etc. would be way way bigger on top of that already big positive effect.
How would it be enforced? Like you said, whistleblowers. If we were trying to ban AI capabilities R&D entirely, that would be super difficult, but if we are just keeping it to 15% of the compute, then cheaters can’t really prosper. It’s really hard to hide 10% of your compute being spent on something it’s not supposed to be spent on; maaayybe you can hide 1% but that wouldn’t be enough to make much difference & so wouldn’t be worth the risk.
This would also prevent a sort of race to the bottom where companies use more and more of their compute on R&D instead of serving customers; instead we’d have the process of AI deployment/diffusion continuing to happen which I think is broadly pretty good for the world.
I don’t feel strongly about the specific numbers above obviously but yeah this seems promising.
I like this! Here’s a specific concrete version of it just to have some numbers on the table:
Companies are legally required to allocate their compute as follows:
1. 60% on serving customers.
2. 15% on internal safety research that’s fully transparent to the public.
3. 10% on external safety research that’s fully transparent to the public & has access to the latest internal models.
4. 15% on whatever else they want (so, presumably, core AI R&D and big training runs)
Suppose this were a US law, that applies to all the major AI companies but not to e.g. Chinese companies or Mistral for example.
15% is less than the roughly 50% they seem to be spending now, so overall the pace of AI progress would slow down to maybe something like half the speed it’s going now. But the amount of compute devoted to capabilities progress would still be higher at each US company than any Chinese company for example, so US companies would stay in the lead.
Meanwhile the amount of internal compute going to safety research would double or quadruple, and then the effect size of a ton of external researchers being able to look at the work, critique it, run their own replications and ablations, run their own experiments, etc. would be way way bigger on top of that already big positive effect.
How would it be enforced? Like you said, whistleblowers. If we were trying to ban AI capabilities R&D entirely, that would be super difficult, but if we are just keeping it to 15% of the compute, then cheaters can’t really prosper. It’s really hard to hide 10% of your compute being spent on something it’s not supposed to be spent on; maaayybe you can hide 1% but that wouldn’t be enough to make much difference & so wouldn’t be worth the risk.
This would also prevent a sort of race to the bottom where companies use more and more of their compute on R&D instead of serving customers; instead we’d have the process of AI deployment/diffusion continuing to happen which I think is broadly pretty good for the world.
I don’t feel strongly about the specific numbers above obviously but yeah this seems promising.