I think it’s great how large a role robotics plays in this analysis. In general, I feel like robotics isn’t given nearly the amount of attention it deserves. Without robotics, AI’s influence on the world is bottlenecked by human hands. So the speed of the feedback loop between AI building better robots to build better robot factories to build better robots seems like a very important crux. So bravo for taking that issue seriously.
Regarding the proposal to make AI research public but not model weights, I think it might be worth giving a bit more thought to the idea the whatever is made public essentially becomes a public good, and public goods tend to be under-supplied—or in this case perhaps “under-supplied”, as reducing the amount of AI research supplied might be considered a good thing or perhaps even the primary benefit of such a policy. Theoretically, if people really were forced to make all AI research public in a way that was instantly useful to competitors, it seems like that would either drastically reduce investment in research or force people to invest in research that other people couldn’t take advantage of, e.g. research that is only useful given certain propriety data or hardware or something. I understand the logic for not making weights public, namely it would be hard to get people to agree, plus you can easily tune out safeguards. But on the flip side, it seems very hard to formalize what exactly counts as “research” (particularly if everyone is trying to skirt that line). Radical total transparency is definitely a big ask, but it seems easier to enforce if agreed. And if weights were mandated to be public, that might also reduce investment, which would be good. Or, again, it might incentivize people to build models where other people couldn’t make use of the weights with out some kind of propriety hardware. (To be clear, I still lean against public weights for the reasons given in the proposal. But I’d be interested to hear a more discussion of this.)
Yep as we say in the piece, we think that it’s better for AI progress to happen mostly via scaling compute rather than mostly via algorithms (that is, in the context of a Plan-A style reversible deal with mutually assured chip destruction). So we think it’s a feature, not a bug, that publishing the algos disincentivizes investment in algo research. There’ll still be some algo progress in Plan A and that’s fine, we model it in the model. (Indeed, it’s probably impossible to solve alignment without making some algo progress as a side-effect)
Two brief random thoughts:
I think it’s great how large a role robotics plays in this analysis. In general, I feel like robotics isn’t given nearly the amount of attention it deserves. Without robotics, AI’s influence on the world is bottlenecked by human hands. So the speed of the feedback loop between AI building better robots to build better robot factories to build better robots seems like a very important crux. So bravo for taking that issue seriously.
Regarding the proposal to make AI research public but not model weights, I think it might be worth giving a bit more thought to the idea the whatever is made public essentially becomes a public good, and public goods tend to be under-supplied—or in this case perhaps “under-supplied”, as reducing the amount of AI research supplied might be considered a good thing or perhaps even the primary benefit of such a policy. Theoretically, if people really were forced to make all AI research public in a way that was instantly useful to competitors, it seems like that would either drastically reduce investment in research or force people to invest in research that other people couldn’t take advantage of, e.g. research that is only useful given certain propriety data or hardware or something. I understand the logic for not making weights public, namely it would be hard to get people to agree, plus you can easily tune out safeguards. But on the flip side, it seems very hard to formalize what exactly counts as “research” (particularly if everyone is trying to skirt that line). Radical total transparency is definitely a big ask, but it seems easier to enforce if agreed. And if weights were mandated to be public, that might also reduce investment, which would be good. Or, again, it might incentivize people to build models where other people couldn’t make use of the weights with out some kind of propriety hardware. (To be clear, I still lean against public weights for the reasons given in the proposal. But I’d be interested to hear a more discussion of this.)
Yep as we say in the piece, we think that it’s better for AI progress to happen mostly via scaling compute rather than mostly via algorithms (that is, in the context of a Plan-A style reversible deal with mutually assured chip destruction). So we think it’s a feature, not a bug, that publishing the algos disincentivizes investment in algo research. There’ll still be some algo progress in Plan A and that’s fine, we model it in the model. (Indeed, it’s probably impossible to solve alignment without making some algo progress as a side-effect)