I’m very impressed by the proposed Total Research Transparency. I actually found it appealing even beyond the many reasons mentioned in the plan. It takes advantage of key properties of the current training paradigm, and this is actually desirable, because these properties are likely to remain in future AI systems:
Model training and serving will keep incentivizing a small number of large neural networks, with agent diversity coming from context. Efficiency of batched inference is too large, the learning algorithm and the hardware have co-evolved around it. My opinion is that continual learning will be hard on hardware like the current one because of loss in batching efficiency. And radically different hardware will take longer to develop.
The auto-regressive architecture will also likely remain, because having a chain of tokens empowers the model with the ability to perform inherently serial computation, making it strictly more computationally powerful. This makes it possible to create verification schemes with teeth (via CoT monitoring etc).
So open sourcing the algorithms, closing the weights and monitoring inference seems like the right balance.
If we create an actual recommendation from this idea, how can we actually get labs to cooperate on it? AI companies will likely dislike it, but for example Boaz Barak actually endorsed the push for increased transparency in this tweet. I hope the authors have a plan to push for it beyond this writeup.
I’m very impressed by the proposed Total Research Transparency. I actually found it appealing even beyond the many reasons mentioned in the plan. It takes advantage of key properties of the current training paradigm, and this is actually desirable, because these properties are likely to remain in future AI systems:
Model training and serving will keep incentivizing a small number of large neural networks, with agent diversity coming from context. Efficiency of batched inference is too large, the learning algorithm and the hardware have co-evolved around it. My opinion is that continual learning will be hard on hardware like the current one because of loss in batching efficiency. And radically different hardware will take longer to develop.
The auto-regressive architecture will also likely remain, because having a chain of tokens empowers the model with the ability to perform inherently serial computation, making it strictly more computationally powerful. This makes it possible to create verification schemes with teeth (via CoT monitoring etc).
So open sourcing the algorithms, closing the weights and monitoring inference seems like the right balance.
If we create an actual recommendation from this idea, how can we actually get labs to cooperate on it? AI companies will likely dislike it, but for example Boaz Barak actually endorsed the push for increased transparency in this tweet. I hope the authors have a plan to push for it beyond this writeup.