Firstly, your researchers normally have access to the model architecture. This is unfortunate if you want to avoid it leaking. It’s not clear how important this is. My sense is that changes to model architecture have been a minority of the algorithmic improvement since the invention of the transformer.
I agree with this, but I’d say it’s good to do this anyways, because if AIs start being able to do more and more research, then the chances of architecture/paradigm changes goes up, and this is especially true if AI labor scales faster than human labor, so it’s worth preventing this possibility early on.
I agree with this, but I’d say it’s good to do this anyways, because if AIs start being able to do more and more research, then the chances of architecture/paradigm changes goes up, and this is especially true if AI labor scales faster than human labor, so it’s worth preventing this possibility early on.
Also, good news on the new Tinker API.