I agree with the thesis of this post and think that the level of model access available to people working on making AI go better is extremely important.
Here are some other arguments for the importance of external model access:
We should expect that the majority of quality-weighted safety labor will be outside of the leading AI company at the point when safety work is most important.
If you’re currently working at a frontier AI company (OpenAI/Anthropic/GDM) and you plan to work on safety during the singularity, you should expect to be outside of the leading AI company when safety work is most important. This is because the company you work at probably won’t be the leading company and you’ll likely end up either on the outside or at a trailing company. If there is good model access, then you’ll at least have the option of working outside the leading company with good model access which might massively increase your future productivity.[1]
Even if you’re working at the leading AI company (on safety), you might not get model access at the most important time: it seems plausible that model access within that company will be restricted to an increasingly narrow subset of employees (possibly in a secret silo within the company). It seems better to push for broader model access and have this fight at an earlier point.
Technical work that helps with external model access also transfers to internal model access (and vice versa). Improved external model access probably requires figuring out technical approaches to mitigate various (myopic) misuse, security, and IP concerns. This especially applies for things like fine-tuning/RL APIs. Internal model access to employees might also be blocked due to these concerns and the useful mitigations are similar.
This last bullet also makes me somewhat more optimistic about access in general and especially access to fine-tuning/RL APIs: over time[2], AI companies will want to depend less on trusting their employees and more on ensuring it’s OK if adversaries get employee-level access (to APIs and models). So companies will be incentivized to make fine-tuning APIs (and other APIs) that are secure, leak less IP, and are robust to misuse. This will make it relatively cheaper to give external groups various types of access but especially things like fine-tuning APIs[3] (while right now companies aren’t that strongly incentivized to make fine-tuning APIs that are fine to give to external groups).
I would distinguish between using AI for automating work / uplift and studying AIs. For the second of these, ideal access looks somewhat different and things like fine-tuning APIs, APIs that give access to model internals, and the ability to prefill assistant turns[4] look much more important. I think access for studying AIs is comparably important to access for uplift.
I wrote this argument in a way where it is directed at frontier AI company employees, but it’s just another argument that lots of relevant safety labor will be external.
Driven by a mix of increased capabilities, wanting to employ a larger number of (less trusted) people, government pressure, and AIs being more useful for building the needed infrastructure.
This doesn’t mean fine-tuning API access will happen or that there won’t be a bunch of additional risks (from the perspective of companies) associated with giving access to external groups. For instance, fine-tuning APIs might still leak some sensitive IP and pose significant additional API misuse risk. If the number of external people given access is significantly smaller than the number of employees with access (e.g. 1⁄5 as many), then it seems totally doable to do this in a way where actual risk doesn’t increase much.
I agree with the thesis of this post and think that the level of model access available to people working on making AI go better is extremely important.
Here are some other arguments for the importance of external model access:
We should expect that the majority of quality-weighted safety labor will be outside of the leading AI company at the point when safety work is most important.
If you’re currently working at a frontier AI company (OpenAI/Anthropic/GDM) and you plan to work on safety during the singularity, you should expect to be outside of the leading AI company when safety work is most important. This is because the company you work at probably won’t be the leading company and you’ll likely end up either on the outside or at a trailing company. If there is good model access, then you’ll at least have the option of working outside the leading company with good model access which might massively increase your future productivity. [1]
Even if you’re working at the leading AI company (on safety), you might not get model access at the most important time: it seems plausible that model access within that company will be restricted to an increasingly narrow subset of employees (possibly in a secret silo within the company). It seems better to push for broader model access and have this fight at an earlier point.
Technical work that helps with external model access also transfers to internal model access (and vice versa). Improved external model access probably requires figuring out technical approaches to mitigate various (myopic) misuse, security, and IP concerns. This especially applies for things like fine-tuning/RL APIs. Internal model access to employees might also be blocked due to these concerns and the useful mitigations are similar.
This last bullet also makes me somewhat more optimistic about access in general and especially access to fine-tuning/RL APIs: over time [2] , AI companies will want to depend less on trusting their employees and more on ensuring it’s OK if adversaries get employee-level access (to APIs and models). So companies will be incentivized to make fine-tuning APIs (and other APIs) that are secure, leak less IP, and are robust to misuse. This will make it relatively cheaper to give external groups various types of access but especially things like fine-tuning APIs [3] (while right now companies aren’t that strongly incentivized to make fine-tuning APIs that are fine to give to external groups).
I would distinguish between using AI for automating work / uplift and studying AIs. For the second of these, ideal access looks somewhat different and things like fine-tuning APIs, APIs that give access to model internals, and the ability to prefill assistant turns [4] look much more important. I think access for studying AIs is comparably important to access for uplift.
I wrote this argument in a way where it is directed at frontier AI company employees, but it’s just another argument that lots of relevant safety labor will be external.
Driven by a mix of increased capabilities, wanting to employ a larger number of (less trusted) people, government pressure, and AIs being more useful for building the needed infrastructure.
This doesn’t mean fine-tuning API access will happen or that there won’t be a bunch of additional risks (from the perspective of companies) associated with giving access to external groups. For instance, fine-tuning APIs might still leak some sensitive IP and pose significant additional API misuse risk. If the number of external people given access is significantly smaller than the number of employees with access (e.g. 1⁄5 as many), then it seems totally doable to do this in a way where actual risk doesn’t increase much.
What Anthropic calls “partial-turn prefill”.