I think it’s plausible that good monitors will make consumer applications of AI more capable and effective. In some sense, safety is a relevant blocker for parts of that at the moment.
Though, I think it is quite unlikely to push the frontier and I think the negative externalities of non-lab developers being faster at coding are very small. On average, it just seems to increase productivity.
I’d also expect that the monitors we build and are not directly targeted at making frontier AIs more effective, don’t happen to be more effective at that then the 100s of employees who push the boundaries of the frontier full-time.
So on balance, I think the risk is pretty low and the benefits are high. This was one of the considerations we thought through in depth before making the decision to make monitors.
The main downside here seems to be that having good monitors will make AIs more capable for commercial use cases --> which means developing foundation models will be (knowingly) more profitable, less likely to be a bubble, and so on.
What do you think?
Also, got opinions about the “sharing examples of misalignment while preserving privacy” part?
Also, you wrote:
the 100s of employees who push the boundaries of the frontier full-time
These employees aren’t all working on things like safety monitors (or whatever your startup would specialize in).
I think it’s plausible that good monitors will make consumer applications of AI more capable and effective. In some sense, safety is a relevant blocker for parts of that at the moment.
Though, I think it is quite unlikely to push the frontier and I think the negative externalities of non-lab developers being faster at coding are very small. On average, it just seems to increase productivity.
I’d also expect that the monitors we build and are not directly targeted at making frontier AIs more effective, don’t happen to be more effective at that then the 100s of employees who push the boundaries of the frontier full-time.
So on balance, I think the risk is pretty low and the benefits are high. This was one of the considerations we thought through in depth before making the decision to make monitors.
The main downside here seems to be that having good monitors will make AIs more capable for commercial use cases --> which means developing foundation models will be (knowingly) more profitable, less likely to be a bubble, and so on.
What do you think?
Also, got opinions about the “sharing examples of misalignment while preserving privacy” part?
Also, you wrote:
These employees aren’t all working on things like safety monitors (or whatever your startup would specialize in).