That period where AIs are more capable than human, but human+AI is even more capable, seems like a particularly crucial window for doing useful things, so extending it is pretty valuable. In particular, both bringing forward augmented human capability, and also pushing back human redundance.
In the context of risks from misalignment I often think about a type of interventions which is something like: “making AIs more useful earlier, at a point when their general abilities and opaque reasoning etc make them less dangerous (as in, less likely to be egregiously misalignment and less likely to be able to successfully subvert safety/security measures)”.
I don’t quite see why this sort of thing (but for boosting human+AI rather than increasing the weak AI boost) helps much with gradual disempowerment. It seems to me like the forces you mention apply almost as much to cases where there is substantial uplift so long as AI companies still have a bunch of control and uplift isn’t widely diffused.
But, I also don’t think I can hold a coherant model of gradual disempowerment which makes sense to me and matches the claims in the paper in my head, so I might just be missing some aspect of the situation.
Regardless, probably not worth getting into it further.
In parallel, I think that a lot of work is defaulting towards ‘fully general agent AI’ because it is an easy and natural target, not because it is the best one, and that if people knew what other kinds of interfaces to build for, that would actually suck some energy out of investing in getting long-term planning/drop-in replacements for everything as soon as possible
I’m skeptical that making AIs more useful for uplifting humans will delay general purpose autonomous capabilities, because probably the most effective ways to get more uplift is to drive forward general capabilities which will transfer to both. Like, making AIs more useful for X will make more people work on X (rather than general capabilities) but will also make more people invest in AI in general.
At a more basic level, the timelines slowing effect has to be pretty minor/marginal.
Reasons 3 and 4 make sense to me in principle, though I don’t think I buy them in practice. Or at least they feel quite weak to me. This might again be downstream of me not being able to model gradual disempowerment in a way that makes sense to me.
On the first point: you’re right that this does in some ways make the problem worse; my current best guess is that it’s basically necessary for a solution. I’m planning to write this up in more detail some time soon and I hope to get your thoughts when I do!
On the second: Yeah, I find this kind of thing pretty hard to be confident about. I could totally see you being right here, and I’d love for someone to think it through in detail.
And I think the differences in 3 and 4 indeed probably come down to deeper assumptions that would be hard to unpick in this thread: I’d tentatively guess I’m putting more weight on the societal impacts of AI, and on the eventual shape of AGI/ASI being easier to affect.
This comment thread probably isn’t the place, but if it ever seems like it would be important/feasible, I’d be happy to try to go deeper on where our models are differing.
In the context of risks from misalignment I often think about a type of interventions which is something like: “making AIs more useful earlier, at a point when their general abilities and opaque reasoning etc make them less dangerous (as in, less likely to be egregiously misalignment and less likely to be able to successfully subvert safety/security measures)”.
I don’t quite see why this sort of thing (but for boosting human+AI rather than increasing the weak AI boost) helps much with gradual disempowerment. It seems to me like the forces you mention apply almost as much to cases where there is substantial uplift so long as AI companies still have a bunch of control and uplift isn’t widely diffused.
But, I also don’t think I can hold a coherant model of gradual disempowerment which makes sense to me and matches the claims in the paper in my head, so I might just be missing some aspect of the situation.
Regardless, probably not worth getting into it further.
I’m skeptical that making AIs more useful for uplifting humans will delay general purpose autonomous capabilities, because probably the most effective ways to get more uplift is to drive forward general capabilities which will transfer to both. Like, making AIs more useful for X will make more people work on X (rather than general capabilities) but will also make more people invest in AI in general.
At a more basic level, the timelines slowing effect has to be pretty minor/marginal.
Reasons 3 and 4 make sense to me in principle, though I don’t think I buy them in practice. Or at least they feel quite weak to me. This might again be downstream of me not being able to model gradual disempowerment in a way that makes sense to me.
Sure, briefly replying:
On the first point: you’re right that this does in some ways make the problem worse; my current best guess is that it’s basically necessary for a solution. I’m planning to write this up in more detail some time soon and I hope to get your thoughts when I do!
On the second: Yeah, I find this kind of thing pretty hard to be confident about. I could totally see you being right here, and I’d love for someone to think it through in detail.
And I think the differences in 3 and 4 indeed probably come down to deeper assumptions that would be hard to unpick in this thread: I’d tentatively guess I’m putting more weight on the societal impacts of AI, and on the eventual shape of AGI/ASI being easier to affect.
This comment thread probably isn’t the place, but if it ever seems like it would be important/feasible, I’d be happy to try to go deeper on where our models are differing.