Because you can speed up AI capabilities much easier while being sloppy than to produce actually good alignment ideas.
Right, my point is, I don’t see any difference between “AIs that produce slop” and “weak AIs” (a.k.a. “dumb AIs”). So from my perspective, the above is similar to : “…Because weak AIs can speed up AI capabilities much easier than they can produce actually good alignment ideas.”
…And then if you follow through the “logic” of this OP, then the argument becomes: “AI alignment is a hard problem, so let’s just make extraordinarily powerful / smart AIs right now, so that they can solve the alignment problem”.
See the error?
If you really think you need to be similarly unsloppy to build ASI than to align ASI, I’d be interested in discussing that. So maybe give some pointers to why you might think that (or tell me to start).
I don’t think that. See the bottom part of the comment you’re replying to. (The part after “Here’s what I would say instead:”)
I don’t think that. See the bottom part of the comment you’re replying to. (The part after “Here’s what I would say instead:”)
Sry my comment was sloppy.
Right, my point is, I don’t see any difference between “AIs that produce slop” and “weak AIs” (a.k.a. “dumb AIs”).
(I agree the way I used sloppy in my comment mostly meant “weak”. But some other thoughts:)
So I think there are some dimensions of intelligence which are more important for solving alignment than for creating ASI. If you read planecrash, WIS and rationality training seem to me more important in that way than INT. I don’t really have much hope for DL-like systems solving alignment but a similar case might be if an early transformative AI recognizes and says “no I cannot solve the alignment problem. the way my intelligence is shaped is not well suited to avoiding value drift. we should stop scaling and take more time where I work with very smart people like Eliezer etc for some years to solve alignment”. And depending on the intelligence profile of the AI it might be more or less likely that this will happen (currently seems quite unlikely). But overall those “better” intelligence dimensions still seem to me too central for AI capabilities, so I wouldn’t publish stuff.
(Btw the way I read John’s post was more like “fake alignment proposals are a main failure mode” rather than also ”… and therefore we should work on making AIs more rational/sane whatever”. So given that I maybe would defend John’s framing, but not sure.)
Right, my point is, I don’t see any difference between “AIs that produce slop” and “weak AIs” (a.k.a. “dumb AIs”). So from my perspective, the above is similar to : “…Because weak AIs can speed up AI capabilities much easier than they can produce actually good alignment ideas.”
I want to explicitly call out my cliff vs gentle slope picture from another recent comment. Sloppy AIs can have a very large set of tasks at which they perform very well, but they have sudden drops in their abilities due to failure to extrapolate well outside of that.
Right, my point is, I don’t see any difference between “AIs that produce slop” and “weak AIs” (a.k.a. “dumb AIs”). So from my perspective, the above is similar to : “…Because weak AIs can speed up AI capabilities much easier than they can produce actually good alignment ideas.”
…And then if you follow through the “logic” of this OP, then the argument becomes: “AI alignment is a hard problem, so let’s just make extraordinarily powerful / smart AIs right now, so that they can solve the alignment problem”.
See the error?
I don’t think that. See the bottom part of the comment you’re replying to. (The part after “Here’s what I would say instead:”)
Sry my comment was sloppy.
(I agree the way I used sloppy in my comment mostly meant “weak”. But some other thoughts:)
So I think there are some dimensions of intelligence which are more important for solving alignment than for creating ASI. If you read planecrash, WIS and rationality training seem to me more important in that way than INT.
I don’t really have much hope for DL-like systems solving alignment but a similar case might be if an early transformative AI recognizes and says “no I cannot solve the alignment problem. the way my intelligence is shaped is not well suited to avoiding value drift. we should stop scaling and take more time where I work with very smart people like Eliezer etc for some years to solve alignment”. And depending on the intelligence profile of the AI it might be more or less likely that this will happen (currently seems quite unlikely).
But overall those “better” intelligence dimensions still seem to me too central for AI capabilities, so I wouldn’t publish stuff.
(Btw the way I read John’s post was more like “fake alignment proposals are a main failure mode” rather than also ”… and therefore we should work on making AIs more rational/sane whatever”. So given that I maybe would defend John’s framing, but not sure.)
I want to explicitly call out my cliff vs gentle slope picture from another recent comment. Sloppy AIs can have a very large set of tasks at which they perform very well, but they have sudden drops in their abilities due to failure to extrapolate well outside of that.