“So coordination to do better than this would be great”.
I’d be curious to know what you’d want to aim for here—both in a mostly ideal world, and what seems most expedient.
As far as the ideal, I happened to write something about in another comment yesterday. Excerpt:
Best: we first prevent hardware progress and stop H100 manufactoring for a bit, then we prevent AI algorithmic progress, and then we stop scaling (ideally in that order). Then, we heavily invest in long run safety research agendas and hold the pause for a long time (20 years sounds good to start). This requires heavy international coordination.
As far as expedient, something like:
Demand labs have good RSPs (or something similar) using inside and outside game, try to get labs to fill in tricky future details of these RSPs as early as possible without depending on “magic” (speculative future science which hasn’t yet been verified). Have AI takeover motivated people work on the underlying tech and implementation.
Work on policy and aim for powerful US policy interventions in parallel. Other countries could also be relevant.
Both of these are unlikely to perfectly succeed, but seems like good directions to push on.
I think pushing for AI lab scaling pauses is probably net negative right now, but I don’t feel very strongly either way (it mostly just feels not that leveraged overall). I think slowing down hardware progress seems clearly good if we could do it at low cost, but seems super intractible.
Thanks, this seems very reasonable. I’d missed your other comment. (Oh and I edited my previous comment for clarity: I guess you were disagreeing with my clumsily misleading wording, rather than what I meant(??))
(Oh and I edited my previous comment for clarity: I guess you were disagreeing with my clumsily misleading wording, rather than what I meant(??))
Corresponding comment text:
This makes sense, but seems to rely on the human spending most of their time tackling well-defined but non-trivial problems where an AI doesn’t need to be re-directed frequently [EDIT: the preceding was poorly worded—I meant that if prior to the availability of AI assistants this were true, it’d allow a lot of speedup as the AIs take over this work; otherwise it’s less clearly so helpful].
I think I disagree with what you meant, but not that strongly. It’s not that important, so I don’t really want to get into it. Basically, I don’t think that “well-defined” is that important (not obviously required for some ability to judge the finished work) and I don’t think “re-direction frequency” is the right way to think about.
As far as the ideal, I happened to write something about in another comment yesterday. Excerpt:
As far as expedient, something like:
Demand labs have good RSPs (or something similar) using inside and outside game, try to get labs to fill in tricky future details of these RSPs as early as possible without depending on “magic” (speculative future science which hasn’t yet been verified). Have AI takeover motivated people work on the underlying tech and implementation.
Work on policy and aim for powerful US policy interventions in parallel. Other countries could also be relevant.
Both of these are unlikely to perfectly succeed, but seems like good directions to push on.
I think pushing for AI lab scaling pauses is probably net negative right now, but I don’t feel very strongly either way (it mostly just feels not that leveraged overall). I think slowing down hardware progress seems clearly good if we could do it at low cost, but seems super intractible.
Thanks, this seems very reasonable. I’d missed your other comment.
(Oh and I edited my previous comment for clarity: I guess you were disagreeing with my clumsily misleading wording, rather than what I meant(??))
Corresponding comment text:
I think I disagree with what you meant, but not that strongly. It’s not that important, so I don’t really want to get into it. Basically, I don’t think that “well-defined” is that important (not obviously required for some ability to judge the finished work) and I don’t think “re-direction frequency” is the right way to think about.