I strongly disagree with your statement here Donald. I think that the level of capability you describe here as ‘not dangerous’ is what I would describe as ‘extremely dangerous’. An AI agent which has super-human capabilities but restricts itself to human-level outputs because of the quirks of its training process can still accomplish everything necessary to destroy humanity. The key limiting factor in your example is not the model’s capability but rather its agency.
Yeah, used carefully and intentionally by well-intentioned actors (not reckless or criminal or suicidal terrorists or...) and no big deal surprises… And no rapid further advances building off of where we’ve gotten so far… If all of those things were somehow true, then yeah, much less dangerous.
I strongly disagree with your statement here Donald. I think that the level of capability you describe here as ‘not dangerous’ is what I would describe as ‘extremely dangerous’. An AI agent which has super-human capabilities but restricts itself to human-level outputs because of the quirks of its training process can still accomplish everything necessary to destroy humanity. The key limiting factor in your example is not the model’s capability but rather its agency.
Ok, maybe my wording should be more like, “this probably wont destroy the world if it is used carefully and there are no extra phenomena we missed.”
Yeah, used carefully and intentionally by well-intentioned actors (not reckless or criminal or suicidal terrorists or...) and no big deal surprises… And no rapid further advances building off of where we’ve gotten so far… If all of those things were somehow true, then yeah, much less dangerous.