The idea that there would be a distinct “before” and “after” is also not supported by current evidence which has shown continuous (though exponential!) growth of capabilities over time.
The time when the AI can optimize itself better then a human is a one-off event. You get the overhang/potential take-off here. Also the AI having a coherent sense of “self” that it could protect by say changing its own code, controlling instances of itself could be an attractor and give “before/after”.
Also I don’t think sense of “self” is a singular event either, indeed already today’s systems are growing in their situational awareness which can be thought as some sense of self. See our scheming paper https://www.antischeming.ai/
The time when the AI can optimize itself better then a human is a one-off event. You get the overhang/potential take-off here. Also the AI having a coherent sense of “self” that it could protect by say changing its own code, controlling instances of itself could be an attractor and give “before/after”.
See this response
Also I don’t think sense of “self” is a singular event either, indeed already today’s systems are growing in their situational awareness which can be thought as some sense of self. See our scheming paper https://www.antischeming.ai/