Thanks for this, I for one hadn’t thought about this variable much and am convinced now that it is one of the more important variables.
--I think acausal trade stuff means that even if all the AIs on Earth are homogenous, the strategic situation may end up being as if they were heterogenous, at least in some ways. I’m not sure, will need to think more about this.
--You talk about this being possible even for gradual, continuous takeoff, yet you also talk about “The first advanced AI system” as if there is a sharp cutoff between advanced and non-advanced AI. I think this isn’t a problem for your overall point, but I’m not sure. For alignment (your point 4) I think this isn’t a problem, because you can just rephrase it as “As we gradually transition from non-advanced systems to advanced systems, it is important that our systems be aligned before we near the end of the transition, and more important the closer we get to the end. Because as our systems become more advanced, their alignment properties become more locked-in.” For deception, I’m less sure. If systems get more advanced gradually and continuously then maybe we can hope there is a “sordid stumble sweet spot” where systems that are deceptive are likely to reveal this to us in non-catastrophic ways, and thus we are fine because we’ll pass through the sweet spot on the way to more advanced AI systems. Or not, but the point is that continuity complicates the point you were making.
Thanks for this, I for one hadn’t thought about this variable much and am convinced now that it is one of the more important variables.
--I think acausal trade stuff means that even if all the AIs on Earth are homogenous, the strategic situation may end up being as if they were heterogenous, at least in some ways. I’m not sure, will need to think more about this.
--You talk about this being possible even for gradual, continuous takeoff, yet you also talk about “The first advanced AI system” as if there is a sharp cutoff between advanced and non-advanced AI. I think this isn’t a problem for your overall point, but I’m not sure. For alignment (your point 4) I think this isn’t a problem, because you can just rephrase it as “As we gradually transition from non-advanced systems to advanced systems, it is important that our systems be aligned before we near the end of the transition, and more important the closer we get to the end. Because as our systems become more advanced, their alignment properties become more locked-in.” For deception, I’m less sure. If systems get more advanced gradually and continuously then maybe we can hope there is a “sordid stumble sweet spot” where systems that are deceptive are likely to reveal this to us in non-catastrophic ways, and thus we are fine because we’ll pass through the sweet spot on the way to more advanced AI systems. Or not, but the point is that continuity complicates the point you were making.