hmm… interesting. I disagree, at least partly. It seems to me that all the examples mentioned are characterized by consisting of ultimately distinct, granular, components that are then assembled to form a complex system. The examples of failures rely on the assumption that this system can then be, so to speak, attacked from all angles. The Germans can and will find any point at which the French defenses are too weak and therefore every point has to be sufficently guarded. ASI seems to be fundamentally different, (although I am no AI researcher, so take this with a grain of salt) it seems as if, rather than consisting of multiple, independent variables each of which could fail to the ruin of all, if we were to understand the inner workings of the model so that we can assert with a high degree of confidence that it is eg. not being deceptive, we could probably also asert that it is unlikely to cause extinction. The point being that this only depends on one variable rather than many.
hmm… interesting. I disagree, at least partly. It seems to me that all the examples mentioned are characterized by consisting of ultimately distinct, granular, components that are then assembled to form a complex system. The examples of failures rely on the assumption that this system can then be, so to speak, attacked from all angles. The Germans can and will find any point at which the French defenses are too weak and therefore every point has to be sufficently guarded. ASI seems to be fundamentally different, (although I am no AI researcher, so take this with a grain of salt) it seems as if, rather than consisting of multiple, independent variables each of which could fail to the ruin of all, if we were to understand the inner workings of the model so that we can assert with a high degree of confidence that it is eg. not being deceptive, we could probably also asert that it is unlikely to cause extinction. The point being that this only depends on one variable rather than many.