So the base rate for the kind of transition entailed by a fast or medium takeoff scenario, in terms of the speed and magnitude of the postulated change, is zero: it lacks precedent outside myth and religion.
Bostrom says this, but seems to use it only as a nice way of introducing a position to argue against—that a medium or fast transition is unlikely—and not so much as evidence. Yet it seems to me that this is potentially substantial evidence, and deserves attention.
If we want to combine this outside view evidence with other object level arguments, one way to do it is to ask how often there are explosive transitions after there have seemed such good reasons for explosive transitions. That is, if this is the first time in history that optimization power has seemed set to substantially increase while recalcitrance has seemed likely to decrease so much, then the previous evidence that explosive transitions have not happened is perhaps irrelevant, and we might expect a large burst of progress. However if humans often look at the future and see declining recalcitrance and increasing optimization power (to the extent that Bostrom sees these things), and yet there haven’t been such transitions, then we should hesitate in accepting that we will get one this time.
Bostrom says this, but seems to use it only as a nice way of introducing a position to argue against—that a medium or fast transition is unlikely—and not so much as evidence. Yet it seems to me that this is potentially substantial evidence, and deserves attention.
If we want to combine this outside view evidence with other object level arguments, one way to do it is to ask how often there are explosive transitions after there have seemed such good reasons for explosive transitions. That is, if this is the first time in history that optimization power has seemed set to substantially increase while recalcitrance has seemed likely to decrease so much, then the previous evidence that explosive transitions have not happened is perhaps irrelevant, and we might expect a large burst of progress. However if humans often look at the future and see declining recalcitrance and increasing optimization power (to the extent that Bostrom sees these things), and yet there haven’t been such transitions, then we should hesitate in accepting that we will get one this time.
Have things looked this way before?