I’m encouraging crisp distinctions between having a high standard of evidence, an explicit demonstration of a specific limit to our ability to forecast, and an unsubstantiated declaration that an entire broad topic is entirely beyond forecasting.
In the case of AGI, this would mean distinguishing between:
“I’d need a stronger argument and evidence for predicting AGI doom to update my credence any further.”
“Even an AGI can’t predict more than n pinball bounces out into the future given atomic-resolution data from only one moment in time.”
“Nobody can predict what will happen with AGI, it’s a case of Knightian uncertainty and simply incalculable! There are just too many possibilities!”
The first two cases are fine, the third one I think is an invalid form of argument.
I’m encouraging crisp distinctions between having a high standard of evidence, an explicit demonstration of a specific limit to our ability to forecast, and an unsubstantiated declaration that an entire broad topic is entirely beyond forecasting.
In the case of AGI, this would mean distinguishing between:
“I’d need a stronger argument and evidence for predicting AGI doom to update my credence any further.”
“Even an AGI can’t predict more than n pinball bounces out into the future given atomic-resolution data from only one moment in time.”
“Nobody can predict what will happen with AGI, it’s a case of Knightian uncertainty and simply incalculable! There are just too many possibilities!”
The first two cases are fine, the third one I think is an invalid form of argument.