I see—yeah I agree that is different from the main thrust of the piece. I think I address this in footnote 9 a bit, where I call it the “humans will keep becoming more powerful” outside view.
My main objection to that would be that there’s not a very strong correlation between humans getting what they want and the total welfare of the world (Agricultural Revolution, factory farming), so the fact that that outside view is strong doesn’t seem very reassuring. (How do you feel about the question of whether the total welfare of the world today is negative because of factory farming?)
Actually, no I changed my mind. It’s still captured by the core conceptual point of the piece, there’s just no good vivid examples of it like the chicken example.
It’s still completely expected a priori that the dominant class of agent at a given stage of runaway Darwinian competition would solve most of their problems and be very powerful. This would be the case even if they were to then proceed to get outcompeted. So if someone hasn’t integrated this conceptual point, it should be an update downwards on how much evidence that outside view is for them.
There’s also the classic meta-epistemic debate here, on how to weigh empirical outside views against gears-level models (even if the “gears-level model” here is something almost tautological).
I see—yeah I agree that is different from the main thrust of the piece. I think I address this in footnote 9 a bit, where I call it the “humans will keep becoming more powerful” outside view.
My main objection to that would be that there’s not a very strong correlation between humans getting what they want and the total welfare of the world (Agricultural Revolution, factory farming), so the fact that that outside view is strong doesn’t seem very reassuring.
(How do you feel about the question of whether the total welfare of the world today is negative because of factory farming?)
Actually, no I changed my mind. It’s still captured by the core conceptual point of the piece, there’s just no good vivid examples of it like the chicken example.
It’s still completely expected a priori that the dominant class of agent at a given stage of runaway Darwinian competition would solve most of their problems and be very powerful. This would be the case even if they were to then proceed to get outcompeted. So if someone hasn’t integrated this conceptual point, it should be an update downwards on how much evidence that outside view is for them.
There’s also the classic meta-epistemic debate here, on how to weigh empirical outside views against gears-level models (even if the “gears-level model” here is something almost tautological).