I think the idea is that we shouldn’t have to indiscriminately divest from the whole industry when we could “just” build fiction-writing AIs and not catastrophically risky AIs (as you suggested about Gwern’s company).
The problem with these catastrophic risk is it manifests in the aggregate. Any use of fossil fuels increases the aggregate risk directly. With AI the increase is less direct but no less real (more money in expands the industry, and makes development look more appealing, encouraging further buy-in). If you had global centralized control of all fossil fuel and the entire AI industry, then yes, targeted mitigations would be sufficient since you could allocate everything so the risk was never catastrophic.
But the world we live in isn’t that, any use of fossil fuels contributes to the risk and any divestment/decrease lessons the risk. In some cases, divesting or boycotting would be so costly as to not be worth it anyways, but the more seriously you view the risk the marginal you should consider the benefits.
Given we do not have centralized control, increasing investment and use (whether fossil fuels or AI) increases the risk.
we could “just” build fiction-writing AIs and not catastrophically risky AIs (as you suggested about Gwern’s company).
I wasn’t suggesting gwern’s company was bad or not, I explicitly said I wasn’t commenting on his business—I don’t know enough to exactly do an analysis and, in any case, it would feel inappropriate even if I had a real position. My comment was particularly on the promotion of AI in general use, encouraging participation in AI-generated fiction and other use cases where I cannot see how someone who believes in a substantial catastrophic risk risk from the AI industry’s development would view that benefit as outweighing the risk.
I think the idea is that we shouldn’t have to indiscriminately divest from the whole industry when we could “just” build fiction-writing AIs and not catastrophically risky AIs (as you suggested about Gwern’s company).
The problem with these catastrophic risk is it manifests in the aggregate. Any use of fossil fuels increases the aggregate risk directly. With AI the increase is less direct but no less real (more money in expands the industry, and makes development look more appealing, encouraging further buy-in). If you had global centralized control of all fossil fuel and the entire AI industry, then yes, targeted mitigations would be sufficient since you could allocate everything so the risk was never catastrophic.
But the world we live in isn’t that, any use of fossil fuels contributes to the risk and any divestment/decrease lessons the risk. In some cases, divesting or boycotting would be so costly as to not be worth it anyways, but the more seriously you view the risk the marginal you should consider the benefits.
Given we do not have centralized control, increasing investment and use (whether fossil fuels or AI) increases the risk.
I wasn’t suggesting gwern’s company was bad or not, I explicitly said I wasn’t commenting on his business—I don’t know enough to exactly do an analysis and, in any case, it would feel inappropriate even if I had a real position. My comment was particularly on the promotion of AI in general use, encouraging participation in AI-generated fiction and other use cases where I cannot see how someone who believes in a substantial catastrophic risk risk from the AI industry’s development would view that benefit as outweighing the risk.