Definitely, I should have been clearer in my OP: it is not a universal observation but an observation of the patterns of some relatively high-profile people in area.
Many certainly do seem to take the approach (which I agree with) of favoring against any investment in AI industry, using it only where one believes there is a very large utility gain in excess of whatever investment one is giving to AI developments (which is the approach I would advocate, whether for AI or using fossil fuels). But some seem to take the attitude of it not mattering (since it is only at the aggregate level of the industry that poses the most serious risks they identify) and actually take positions that actively, publicly, promote AI developments, usage and dependance. If one is an altruist (if you are purely self-interested there are different calculations of the risk), this seems to me to be fallacious and counter productive in the case of fossil fuels as well as AI.
If one is an altruist (if you are purely self-interested there are different calculations of the risk), this seems to me to be fallacious and counter productive in the case of fossil fuels as well as AI.
Does your altruism take into account benefits from economic growth? I think the people whose behavior you’re puzzled by have an attitude more like “AI and fossil fuels are great except for the potential for catastrophic risk, which demands targeted mitigations” (as contrasted to “AI and fossil fuels are Bad in general; only use if truly unavoidable”).
Does your altruism take into account benefits from economic growth?
Yes, I explicitly described both in the case of fossil fuels and AI. Because the results of expanded development are catastrophic, the default leans heavily in one direction.
If the rent is large enough to be in excess of whatever the investment is, then it is reasonably justified. However, the example I gave was, e.g. using AI to produce more fiction. It seems pretty clear to me that the utility gained by having AIs produce fiction instead is far less than any reasonable consideration of the harm of investing in AI if you believe there is a sizable risk of AI killing all humans.
I think the idea is that we shouldn’t have to indiscriminately divest from the whole industry when we could “just” build fiction-writing AIs and not catastrophically risky AIs (as you suggested about Gwern’s company).
The problem with these catastrophic risk is it manifests in the aggregate. Any use of fossil fuels increases the aggregate risk directly. With AI the increase is less direct but no less real (more money in expands the industry, and makes development look more appealing, encouraging further buy-in). If you had global centralized control of all fossil fuel and the entire AI industry, then yes, targeted mitigations would be sufficient since you could allocate everything so the risk was never catastrophic.
But the world we live in isn’t that, any use of fossil fuels contributes to the risk and any divestment/decrease lessons the risk. In some cases, divesting or boycotting would be so costly as to not be worth it anyways, but the more seriously you view the risk the marginal you should consider the benefits.
Given we do not have centralized control, increasing investment and use (whether fossil fuels or AI) increases the risk.
we could “just” build fiction-writing AIs and not catastrophically risky AIs (as you suggested about Gwern’s company).
I wasn’t suggesting gwern’s company was bad or not, I explicitly said I wasn’t commenting on his business—I don’t know enough to exactly do an analysis and, in any case, it would feel inappropriate even if I had a real position. My comment was particularly on the promotion of AI in general use, encouraging participation in AI-generated fiction and other use cases where I cannot see how someone who believes in a substantial catastrophic risk risk from the AI industry’s development would view that benefit as outweighing the risk.
An important fact is that many of us are not, or have divested from investments made in rosier times.
Definitely, I should have been clearer in my OP: it is not a universal observation but an observation of the patterns of some relatively high-profile people in area.
Many certainly do seem to take the approach (which I agree with) of favoring against any investment in AI industry, using it only where one believes there is a very large utility gain in excess of whatever investment one is giving to AI developments (which is the approach I would advocate, whether for AI or using fossil fuels). But some seem to take the attitude of it not mattering (since it is only at the aggregate level of the industry that poses the most serious risks they identify) and actually take positions that actively, publicly, promote AI developments, usage and dependance. If one is an altruist (if you are purely self-interested there are different calculations of the risk), this seems to me to be fallacious and counter productive in the case of fossil fuels as well as AI.
Does your altruism take into account benefits from economic growth? I think the people whose behavior you’re puzzled by have an attitude more like “AI and fossil fuels are great except for the potential for catastrophic risk, which demands targeted mitigations” (as contrasted to “AI and fossil fuels are Bad in general; only use if truly unavoidable”).
Yes, I explicitly described both in the case of fossil fuels and AI. Because the results of expanded development are catastrophic, the default leans heavily in one direction.
If the rent is large enough to be in excess of whatever the investment is, then it is reasonably justified. However, the example I gave was, e.g. using AI to produce more fiction. It seems pretty clear to me that the utility gained by having AIs produce fiction instead is far less than any reasonable consideration of the harm of investing in AI if you believe there is a sizable risk of AI killing all humans.
I think the idea is that we shouldn’t have to indiscriminately divest from the whole industry when we could “just” build fiction-writing AIs and not catastrophically risky AIs (as you suggested about Gwern’s company).
The problem with these catastrophic risk is it manifests in the aggregate. Any use of fossil fuels increases the aggregate risk directly. With AI the increase is less direct but no less real (more money in expands the industry, and makes development look more appealing, encouraging further buy-in). If you had global centralized control of all fossil fuel and the entire AI industry, then yes, targeted mitigations would be sufficient since you could allocate everything so the risk was never catastrophic.
But the world we live in isn’t that, any use of fossil fuels contributes to the risk and any divestment/decrease lessons the risk. In some cases, divesting or boycotting would be so costly as to not be worth it anyways, but the more seriously you view the risk the marginal you should consider the benefits.
Given we do not have centralized control, increasing investment and use (whether fossil fuels or AI) increases the risk.
I wasn’t suggesting gwern’s company was bad or not, I explicitly said I wasn’t commenting on his business—I don’t know enough to exactly do an analysis and, in any case, it would feel inappropriate even if I had a real position. My comment was particularly on the promotion of AI in general use, encouraging participation in AI-generated fiction and other use cases where I cannot see how someone who believes in a substantial catastrophic risk risk from the AI industry’s development would view that benefit as outweighing the risk.