The effect size of divestment just seems way too small to matter to me—AI is the current biggest frenzy among investors, data centres are holding up US GDP, it’s making trillion dollar companies, some of the fastest growing consumer products ever, etc. And there’s substantial productivity advantages to people using AI. I don’t feel like a plan involving divestment and boycotting will work and I would be sad to see safety orgs following it rather than things I thought could meaningfully reduce x risk.
I don’t feel like a plan involving divestment and boycotting will work and I would be sad to see safety orgs following it rather than things I thought could meaningfully reduce x risk.
They are not either-or, rather obviously. It is much easier to get regulation passed and get firms to sign on to agreements when something is unprofitable and unpopular. It is very hard to get them to sign on when it is profitable.
I am not even saying they should be focusing on divestment and boycotting campaigns, I am saying the engaging in investment and promotion campaigns seems rather non-sensical.
Like, I think it is fine that a lot of climate activists don’t do the absolute utmost to eliminate their fossil fuel usage and exposure to it, but if they went around promoting contests for like ‘best steak made on a coal-fired grill’ and various uses of coal power, I would look at them like wtf are you doing.
It is much easier to get regulation passed and get firms to sign on to agreements when something is unprofitable and unpopular. It is very hard to get them to sign on when it is profitable
About the investment question specifically: The point is that it seems very hard to make much progress to making AI unprofitable from just not investing. I haven’t done a BOTEC yet so I’m open to be convinced by a BOTEC.
There are various advantages for someone who cares about safety to invest in major AI companies, even if they are fully altruistic. The biggest is that it can enable you to buy compute right before and during the Intelligence Explosion
To me, the steelman best reason not invest is not like “making firms unprofitable” but rather psychological reasons to not secretly hope for your portfolio to go up. You can mostly mitigate this by having investments to be secure against an AI crash.
The point is that it seems very hard to make much progress to making AI unprofitable from just not investing.
It is a gradient. More profitable and popular the industry is, the more difficult it is to regulate. The less profitable and popular it is, the easier it is to regulate. Also, on an adjacent point, PR wise for advocates against something, being seen to publicly promote the thing you are saying could kill all humans doesn’t look great.
I agree, spending a few thousand less in AI contests is unlikely to move them from the black to the red (most are technically in the red atm).
To me, the steelman best reason not invest is not like “making firms unprofitable” but rather psychological reasons to not secretly hope for your portfolio to go up.
There is more to it. If I understand what you are saying, betting on the death of everyone is dumb even if you are right, by any sensible consideration of utility. Having $10 and being alive is infinitely preferable to having 1,000,000,000 in an account and everyone being dead. “Everyone dies” is not a scenerio it makes sense to expose yourself yourself to or hedge against regardless of how likely.
It is a gradient. More profitable and popular the industry is, the more difficult it is to regulate. The less profitable and popular it is, the easier it is to regulate.
I agree its a gradient, but the point I was making was that I think the movement along the gradient from not investing is minimal relative to other factors.
If I understand what you are saying, betting on the death of everyone is dumb even if you are right, by any sensible consideration of utility.
I was a bit unclear in my comment above, but the potential counterargument was that “If you invest too much on AI and especially if you’re exposed to an AI crash, you may end up doing motivated reasoning against a pause because you want your portfolio to go up”. To me its worth engaging with this argument, but as I said, there are ways to circumvent this like not being overly exposed to crash.
Just wrote this to clarify a few points because I was a bit unclear.
I agree its a gradient, but the point I was making was that I think the movement along the gradient seems to be minimal relative to other factors.
I agree it is relatively small, but when if you think the risk in question is one of a substantial chance of everyone on earth dying, even extremly tiny pushes towards the end times seem rather unappealing. Also, I think it is an especially meaningful calculation for the people who are relatively ‘high profile’ around the AI risks, since there are knock-on effects with their public actions. Any promotion, purchase or investment to some extent promotes more buy-in by making the industry look a little better in public or the bottom line, but when it is publicly signaled by someone with some perceived authority there are additional knock-on effects.
I was a bit unclear in my comment above and will edit for clarity, but the potential counterargument was that “If you invest too much on AI and especially if you’re exposed to an AI crash, you may end up doing motivated reasoning against a pause because you want your portfolio to go up”.
Got it, yea, I wasn’t discussing the motivated reasoning side just the actual effects, which when it comes to potentially catastrophic risks like AI and climate change, the risks are present in the aggregate so anything that has a promotes a drivers of the risk (whether CO2 emissions or AI development) is inseperatable from the larger system-wide movement on those drivers.
If one existed in a world where the larger, global system-wide movements of the risk drivers could be reasonably controlled, then use can be managed on a system wide level so the individual drivers never rise to the level of having catastrophic risks. However, in our present world there is no centralized global control over carbon emissions or AI development and as such imagining whether or not in such a world your use would be within tolerances isn’t meaningful.
The effect size of divestment just seems way too small to matter to me—AI is the current biggest frenzy among investors, data centres are holding up US GDP, it’s making trillion dollar companies, some of the fastest growing consumer products ever, etc. And there’s substantial productivity advantages to people using AI. I don’t feel like a plan involving divestment and boycotting will work and I would be sad to see safety orgs following it rather than things I thought could meaningfully reduce x risk.
They are not either-or, rather obviously. It is much easier to get regulation passed and get firms to sign on to agreements when something is unprofitable and unpopular. It is very hard to get them to sign on when it is profitable.
I am not even saying they should be focusing on divestment and boycotting campaigns, I am saying the engaging in investment and promotion campaigns seems rather non-sensical.
Like, I think it is fine that a lot of climate activists don’t do the absolute utmost to eliminate their fossil fuel usage and exposure to it, but if they went around promoting contests for like ‘best steak made on a coal-fired grill’ and various uses of coal power, I would look at them like wtf are you doing.
About the investment question specifically: The point is that it seems very hard to make much progress to making AI unprofitable from just not investing. I haven’t done a BOTEC yet so I’m open to be convinced by a BOTEC.
There are various advantages for someone who cares about safety to invest in major AI companies, even if they are fully altruistic. The biggest is that it can enable you to buy compute right before and during the Intelligence Explosion
To me, the steelman best reason not invest is not like “making firms unprofitable” but rather psychological reasons to not secretly hope for your portfolio to go up. You can mostly mitigate this by having investments to be secure against an AI crash.
It is a gradient. More profitable and popular the industry is, the more difficult it is to regulate. The less profitable and popular it is, the easier it is to regulate. Also, on an adjacent point, PR wise for advocates against something, being seen to publicly promote the thing you are saying could kill all humans doesn’t look great.
I agree, spending a few thousand less in AI contests is unlikely to move them from the black to the red (most are technically in the red atm).
There is more to it. If I understand what you are saying, betting on the death of everyone is dumb even if you are right, by any sensible consideration of utility. Having $10 and being alive is infinitely preferable to having 1,000,000,000 in an account and everyone being dead. “Everyone dies” is not a scenerio it makes sense to expose yourself yourself to or hedge against regardless of how likely.
I agree its a gradient, but the point I was making was that I think the movement along the gradient from not investing is minimal relative to other factors.
I was a bit unclear in my comment above, but the potential counterargument was that “If you invest too much on AI and especially if you’re exposed to an AI crash, you may end up doing motivated reasoning against a pause because you want your portfolio to go up”. To me its worth engaging with this argument, but as I said, there are ways to circumvent this like not being overly exposed to crash.
Just wrote this to clarify a few points because I was a bit unclear.
I agree it is relatively small, but when if you think the risk in question is one of a substantial chance of everyone on earth dying, even extremly tiny pushes towards the end times seem rather unappealing. Also, I think it is an especially meaningful calculation for the people who are relatively ‘high profile’ around the AI risks, since there are knock-on effects with their public actions. Any promotion, purchase or investment to some extent promotes more buy-in by making the industry look a little better in public or the bottom line, but when it is publicly signaled by someone with some perceived authority there are additional knock-on effects.
Got it, yea, I wasn’t discussing the motivated reasoning side just the actual effects, which when it comes to potentially catastrophic risks like AI and climate change, the risks are present in the aggregate so anything that has a promotes a drivers of the risk (whether CO2 emissions or AI development) is inseperatable from the larger system-wide movement on those drivers.
If one existed in a world where the larger, global system-wide movements of the risk drivers could be reasonably controlled, then use can be managed on a system wide level so the individual drivers never rise to the level of having catastrophic risks. However, in our present world there is no centralized global control over carbon emissions or AI development and as such imagining whether or not in such a world your use would be within tolerances isn’t meaningful.