Sort of an interesting idea, but seems like it would need to engage with a few core issues for it to be taken seriously. (Disclaimer that I don’t really understand how this works with banks.)
First, banks are regulated to keep capital reserves because they pose direct financial risks, right? But those who worry about safety and misalignment risks, at least in forums like Less Wrong, are not primarily concerned about financial risks. They are more concerned about extinction risks and impacts on livelihood.
Regardless, AI companies certainly do pose financial risks, either through causing catastrophic life loss, going bankrupt, or putting people out of work. But those are all second-order, indirect effects in the sense they are not actively lending to other parties and exposing the economy to direct financial risk.
It’s not clear to me how you would justify regulating AI companies in the same way as banks, given these fundamental differences.
Thanks for reading my post. I realise that misalignment is not just a financial cost, it could have permanent societal and existential impacts. The non-financial risk are more important than the financial risks, I agree with you.
My viewpoint is very much about the financial impact of misalignment, because this is a dimension that is tangible and one policymakers can control. It is probably the only stick they have to hand. And I’m sure my response has been conditioned on my previous career in banking where I saw capital-adequacy rules created/added as a result of risk-taking that was getting out of control, due to unconstrained competition and animal spirits. I think that is very much alive in the leading AI labs. Without alternative rules, I also don’t think they have a choice but to continue the arms race.
I think your point is valid in that AI labs don’t provide finance to the economy, so why should they need to hold it in reserve? Would a regulatory regime where they can be fined for providing misaligned AI into the economy (AI being a commodity) be better? This is how other utility companies tend to be dealt with.
My thought here is that, if we want markets to operate as efficiently as they can so that human-aligned AI can have maximal positive societal and economic impact, then a capital adequacy-style regime would be more efficient than a regime where the Labs hold capital in anticipation of fines. They would likely not be incentivised to hold such fine-covering provisions in anticipation (such behaviour would lower their Return on Equity). And I think recognising that the Labs monetise the intelligence they provide through revenues to corporates, rather than being commissioned by the state to provide it, gives scope to think of AI as being unique as a utility.
One final thought is that the regulation could take place at the corporate consumer level instead. I think it would lead to a similar end goal. If AI-using companies were to bear the misalignment risk (a financial cost from regulators) of the AI models that they are using, then the AI labs would compete on Intelligence, Alignment and Cost.
Sorry for the long answer, thanks for an excellent question.
Sort of an interesting idea, but seems like it would need to engage with a few core issues for it to be taken seriously. (Disclaimer that I don’t really understand how this works with banks.)
First, banks are regulated to keep capital reserves because they pose direct financial risks, right? But those who worry about safety and misalignment risks, at least in forums like Less Wrong, are not primarily concerned about financial risks. They are more concerned about extinction risks and impacts on livelihood.
Regardless, AI companies certainly do pose financial risks, either through causing catastrophic life loss, going bankrupt, or putting people out of work. But those are all second-order, indirect effects in the sense they are not actively lending to other parties and exposing the economy to direct financial risk.
It’s not clear to me how you would justify regulating AI companies in the same way as banks, given these fundamental differences.
Hi Mordechai,
Thanks for reading my post. I realise that misalignment is not just a financial cost, it could have permanent societal and existential impacts. The non-financial risk are more important than the financial risks, I agree with you.
My viewpoint is very much about the financial impact of misalignment, because this is a dimension that is tangible and one policymakers can control. It is probably the only stick they have to hand. And I’m sure my response has been conditioned on my previous career in banking where I saw capital-adequacy rules created/added as a result of risk-taking that was getting out of control, due to unconstrained competition and animal spirits. I think that is very much alive in the leading AI labs. Without alternative rules, I also don’t think they have a choice but to continue the arms race.
I think your point is valid in that AI labs don’t provide finance to the economy, so why should they need to hold it in reserve? Would a regulatory regime where they can be fined for providing misaligned AI into the economy (AI being a commodity) be better? This is how other utility companies tend to be dealt with.
My thought here is that, if we want markets to operate as efficiently as they can so that human-aligned AI can have maximal positive societal and economic impact, then a capital adequacy-style regime would be more efficient than a regime where the Labs hold capital in anticipation of fines. They would likely not be incentivised to hold such fine-covering provisions in anticipation (such behaviour would lower their Return on Equity). And I think recognising that the Labs monetise the intelligence they provide through revenues to corporates, rather than being commissioned by the state to provide it, gives scope to think of AI as being unique as a utility.
One final thought is that the regulation could take place at the corporate consumer level instead. I think it would lead to a similar end goal. If AI-using companies were to bear the misalignment risk (a financial cost from regulators) of the AI models that they are using, then the AI labs would compete on Intelligence, Alignment and Cost.
Sorry for the long answer, thanks for an excellent question.
Peter