Thanks for reading my post. I realise that misalignment is not just a financial cost, it could have permanent societal and existential impacts. The non-financial risk are more important than the financial risks, I agree with you.
My viewpoint is very much about the financial impact of misalignment, because this is a dimension that is tangible and one policymakers can control. It is probably the only stick they have to hand. And I’m sure my response has been conditioned on my previous career in banking where I saw capital-adequacy rules created/added as a result of risk-taking that was getting out of control, due to unconstrained competition and animal spirits. I think that is very much alive in the leading AI labs. Without alternative rules, I also don’t think they have a choice but to continue the arms race.
I think your point is valid in that AI labs don’t provide finance to the economy, so why should they need to hold it in reserve? Would a regulatory regime where they can be fined for providing misaligned AI into the economy (AI being a commodity) be better? This is how other utility companies tend to be dealt with.
My thought here is that, if we want markets to operate as efficiently as they can so that human-aligned AI can have maximal positive societal and economic impact, then a capital adequacy-style regime would be more efficient than a regime where the Labs hold capital in anticipation of fines. They would likely not be incentivised to hold such fine-covering provisions in anticipation (such behaviour would lower their Return on Equity). And I think recognising that the Labs monetise the intelligence they provide through revenues to corporates, rather than being commissioned by the state to provide it, gives scope to think of AI as being unique as a utility.
One final thought is that the regulation could take place at the corporate consumer level instead. I think it would lead to a similar end goal. If AI-using companies were to bear the misalignment risk (a financial cost from regulators) of the AI models that they are using, then the AI labs would compete on Intelligence, Alignment and Cost.
Sorry for the long answer, thanks for an excellent question.
Hi Mordechai,
Thanks for reading my post. I realise that misalignment is not just a financial cost, it could have permanent societal and existential impacts. The non-financial risk are more important than the financial risks, I agree with you.
My viewpoint is very much about the financial impact of misalignment, because this is a dimension that is tangible and one policymakers can control. It is probably the only stick they have to hand. And I’m sure my response has been conditioned on my previous career in banking where I saw capital-adequacy rules created/added as a result of risk-taking that was getting out of control, due to unconstrained competition and animal spirits. I think that is very much alive in the leading AI labs. Without alternative rules, I also don’t think they have a choice but to continue the arms race.
I think your point is valid in that AI labs don’t provide finance to the economy, so why should they need to hold it in reserve? Would a regulatory regime where they can be fined for providing misaligned AI into the economy (AI being a commodity) be better? This is how other utility companies tend to be dealt with.
My thought here is that, if we want markets to operate as efficiently as they can so that human-aligned AI can have maximal positive societal and economic impact, then a capital adequacy-style regime would be more efficient than a regime where the Labs hold capital in anticipation of fines. They would likely not be incentivised to hold such fine-covering provisions in anticipation (such behaviour would lower their Return on Equity). And I think recognising that the Labs monetise the intelligence they provide through revenues to corporates, rather than being commissioned by the state to provide it, gives scope to think of AI as being unique as a utility.
One final thought is that the regulation could take place at the corporate consumer level instead. I think it would lead to a similar end goal. If AI-using companies were to bear the misalignment risk (a financial cost from regulators) of the AI models that they are using, then the AI labs would compete on Intelligence, Alignment and Cost.
Sorry for the long answer, thanks for an excellent question.
Peter