Ball’s proposal primarily focuses on how to approach the question of civil liability for the customer misuse of models. His proposal outlines a “marketplace” of hybrid public-private standards setting organizations, which must be licensed by the federal[1] government but are not government agencies themselves. Meeting these standards would not necessarily be mandatory for labs seeking to deploy models, but labs who opted in to meeting at least one such standard would be shielded from tort liability as a result of customer misuse of the models once deployed. In his own words,
Private bodies, authorized and overseen by government, provide certifications to developers of frontier AI systems on an opt-in basis. In exchange for opting in, frontier AI firms receive protections from tort liability for customer misuse of their models
In one of his Substack posts, Ball did mention FINRA as an example of a successful non-governmental standards setting organization. However, FINRA has what is essentially a monopoly within its industry and as such is not a great example of a regulatory marketplace.
When I asked him for an example of a successful ‘regulatory marketplace’ Ball pointed me to the London Stock Exchange’s Alternative Investment Market. The LSE’s AIM only allows companies to list if they have engaged a “Nominated Advisor” (NOMAD), these are private entities who compete for business in much the same way Ball envisions standard setters competing to certify labs. A NOMAD must confirm that the AIM issuer meets admission rules, provide ongoing oversight, and notify the LSE of breaches.
While it’s unlikely to be a perfect 1⁄1 comparison, when reading the following proposal you can keep the NOMAD system in mind as a “market comp”.
With no further ado, let’s get into the substance of his proposal.
The Proposed Framework
1. A legislature authorizes a government commission to license private AI standards-setting and regulatory organizations. These licenses are granted to organizations with technical and legal credibility, and with demonstrated independence from industry.
2. AI developers, in turn, can opt in to receiving certifications from those private bodies. The certifications verify that an AI developer meets technical standards for security and safety published by the private body. The private body periodically (once per year) conducts audits of each developer to ensure that they are, in fact, meeting the standards.
3. In exchange for being certified, AI developers receive safe harbor from all tort liability related to misuse by others that results in tortious harm.
4. The authorizing government body periodically audits and re-licenses each private regulatory body.
5. If an AI developer behaves in a way that would legally qualify as reckless, deceitful, or grossly negligent, the safe harbor does not apply.
6. The private governance body can revoke an AI developer’s safe harbor protections for non-compliance.
7. The authorizing government body has the power to revoke a private regulator’s license if they are found to have behaved negligently (for example, ignoring instances of developer non-compliance).
Ball does leave room for flexibility wherein developers may have some sort of liability exposure in extreme cases;
The safe harbor could be modified such that it only applies to harms below a certain objective count, for example incidents involving property damage of less than $500 million or deaths below a threshold. Above that threshold, the protection could shift to a rebuttable presumption of reasonable care on the part of the developer, or protections could be eliminated altogether
He provides the following guidance on the structure and power of the governmental agency which licenses the private standards setting organization:
The authorizing government body can (and ideally, should) authorize multiple private governance organizations.
[...]
The priority of all parties—the private governance organizations and the authorizing government body alike—would be on the mitigation of plausible tort-related harm, most notably loss of property and physical harm. All other considerations should be explicitly disallowed by any statutory implementation of this proposal, unless the proposal is modified to provide liability protections in analogous legal domains.
[...]
To mitigate against mission creep and political interference, the authorizing government body’s powers should remain both significant and narrow. It should retain an absolute right to certify and decertify private governance organizations, and to conduct relevant investigations into the conduct of the organizations and the firms they certify. But the authorizing government body should not, for example, be granted broad rulemaking authority.
[...]
The authorizing government body has the power to revoke a private governance organization’s license in the event that negligence or misfeasance is discovered. Doing so would eliminate tort liability protections for all companies covered by that private governance organization
Ball’s intention is that the private regulatory agencies themselves would be subject to minimal government regulations, with the idea that marketplace dynamics would function as a better constraining function (more on that in the next section):
The proposal envisions minimal regulations or restrictions on how these private bodies would go about their tasks, beyond the oversight (both ad hoc and scheduled, as mandated by the periodic license reviews) by the authorizing government body.
Ball is careful to clarify that he sees this as a regulatory approach which is inherently transitory and advocates that at a later date when we have more data on the risks of real world use, a new approach can be adopted:
After the transition to advanced AI has gained hold, and some measure of stability has been found, an altogether different governance regime may be called for than the one designed to govern the frontier. This new regime may be more expansive or considerably lighter touch.
Ball also separates harms resulting from a “loss of control” and makes it clear developers would not be protected by their safe harbor in such a situation:
More seriously, few would dispute that if an OpenAI model, in testing for autonomy capabilities, were to exfiltrate itself from its secure testing environment and begin defrauding unsuspecting people on the internet (for example), OpenAI could and likely should face tort liability for this.
[...]
Tort liability exposure is also preserved for harms stemming from first-party use of frontier models (e.g. an internal deployment of a model that exfiltrates itself from its secure environment and commits tortious acts against third parties)
Critiques Towards Alternatives
Ball’s motivation on providing developers a path to a safe harbor against tort liability specifically from misuse, can be said to arise from a general skepticism around the application of tort liability for “platform technologies” providers:
If I use the phone system to defraud thousands of people, the victims have no recourse to sue the phone company. An electricity company can be held liable (often in strict liability) if its transmission lines fail due to improper maintenance and cause a fire; but it would be incoherent to speak of suing the power company because one of their customers misused electricity.
He also spends much of the paper criticizing other commonly proposed regulatory alternatives.
Ball expresses skepticism about international governance. He prefers that even in the event the US engages in such discussions it does so from a position of having already established a robust domestic regulatory framework:
Many academics and industry leaders [...] have advocated for some kind of global governance system to oversee advanced AI. While these esteemed figures may be correct that AI systems may one day reach a level of capability so extreme as to require such an arrangement, there is currently no path to achieving it.
[...]
There may be narrow windows of opportunity in the future for international AI governance cooperation, and those windows are most likely to be seized by the United States if the country already has well developed, demonstrably functional governance practices in place.
Ball is also lukewarm towards compute governance. He acknowledges that it was effective at denying China an early advantage in the space, but also thinks in the long term it will prove counterproductive:
even under the best of circumstances, compute export controls have significant tradeoffs. The tradeoffs may have been worth it, and the policy as a whole may have been wise. But any additional compute governance mechanisms—such as the on-chip governance discussed briefly above—are likely to only add to the incentive for adversary countries to develop an alternative computing ecosystem. Once that ecosystem is in place—and this is an inevitable outcome— compute governance as a foreign policy tool is no longer viable.
And he is quite firmly against any sort of domestic compute governance, in particular anything involving putting backdoors on chips:
it would be senseless, of course, to try to limit or deny compute to the domestic economy
[...]
The significant and unshakeable tradeoff is that this on-chip governance mechanism would amount to, at the very least, a government-mandated security vulnerability in all frontier AI computing hardware, and at worst, a backdoor giving government the option to surveil all AI computing performed using those chips.
Pros of the Regulatory Marketplace
Safe harbor is probably the most boilerplate part of the proposal. What is novel about Ball’s framework is the “regulatory marketplace” approach, and as such I will focus on the benefits he sees arising from such a framework.
One of the primary motivators of the regulatory marketplace structure is the need for “standard setters” who are capable of keeping pace with an industry undergoing rapid technological development. Ball believes that government entities are unlikely to be able to stay up to date to the extent required to do their job, unlike private entities incentivized by a competitive environment.[2]
Further, he expresses concern about funding for any government agency becoming a political football at a later date, which I think is part of why he prefers to keep the role of such an agency as minimal as possible.
Of course in reading this proposal one of the first questions that any concerned party will ask is about regulatory capture. Ball discusses this in the context of efforts not only from labs themselves but also from various industry/labor special interests who will attempt to exert influence as automation percolates throughout the economy. Ball argues that keeping the government’s role minimal[3] avoids a “chokepoint” which can be used for easy regulatory capture:
empowering a centralized, general-purpose regulator of frontier AI systems risks creating a legal chokepoint, allowing any and all of these groups to lobby for regulations that hinder the velocity of AI development, the utility of AI products (for example, by requiring AI outputs that compete with the work of licensed professionals to be reviewed by a licensed human in that field), and the overall diffusion of AI technology in society.
Some of the other benefits of multiple standard setting organizations as Ball sees them are:
mitigating against the tendency of regulations to become more complex (and with an increasing compliance cost) over time
allowing for innovation/experimentation at the governance level (Ball specifically cites here he thinks private entities will be nimbler at using technology to craft/enforce/test standards)
avoiding a “one-size-fits-all” approach to regulation
On this last point he envisions that different organizations might form around creating standards for the use of models in different sectors/industries:
New markets that develop over time, such as robotics or biological foundation models, could be served with dedicated governance organizations, rather than having to be squeezed into some pre-existing regulatory design[4].
Making the Marketplace Maximally Efficient
I’m not going to argue for or against the regulatory marketplace concept in general here. I’m going to approach this as if I were assuming prima facie the marketplace idea is the best possible structure, and provide constructive suggestions within those bounds.
With that in mind, here is how I would iterate on Ball’s proposed structure to make the “marketplace” more efficient.
1. Skin in the Game
What makes markets efficient is that they are adversarial environments where each participant has what Nassim Taleb calls “Skin in the Game”. This forces everyone to ruthlessly question their own assumptions and worldviews at all times, or risk losing their own capital.
While standard setting organizations being at risk of losing future revenue prospects if the government revokes their status does provide some risk, there’s a stark difference between losing hypothetical future revenue and losing capital you already provided.
Future revenues are hypothetical, so financial models always discount them. This is true whether they’re determining valuations or whether they’re modelling how much risk the potential loss of them represents. Limiting standard setter downside risk to a loss of possible future income leaves them with too little skin in the game, and enables parties who are willing to be lax about standards for short term gain.
Mandating that standard setting organizations bond capital, in order to license labs, fixes this. It incentivizes the standard setters to take greater care. It could also serve as a source for a “victim compensation fund” in the event that people suffer harm in excess of the amount which the party who misused the model is able to compensate them for.
In that same vein, one of the elements which Ball mentioned as optional (a threshold for damages beyond which safe harbor does not apply for labs regardless of licensure) is in fact a necessary part of making sure this marketplace functions efficiently. It is important for labs to internalize some risk, so that they have skin in the game as a marketplace participant.
The article gives a threshold of $500m, reducing that[5] and requiring labs to carry insurance, provides immediate incentive to labs to be careful. It would also have insurers looking very carefully at all testing procedures/data, with the kind of skeptical eye we want pouring over these things. If for no reason other than to reduce their own insurance premiums, labs will tend to be more careful about releasing models, and pick certifying agencies whose standards are rigorous enough that they give insurers more confidence.
2. Increased Transparency
Markets function more efficiently when there are required disclosures and transparency. While nothing about Ball’s proposal disclaims this, I argue that one of the cornerstone principles of the government agency which certifies private standard setters should be a requirement for transparent disclosure of methods and results. In other words, standard setting agencies should only be able to grant licenses and safe harbor, if they disclose testing methods and results.
3. Public Feedback & Testing
Markets are best at price discovery when they have a wide array of participants engaging.
Analogous to this, we have examples of instances where public feedback during the testing of frontier models pre-deployment has led to good outcomes. In Anthropic’s CBRN public contest, four different teams were able to jailbreak Claude 3.5 Sonnet and claim bounties. Presumably, this helped Anthropic shore up the model against customer misuse before deployment.
If we want this regulatory marketplace to be maximally efficient, providing public opportunities for testing and commentary before or even after licensure is a mechanism by which we can crowdsource useful results. Especially when it comes to jailbreaking, where perhaps the single most prolific expert is anonymous, the collective power of millions of potential contributors is nothing to scoff at.
I think this forms the basis for another reasonable requirement. Licenses are being granted to labs and not on a model by model basis, but that doesn’t mean we can’t require that all models undergo some sort of public testing/feedback period before/at deployment.
We don’t even have to require the models “pass”. The beauty of the marketplace structure is that if the models fail these tests with no clear reaction from the standard setting organizations and/or labs, then insurers will justifiably raise their premiums or the lack of reaction will be used as evidence by attorneys when damages arise. Another benefit of all parties having skin in the game.
A less burdensome alternative to having this structured as a pre-release public testing period would just be some sort of public commentary board, where those who have successfully jailbroken models can publish how. This would be less expensive for deployers/labs, but also means that public feedback can only be “reactive” and not “proactive”.
Thanks to Dean Ball for taking the time to answer my questions on this.
Ball does mention that state governments could implement this as well, but only to clarify that from an implementation perspective such a thing is possible. It’s clear from his mentions of wanting to avoid a “patchwork” liability framework in the US, that he prefers this is handled on a federal level.
This same logic is also why he prefers a licensing scheme which focuses the labs themselves, instead of licensing on a model by model basis.
Many frontier AI policy proposals have focused on regulating frontier models. This approach typically involves creating a threshold based on characteristics of the model above which regulation of some kind is triggered. The challenge with this approach is that the frontier is in constant and rapid motion; today’s frontier is tomorrow’s old news.
[...]
A better approach would be to rely upon a concept that is unlikely to go out of fashion so rapidly: the corporate entity.
Regulating frontier AI firms could include not just their largest and most expensive models, but all models developed by the firm, whether for internal or public deployment, as well as the firm’s business and research practices, internal processes, and holistic approach to safety and security.
This parallels some of what Novelli/Mocanu argue in their proposed legal personality framework for models:
The conditions that make legal personality appropriate in one context (e.g., e-commerce) may be very different from those that make it useful in another (e.g., robots used in health care or in manufacturing).
A tailored “last mile” approach to liability seems pretty popular.
Really where the number is placed is a subject for its own research, but it should be low enough that labs are actually worried about it, while being high enough that it won’t incentivize lawsuits arbitrarily attempting to hit that number. The median “nuclear” tort verdict (damages of over $10m) is $23.8m. 2-3X that seems like a reasonable starting point.
Liability for Misuse of Models—Dean Ball’s Proposal
Introduction
This article explores White House Office of Science and Technology Policy advisor Dean Ball’s proposal as detailed in his paper “A Framework for the Private Governance of Frontier Artificial Intelligence”. I think the paper provides a useful insight to how some of the people advising the White House on regulating these issues are thinking about the coming economic transition.
Ball’s proposal primarily focuses on how to approach the question of civil liability for the customer misuse of models. His proposal outlines a “marketplace” of hybrid public-private standards setting organizations, which must be licensed by the federal[1] government but are not government agencies themselves. Meeting these standards would not necessarily be mandatory for labs seeking to deploy models, but labs who opted in to meeting at least one such standard would be shielded from tort liability as a result of customer misuse of the models once deployed. In his own words,
In one of his Substack posts, Ball did mention FINRA as an example of a successful non-governmental standards setting organization. However, FINRA has what is essentially a monopoly within its industry and as such is not a great example of a regulatory marketplace.
When I asked him for an example of a successful ‘regulatory marketplace’ Ball pointed me to the London Stock Exchange’s Alternative Investment Market. The LSE’s AIM only allows companies to list if they have engaged a “Nominated Advisor” (NOMAD), these are private entities who compete for business in much the same way Ball envisions standard setters competing to certify labs. A NOMAD must confirm that the AIM issuer meets admission rules, provide ongoing oversight, and notify the LSE of breaches.
While it’s unlikely to be a perfect 1⁄1 comparison, when reading the following proposal you can keep the NOMAD system in mind as a “market comp”.
With no further ado, let’s get into the substance of his proposal.
The Proposed Framework
Ball does leave room for flexibility wherein developers may have some sort of liability exposure in extreme cases;
He provides the following guidance on the structure and power of the governmental agency which licenses the private standards setting organization:
Ball’s intention is that the private regulatory agencies themselves would be subject to minimal government regulations, with the idea that marketplace dynamics would function as a better constraining function (more on that in the next section):
Ball is careful to clarify that he sees this as a regulatory approach which is inherently transitory and advocates that at a later date when we have more data on the risks of real world use, a new approach can be adopted:
Ball also separates harms resulting from a “loss of control” and makes it clear developers would not be protected by their safe harbor in such a situation:
Critiques Towards Alternatives
Ball’s motivation on providing developers a path to a safe harbor against tort liability specifically from misuse, can be said to arise from a general skepticism around the application of tort liability for “platform technologies” providers:
He also spends much of the paper criticizing other commonly proposed regulatory alternatives.
Ball expresses skepticism about international governance. He prefers that even in the event the US engages in such discussions it does so from a position of having already established a robust domestic regulatory framework:
Ball is also lukewarm towards compute governance. He acknowledges that it was effective at denying China an early advantage in the space, but also thinks in the long term it will prove counterproductive:
And he is quite firmly against any sort of domestic compute governance, in particular anything involving putting backdoors on chips:
Pros of the Regulatory Marketplace
Safe harbor is probably the most boilerplate part of the proposal. What is novel about Ball’s framework is the “regulatory marketplace” approach, and as such I will focus on the benefits he sees arising from such a framework.
One of the primary motivators of the regulatory marketplace structure is the need for “standard setters” who are capable of keeping pace with an industry undergoing rapid technological development. Ball believes that government entities are unlikely to be able to stay up to date to the extent required to do their job, unlike private entities incentivized by a competitive environment.[2]
Further, he expresses concern about funding for any government agency becoming a political football at a later date, which I think is part of why he prefers to keep the role of such an agency as minimal as possible.
Of course in reading this proposal one of the first questions that any concerned party will ask is about regulatory capture. Ball discusses this in the context of efforts not only from labs themselves but also from various industry/labor special interests who will attempt to exert influence as automation percolates throughout the economy. Ball argues that keeping the government’s role minimal[3] avoids a “chokepoint” which can be used for easy regulatory capture:
Some of the other benefits of multiple standard setting organizations as Ball sees them are:
mitigating against the tendency of regulations to become more complex (and with an increasing compliance cost) over time
allowing for innovation/experimentation at the governance level (Ball specifically cites here he thinks private entities will be nimbler at using technology to craft/enforce/test standards)
avoiding a “one-size-fits-all” approach to regulation
On this last point he envisions that different organizations might form around creating standards for the use of models in different sectors/industries:
Making the Marketplace Maximally Efficient
I’m not going to argue for or against the regulatory marketplace concept in general here. I’m going to approach this as if I were assuming prima facie the marketplace idea is the best possible structure, and provide constructive suggestions within those bounds.
With that in mind, here is how I would iterate on Ball’s proposed structure to make the “marketplace” more efficient.
1. Skin in the Game
What makes markets efficient is that they are adversarial environments where each participant has what Nassim Taleb calls “Skin in the Game”. This forces everyone to ruthlessly question their own assumptions and worldviews at all times, or risk losing their own capital.
While standard setting organizations being at risk of losing future revenue prospects if the government revokes their status does provide some risk, there’s a stark difference between losing hypothetical future revenue and losing capital you already provided.
Future revenues are hypothetical, so financial models always discount them. This is true whether they’re determining valuations or whether they’re modelling how much risk the potential loss of them represents. Limiting standard setter downside risk to a loss of possible future income leaves them with too little skin in the game, and enables parties who are willing to be lax about standards for short term gain.
Mandating that standard setting organizations bond capital, in order to license labs, fixes this. It incentivizes the standard setters to take greater care. It could also serve as a source for a “victim compensation fund” in the event that people suffer harm in excess of the amount which the party who misused the model is able to compensate them for.
In that same vein, one of the elements which Ball mentioned as optional (a threshold for damages beyond which safe harbor does not apply for labs regardless of licensure) is in fact a necessary part of making sure this marketplace functions efficiently. It is important for labs to internalize some risk, so that they have skin in the game as a marketplace participant.
The article gives a threshold of $500m, reducing that[5] and requiring labs to carry insurance, provides immediate incentive to labs to be careful. It would also have insurers looking very carefully at all testing procedures/data, with the kind of skeptical eye we want pouring over these things. If for no reason other than to reduce their own insurance premiums, labs will tend to be more careful about releasing models, and pick certifying agencies whose standards are rigorous enough that they give insurers more confidence.
2. Increased Transparency
Markets function more efficiently when there are required disclosures and transparency. While nothing about Ball’s proposal disclaims this, I argue that one of the cornerstone principles of the government agency which certifies private standard setters should be a requirement for transparent disclosure of methods and results. In other words, standard setting agencies should only be able to grant licenses and safe harbor, if they disclose testing methods and results.
3. Public Feedback & Testing
Markets are best at price discovery when they have a wide array of participants engaging.
Analogous to this, we have examples of instances where public feedback during the testing of frontier models pre-deployment has led to good outcomes. In Anthropic’s CBRN public contest, four different teams were able to jailbreak Claude 3.5 Sonnet and claim bounties. Presumably, this helped Anthropic shore up the model against customer misuse before deployment.
If we want this regulatory marketplace to be maximally efficient, providing public opportunities for testing and commentary before or even after licensure is a mechanism by which we can crowdsource useful results. Especially when it comes to jailbreaking, where perhaps the single most prolific expert is anonymous, the collective power of millions of potential contributors is nothing to scoff at.
I think this forms the basis for another reasonable requirement. Licenses are being granted to labs and not on a model by model basis, but that doesn’t mean we can’t require that all models undergo some sort of public testing/feedback period before/at deployment.
We don’t even have to require the models “pass”. The beauty of the marketplace structure is that if the models fail these tests with no clear reaction from the standard setting organizations and/or labs, then insurers will justifiably raise their premiums or the lack of reaction will be used as evidence by attorneys when damages arise. Another benefit of all parties having skin in the game.
A less burdensome alternative to having this structured as a pre-release public testing period would just be some sort of public commentary board, where those who have successfully jailbroken models can publish how. This would be less expensive for deployers/labs, but also means that public feedback can only be “reactive” and not “proactive”.
Thanks to Dean Ball for taking the time to answer my questions on this.
Ball does mention that state governments could implement this as well, but only to clarify that from an implementation perspective such a thing is possible. It’s clear from his mentions of wanting to avoid a “patchwork” liability framework in the US, that he prefers this is handled on a federal level.
This same logic is also why he prefers a licensing scheme which focuses the labs themselves, instead of licensing on a model by model basis.
My words not his.
This parallels some of what Novelli/Mocanu argue in their proposed legal personality framework for models:
A tailored “last mile” approach to liability seems pretty popular.
Really where the number is placed is a subject for its own research, but it should be low enough that labs are actually worried about it, while being high enough that it won’t incentivize lawsuits arbitrarily attempting to hit that number. The median “nuclear” tort verdict (damages of over $10m) is $23.8m. 2-3X that seems like a reasonable starting point.