You don’t think evidence of the flimsiness of red lines should be a reason to be pessimistic? I mean as far as companies go, this was one of the biggest. And as far as red line topics go, autonomous weapons is among the biggest. I just struggle to see how every company minus Anthropic signing on is not a bad sign.
this was not enforced governance at all. this was a petition. it may be a reason to be pessimistic about petitions that are not enforced laws. and a reason to invest more in enforced laws
We might be talking about two separate kinds of governance. When you say “enforced laws” you seem to be talking about legal liability, and the sorts of things that shape behaviour through penalties after the fact. But the legal system treats criminal matters and market issues quite differently. In liberal democracies, it’s rather hard to get preemptive actions that restrict the freedom of market actors because of the belief that things work best without intervention.
Even when companies break laws which are ‘enforced laws’ like dumping chemicals into a body of water, the legal system is not quick to act. And if remedies ever come, they often come years after a gruelling series of trials, stays, appeals, etc.
When I think of governance I typically think of the domain of policy, and things like program evaluation, CBAs, agenda-setting, and mechanism design—not legal liability. I guess what I’m trying to say partly is, governance succeeds best in the world of policy, when we can get people to want to do what we want, rather than in the domain of law, where we punish those who do not.
And so the reason I think this is not merely a petition is because of its substance. At a surface level, I get you, this document is a petition.
But the petition was for a framework whose actual contents are a series of policies that, if implemented, would enforce action on the very principles DeepMind had already committed to. To me, this seems like enforced governance. The petition was a critical part of the mechanism for enforcement (internal assent providing pressure on high-level decision-makers in the company).
Because think about it for a second, in terms of the actual law that we might want enforced, what would it be? If the law is “No company can develop autonomous weapons. Non-compliance results in a fine worth 10% of annual revenue.” that’s excellent, but political unviable in the current climate.
If the law we would want instead is “Any company that violates principles they have committed to publicly will be fined”, that works, but now we face a bigger problem:
What rationale does the legal system use to justify punishing actors for commitments to themselves?
The legal system is largely equipped to punish violations of (contractual) commitments to other actors. You can violate a ‘contract’ with yourself, but it’s not a contract, and the law does not punish violating mere promises.
e.g. if a company commits to only making solar panels and green products, and then expands into making diesel generators, is our legal system genuinely prepared to punish a business for expanding into new sectors, because it was a self-commitment? The composition of the current conservative-dominated Supreme Court makes this seem unlikely.
Will other companies still want to keep having principles, or will they be incentivized to downgrade their commitments from binding principles to lower-level mission statements type things?
This does not even require that the other companies suddenly become ‘evil’. Motivated reasoning can easily allow companies to reason that:
(a) they know they are good, so they know they will continue to follow their principles,
(b) so there’s no need for it to be binding because when the time comes they will still do the right thing,
(c) hence making their principles binding commitments opens them to potential legal liability from bad actors (say rival labs) and mistaken, paranoid assessments from moral panic
I don’t mean to diminish your desire. These are all the more reasons to invest in enforced laws. But reasons, however good, are not sufficient to motivate action. We already have reasons to implement carbon taxes and carbon border adjustment mechanisms—rather good ones, in fact.
But if all we do is continue accumulating good reasons to invest in things we never do, then you don’t have enforced governance either. Just more bullet points to add on a future petition which, if I understand your perspective right, is already a mechanism you don’t believe in.
I hope I’m not misrepresenting your arguments. If there was something else you meant by laws you would like enforced, I would really like to hear what you’re thinking. It’s just that from my experience in the legal discipline, there are no paradigms, frameworks, or jurisprudential theories in common law that allow a state to sue an actor for changing its mind.
You don’t think evidence of the flimsiness of red lines should be a reason to be pessimistic? I mean as far as companies go, this was one of the biggest. And as far as red line topics go, autonomous weapons is among the biggest. I just struggle to see how every company minus Anthropic signing on is not a bad sign.
this was not enforced governance at all. this was a petition. it may be a reason to be pessimistic about petitions that are not enforced laws. and a reason to invest more in enforced laws
We might be talking about two separate kinds of governance. When you say “enforced laws” you seem to be talking about legal liability, and the sorts of things that shape behaviour through penalties after the fact. But the legal system treats criminal matters and market issues quite differently. In liberal democracies, it’s rather hard to get preemptive actions that restrict the freedom of market actors because of the belief that things work best without intervention.
Even when companies break laws which are ‘enforced laws’ like dumping chemicals into a body of water, the legal system is not quick to act. And if remedies ever come, they often come years after a gruelling series of trials, stays, appeals, etc.
When I think of governance I typically think of the domain of policy, and things like program evaluation, CBAs, agenda-setting, and mechanism design—not legal liability. I guess what I’m trying to say partly is, governance succeeds best in the world of policy, when we can get people to want to do what we want, rather than in the domain of law, where we punish those who do not.
And so the reason I think this is not merely a petition is because of its substance. At a surface level, I get you, this document is a petition.
But the petition was for a framework whose actual contents are a series of policies that, if implemented, would enforce action on the very principles DeepMind had already committed to. To me, this seems like enforced governance. The petition was a critical part of the mechanism for enforcement (internal assent providing pressure on high-level decision-makers in the company).
Because think about it for a second, in terms of the actual law that we might want enforced, what would it be? If the law is “No company can develop autonomous weapons. Non-compliance results in a fine worth 10% of annual revenue.” that’s excellent, but political unviable in the current climate.
If the law we would want instead is “Any company that violates principles they have committed to publicly will be fined”, that works, but now we face a bigger problem:
What rationale does the legal system use to justify punishing actors for commitments to themselves?
The legal system is largely equipped to punish violations of (contractual) commitments to other actors. You can violate a ‘contract’ with yourself, but it’s not a contract, and the law does not punish violating mere promises.
e.g. if a company commits to only making solar panels and green products, and then expands into making diesel generators, is our legal system genuinely prepared to punish a business for expanding into new sectors, because it was a self-commitment? The composition of the current conservative-dominated Supreme Court makes this seem unlikely.
Will other companies still want to keep having principles, or will they be incentivized to downgrade their commitments from binding principles to lower-level mission statements type things?
This does not even require that the other companies suddenly become ‘evil’. Motivated reasoning can easily allow companies to reason that:
(a) they know they are good, so they know they will continue to follow their principles,
(b) so there’s no need for it to be binding because when the time comes they will still do the right thing,
(c) hence making their principles binding commitments opens them to potential legal liability from bad actors (say rival labs) and mistaken, paranoid assessments from moral panic
I don’t mean to diminish your desire. These are all the more reasons to invest in enforced laws. But reasons, however good, are not sufficient to motivate action. We already have reasons to implement carbon taxes and carbon border adjustment mechanisms—rather good ones, in fact.
But if all we do is continue accumulating good reasons to invest in things we never do, then you don’t have enforced governance either. Just more bullet points to add on a future petition which, if I understand your perspective right, is already a mechanism you don’t believe in.
I hope I’m not misrepresenting your arguments. If there was something else you meant by laws you would like enforced, I would really like to hear what you’re thinking. It’s just that from my experience in the legal discipline, there are no paradigms, frameworks, or jurisprudential theories in common law that allow a state to sue an actor for changing its mind.