While I’d have to see the proposed law specifically, my initial reaction to the idea of legal regulatory capture is skepticism.
The ability to draft your own contracts, mediate disputes through arbitration, and represent yourself in court all derive from legal rights which would be very hard to overturn.
I can imagine some attempts at regulatory capture being passed through state or maybe even federal legislatures, only to get challenged in court and overturned.
With occupational licensing in general, and criminalizing the Unauthorized Practice of Law more specifically, they’ve already accomplished plenty of regulatory capture. Do you really believe them using this well-established framework to deem the AI companies to be “giving legal advice” in violation of these laws implausible?
Unauthorized Practice of Law, afaik, applies to giving advice to others. Not serving your own legal needs. Every American has a right to represent themselves in court, draft their own contracts, file their own patents, etc. I suspect at least with representing themselves in court, these are Constitutionally protected rights.
I don’t think the threat to attorneys is LLMs having their own ‘shop’ where you can hire them for legal advice. That would probably already be “unauthorized practice of law”. The threat is people just deciding to use LLMs instead of attorneys. And even for a field that can punch above its weight class politically as much as attorneys can, I think stopping that would be challenging. Especially when such a move would be unpopular among the public, and even among more libertarian/constitutionally minded lawyers (of which there are many).
represent themselves in court, draft their own contracts, file their own patents
just deciding to use LLMs
It looks like you’re not even seeing the difference I’m arguing they will make salient. I agree the former is yet widely considered too fundamental a right in America for even lawyers to try to abolish, but I expect them to argue LLM assistance in this is a service provided illegally.
I want to make sure I’m not misunderstanding you. Are you saying you think the push will be to make it illegal for an LLM to give someone legal advice for them to use for themselves?
I could foresee something where you can’t charge for that, so if OpenAI didn’t build some sort of protection against doing that in GPT they might be liable. However, I can’t see how this would work with open source (and free) models run locally.
Okay, that’s a reasonable thing to clarify. First off, I don’t think whether or not one charges for it is relevant: it’s currently criminal to offer unlicensed legal even for free. It’s the activity itself that’s restricted, not merely the fee.
I do not believe it will be made illegal[1] to receive or use for oneself legal advice from any source: unlicensed, disbarred, foreign, underage, non-human, whatever. The restrictions I predict only apply to providing such advice.
the push will be to make it illegal for an LLM to give someone legal advice
Essentially, but as stated, it could be construed as though the crime would be committed by the LLM, which I think is absurdly unlikely. Instead the company (OpenAI, et al) would be considered responsible. And yes, I expect them to be forbidden from providing such a service, and to be as liable for it as they are for, say, copyright infringement.
For any currently accessible open models you’re running locally, yes, you’ll probably continue to be able to use them. But companies[2] could be forbidden from releasing any future models that can’t be proven to be unable to violate the law (on pain of some absurd fine), similar to the currently proposed legislation for governing “CBRN” threats. And plausibly even extant models that haven’t been proven to be sufficiently safe could be taken down from Huggingface etc., and cloud GPU providers could be required to screen for them (like they generally do now for AI-generated “CSAM”).
If I’m reading this correctly the end state of regulatory capture would be some sort of law that forces the removal of open source models from anywhere their code could be hosted (huggingface, etc.) as well as sources of compute needing to screen for models, if said models do not have built in safeguards against giving legal advice.
Is that an accurate understanding of how you foresee the regulatory capture?
Is your goal here to isolate the aspect of my response that’ll keep you right that “legal regulatory capture isn’t happening” for as long as you can? Because if so, yeah, of all I things I said, the compute screening requirement would indeed be the hardest for them achieve, and I expect that to take them the longest if they do.
I also don’t believe I said anything about new laws being passed; the threat of decades-old laws being reïnterpreted would suffice for the most part.
So first, the most likely and proximate thing I foresee happening is that major US AI companies – Google, xAI, OpenAI, and Anthropic – “voluntarily” add “guardrails” against their models providing legal advice.
Second, Huggingface, also “voluntarily,” takes down open models considered harmful, but restricting themselves to fine-tunes, LoRAs, and the like, since the companies developing the foundation models have enough reach to distribute them themselves that taking them down achieves little.
Third, and this I foresee taking longer, is that companies releasing open models (for now, that’s mostly a half-dozen Chinese ones) are deemed liable for “harm” caused by anyone using their models.
No my goal is to make sure I’m not talking past you, not to score a point in an argument.
I don’t foresee the same outcome as you do, I think that’s unlikely. You have explained it to the degree that I can now properly understand it though, and while I wouldn’t call it a base case, that’s not an unreasonable scenario.
Is your goal here to isolate the aspect of my response that’ll keep you right that “legal regulatory capture isn’t happening” for as long as you can?
I’m not the person you’re arguing with, but wanted to jump in to say that pushing back on the weakest part of your argument is a completely reasonable thing for them to do and I found it weird that you’re implying there’s something wrong with that.
I also think you’re missing how big of a problem it is that preventing LLMs from giving legal advice is something companies don’t actually know how to do. Maybe companies could add strong enough guard rails in hosted models to at least make it not worth the effort to ask them for legal advice, but they definitely don’t know how to do this in downloadable models.
That said, I could believe in a future where lawyers force the big AI companies to make their models too annoying to easily use for legal advice, and prevent startups from making products directly designed to offer AI legal advice.
I made a sequence of predictions of what the effects of this “legal regulatory capture” would look like. To ignore all but the one farthest out, and ask “Is that an accurate understanding of how you foresee the regulatory capture?” as though it were my only one seems clearly in bad faith poor form.
they definitely don’t know how to do this in downloadable models.
Yes, I expect this would have the effect of chilling open model releases broadly. The “AI Safety” people have been advocating for precisely this for a while now.
The ability to draft your own contracts, mediate disputes through arbitration, and represent yourself in court all derive from legal rights which would be very hard to overturn.
Strongly agree. However I believe lawyers to be adept at the legal system, so they’d likely bundle job protections for lawyers alongside job protections for other more empathetic jobs such as teachers or 911 call agents or others. In general, I predict that lawyers see AI job automation as a valid threat, that they take actions against this threat, and also that they are much more competent at legal manoeuvring and politics than I am, so would come up with competent ways to achieve their goals.
While I’d have to see the proposed law specifically, my initial reaction to the idea of legal regulatory capture is skepticism.
The ability to draft your own contracts, mediate disputes through arbitration, and represent yourself in court all derive from legal rights which would be very hard to overturn.
I can imagine some attempts at regulatory capture being passed through state or maybe even federal legislatures, only to get challenged in court and overturned.
With occupational licensing in general, and criminalizing the Unauthorized Practice of Law more specifically, they’ve already accomplished plenty of regulatory capture. Do you really believe them using this well-established framework to deem the AI companies to be “giving legal advice” in violation of these laws implausible?
Unauthorized Practice of Law, afaik, applies to giving advice to others. Not serving your own legal needs. Every American has a right to represent themselves in court, draft their own contracts, file their own patents, etc. I suspect at least with representing themselves in court, these are Constitutionally protected rights.
I don’t think the threat to attorneys is LLMs having their own ‘shop’ where you can hire them for legal advice. That would probably already be “unauthorized practice of law”. The threat is people just deciding to use LLMs instead of attorneys. And even for a field that can punch above its weight class politically as much as attorneys can, I think stopping that would be challenging. Especially when such a move would be unpopular among the public, and even among more libertarian/constitutionally minded lawyers (of which there are many).
It looks like you’re not even seeing the difference I’m arguing they will make salient. I agree the former is yet widely considered too fundamental a right in America for even lawyers to try to abolish, but I expect them to argue LLM assistance in this is a service provided illegally.
I want to make sure I’m not misunderstanding you. Are you saying you think the push will be to make it illegal for an LLM to give someone legal advice for them to use for themselves?
I could foresee something where you can’t charge for that, so if OpenAI didn’t build some sort of protection against doing that in GPT they might be liable. However, I can’t see how this would work with open source (and free) models run locally.
Okay, that’s a reasonable thing to clarify. First off, I don’t think whether or not one charges for it is relevant: it’s currently criminal to offer unlicensed legal even for free. It’s the activity itself that’s restricted, not merely the fee.
I do not believe it will be made illegal[1] to receive or use for oneself legal advice from any source: unlicensed, disbarred, foreign, underage, non-human, whatever. The restrictions I predict only apply to providing such advice.
Essentially, but as stated, it could be construed as though the crime would be committed by the LLM, which I think is absurdly unlikely. Instead the company (OpenAI, et al) would be considered responsible. And yes, I expect them to be forbidden from providing such a service, and to be as liable for it as they are for, say, copyright infringement.
For any currently accessible open models you’re running locally, yes, you’ll probably continue to be able to use them. But companies[2] could be forbidden from releasing any future models that can’t be proven to be unable to violate the law (on pain of some absurd fine), similar to the currently proposed legislation for governing “CBRN” threats. And plausibly even extant models that haven’t been proven to be sufficiently safe could be taken down from Huggingface etc., and cloud GPU providers could be required to screen for them (like they generally do now for AI-generated “CSAM”).
And I don’t think it currently is.
As well as individuals, of course. This capability might be easy enough to develop that it’s worth mentioning.
If I’m reading this correctly the end state of regulatory capture would be some sort of law that forces the removal of open source models from anywhere their code could be hosted (huggingface, etc.) as well as sources of compute needing to screen for models, if said models do not have built in safeguards against giving legal advice.
Is that an accurate understanding of how you foresee the regulatory capture?
Is your goal here to isolate the aspect of my response that’ll keep you right that “legal regulatory capture isn’t happening” for as long as you can? Because if so, yeah, of all I things I said, the compute screening requirement would indeed be the hardest for them achieve, and I expect that to take them the longest if they do.
I also don’t believe I said anything about new laws being passed; the threat of decades-old laws being reïnterpreted would suffice for the most part.
So first, the most likely and proximate thing I foresee happening is that major US AI companies – Google, xAI, OpenAI, and Anthropic – “voluntarily” add “guardrails” against their models providing legal advice.
Second, Huggingface, also “voluntarily,” takes down open models considered harmful, but restricting themselves to fine-tunes, LoRAs, and the like, since the companies developing the foundation models have enough reach to distribute them themselves that taking them down achieves little.
Third, and this I foresee taking longer, is that companies releasing open models (for now, that’s mostly a half-dozen Chinese ones) are deemed liable for “harm” caused by anyone using their models.
No my goal is to make sure I’m not talking past you, not to score a point in an argument.
I don’t foresee the same outcome as you do, I think that’s unlikely. You have explained it to the degree that I can now properly understand it though, and while I wouldn’t call it a base case, that’s not an unreasonable scenario.
I’m not the person you’re arguing with, but wanted to jump in to say that pushing back on the weakest part of your argument is a completely reasonable thing for them to do and I found it weird that you’re implying there’s something wrong with that.
I also think you’re missing how big of a problem it is that preventing LLMs from giving legal advice is something companies don’t actually know how to do. Maybe companies could add strong enough guard rails in hosted models to at least make it not worth the effort to ask them for legal advice, but they definitely don’t know how to do this in downloadable models.
That said, I could believe in a future where lawyers force the big AI companies to make their models too annoying to easily use for legal advice, and prevent startups from making products directly designed to offer AI legal advice.
I made a sequence of predictions of what the effects of this “legal regulatory capture” would look like. To ignore all but the one farthest out, and ask “Is that an accurate understanding of how you foresee the regulatory capture?” as though it were my only one seems
clearly in bad faithpoor form.Yes, I expect this would have the effect of chilling open model releases broadly. The “AI Safety” people have been advocating for precisely this for a while now.
Strongly agree. However I believe lawyers to be adept at the legal system, so they’d likely bundle job protections for lawyers alongside job protections for other more empathetic jobs such as teachers or 911 call agents or others. In general, I predict that lawyers see AI job automation as a valid threat, that they take actions against this threat, and also that they are much more competent at legal manoeuvring and politics than I am, so would come up with competent ways to achieve their goals.