AI is a novel form of software service. It is ‘non-deterministic’, compared to conventional software, which already had limited liability associated with its failures. AI companies do not make guarantees about the performance of their services. Companies building on top of the models likewise seem not to have thought through the liability model. I wonder, if the doctor’s office had approved the order and you consumed the meds, who would you be able to hold legally accountable in your opinion, if you had an adverse outcome? I’m not a lawyer, I genuinely wonder. But I have no doubt that the AI model vendor would not be held responsible in anyway.
I agree with the sentiment expressed below about how it is amazing how little effort is put into validation and monitoring of AI systems, but as I reflect on the amount of untested production code there is currently in existence, my amazement decreases.
Ultimately, human workers provide an easy sink for accountability. If a human had done what you describe, regardless of who ultimately got sued, it seems undoubtedly true they would have been fired. But firing an AI phone system is not nearly so easy to do, since it would entail minimally changing contracts to a different vendor, or possibly fully reviving human staffing for the role. So, I feel the short-term future is just less accountable as a whole.
AI is a novel form of software service. It is ‘non-deterministic’, compared to conventional software, which already had limited liability associated with its failures. AI companies do not make guarantees about the performance of their services. Companies building on top of the models likewise seem not to have thought through the liability model. I wonder, if the doctor’s office had approved the order and you consumed the meds, who would you be able to hold legally accountable in your opinion, if you had an adverse outcome? I’m not a lawyer, I genuinely wonder. But I have no doubt that the AI model vendor would not be held responsible in anyway.
I agree with the sentiment expressed below about how it is amazing how little effort is put into validation and monitoring of AI systems, but as I reflect on the amount of untested production code there is currently in existence, my amazement decreases.
Ultimately, human workers provide an easy sink for accountability. If a human had done what you describe, regardless of who ultimately got sued, it seems undoubtedly true they would have been fired. But firing an AI phone system is not nearly so easy to do, since it would entail minimally changing contracts to a different vendor, or possibly fully reviving human staffing for the role. So, I feel the short-term future is just less accountable as a whole.