″ Especially for corporate customers, safety and reliability are important unique selling points,
Only very prosaic, not catastrophic risks(as a customer I would not care at all about likelihood of catastrophic risks from my vendor—that’s something that affects humanity regardless of customer relationships).
Only if a particular secret is useful for preventing both catastrophes and prosaic ‘safety failures’ would this be a consideration for us—catastrophic risks increasing due to companies trying to have a competitive edge in prosaic safety risks
I can see why you’d say that, but I think for me the two are often intermingled and hard to separate. Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale (which I’m not sure I buy on its own), they’re probably going to want to avoid the economic turbulence which would ensue from egregiously-misaligned, capable AIs being deployed.
For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons (e.g. cost, lower performance, etc.). If a lab can demonstrate that their models won’t attempt things of this nature, that’s a win for the commercial customers.
>For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons
Agreed.
>Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale
To clarify I didn’t meant that clients wouldn’t care about xrisk, but that they would not care about xrisk specifically from their vendor, or about the marginally increased x risk arising from using a certain unsafe frontier lab as a vendor(versus another). This marginally increased x risk is borne by all of humanity of which the client is a tiny slice, so they would undercare(versus the socially optimal amount of caring) about their marginal impact on this risk, as human extinction would be an externality. Similar to companies undercaring about GHG emissions.
You propose that companies would use vendors because they advertise “10% less existential risk!”—this is similar to using a company because they promise to donate 10% of their revenues to climate charities as a USP. Except in this case frontier labs can share their safety research with each other for free.
So even if the clients are perfectly altruistic, a frontier lab could reduce x risk by
Doing AIS research
Publishing AIS research
So why would clients reward vendors for (1) but not (2)?
Only very prosaic, not catastrophic risks(as a customer I would not care at all about likelihood of catastrophic risks from my vendor—that’s something that affects humanity regardless of customer relationships).
Only if a particular secret is useful for preventing both catastrophes and prosaic ‘safety failures’ would this be a consideration for us—catastrophic risks increasing due to companies trying to have a competitive edge in prosaic safety risks
Sorry for the late reply!
I can see why you’d say that, but I think for me the two are often intermingled and hard to separate. Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale (which I’m not sure I buy on its own), they’re probably going to want to avoid the economic turbulence which would ensue from egregiously-misaligned, capable AIs being deployed.
For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons (e.g. cost, lower performance, etc.). If a lab can demonstrate that their models won’t attempt things of this nature, that’s a win for the commercial customers.
Apologies for the late response as well.
>For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons
Agreed.
>Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale
To clarify I didn’t meant that clients wouldn’t care about xrisk, but that they would not care about xrisk specifically from their vendor, or about the marginally increased x risk arising from using a certain unsafe frontier lab as a vendor(versus another). This marginally increased x risk is borne by all of humanity of which the client is a tiny slice, so they would undercare(versus the socially optimal amount of caring) about their marginal impact on this risk, as human extinction would be an externality. Similar to companies undercaring about GHG emissions.
You propose that companies would use vendors because they advertise “10% less existential risk!”—this is similar to using a company because they promise to donate 10% of their revenues to climate charities as a USP. Except in this case frontier labs can share their safety research with each other for free.
So even if the clients are perfectly altruistic, a frontier lab could reduce x risk by
Doing AIS research
Publishing AIS research
So why would clients reward vendors for (1) but not (2)?