I can see why you’d say that, but I think for me the two are often intermingled and hard to separate. Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale (which I’m not sure I buy on its own), they’re probably going to want to avoid the economic turbulence which would ensue from egregiously-misaligned, capable AIs being deployed.
For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons (e.g. cost, lower performance, etc.). If a lab can demonstrate that their models won’t attempt things of this nature, that’s a win for the commercial customers.
>For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons
Agreed.
>Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale
To clarify I didn’t meant that clients wouldn’t care about xrisk, but that they would not care about xrisk specifically from their vendor, or about the marginally increased x risk arising from using a certain unsafe frontier lab as a vendor(versus another). This marginally increased x risk is borne by all of humanity of which the client is a tiny slice, so they would undercare(versus the socially optimal amount of caring) about their marginal impact on this risk, as human extinction would be an externality. Similar to companies undercaring about GHG emissions.
You propose that companies would use vendors because they advertise “10% less existential risk!”—this is similar to using a company because they promise to donate 10% of their revenues to climate charities as a USP. Except in this case frontier labs can share their safety research with each other for free.
So even if the clients are perfectly altruistic, a frontier lab could reduce x risk by
Doing AIS research
Publishing AIS research
So why would clients reward vendors for (1) but not (2)?
Sorry for the late reply!
I can see why you’d say that, but I think for me the two are often intermingled and hard to separate. Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale (which I’m not sure I buy on its own), they’re probably going to want to avoid the economic turbulence which would ensue from egregiously-misaligned, capable AIs being deployed.
For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons (e.g. cost, lower performance, etc.). If a lab can demonstrate that their models won’t attempt things of this nature, that’s a win for the commercial customers.
Apologies for the late response as well.
>For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons
Agreed.
>Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale
To clarify I didn’t meant that clients wouldn’t care about xrisk, but that they would not care about xrisk specifically from their vendor, or about the marginally increased x risk arising from using a certain unsafe frontier lab as a vendor(versus another). This marginally increased x risk is borne by all of humanity of which the client is a tiny slice, so they would undercare(versus the socially optimal amount of caring) about their marginal impact on this risk, as human extinction would be an externality. Similar to companies undercaring about GHG emissions.
You propose that companies would use vendors because they advertise “10% less existential risk!”—this is similar to using a company because they promise to donate 10% of their revenues to climate charities as a USP. Except in this case frontier labs can share their safety research with each other for free.
So even if the clients are perfectly altruistic, a frontier lab could reduce x risk by
Doing AIS research
Publishing AIS research
So why would clients reward vendors for (1) but not (2)?