″ Especially for corporate customers, safety and reliability are important unique selling points,
Only very prosaic, not catastrophic risks(as a customer I would not care at all about likelihood of catastrophic risks from my vendor—that’s something that affects humanity regardless of customer relationships).
Only if a particular secret is useful for preventing both catastrophes and prosaic ‘safety failures’ would this be a consideration for us—catastrophic risks increasing due to companies trying to have a competitive edge in prosaic safety risks
I can see why you’d say that, but I think for me the two are often intermingled and hard to separate. Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale (which I’m not sure I buy on its own), they’re probably going to want to avoid the economic turbulence which would ensue from egregiously-misaligned, capable AIs being deployed.
For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons (e.g. cost, lower performance, etc.). If a lab can demonstrate that their models won’t attempt things of this nature, that’s a win for the commercial customers.
Only very prosaic, not catastrophic risks(as a customer I would not care at all about likelihood of catastrophic risks from my vendor—that’s something that affects humanity regardless of customer relationships).
Only if a particular secret is useful for preventing both catastrophes and prosaic ‘safety failures’ would this be a consideration for us—catastrophic risks increasing due to companies trying to have a competitive edge in prosaic safety risks
Sorry for the late reply!
I can see why you’d say that, but I think for me the two are often intermingled and hard to separate. Even assuming that the most greedy/single-minded business leaders wouldn’t care about catastrophic risks on a global scale (which I’m not sure I buy on its own), they’re probably going to want to avoid the economic turbulence which would ensue from egregiously-misaligned, capable AIs being deployed.
For a more fine-grained example, actions like siphoning compute to run unauthorised tasks might be a signal that a model poses significantly higher catastrophic risk, but would also be something a commercial business would want to prevent for their own reasons (e.g. cost, lower performance, etc.). If a lab can demonstrate that their models won’t attempt things of this nature, that’s a win for the commercial customers.