In the absence of a fire alarm, what other observable signals or patterns might we look out for that this perennial line-crossing predicates harm?
Are there behaviors or events we can monitor for in the world that, having crossed some red line, a real threat is emerging as a result of some model capability?
Or are defining the red lines the only warning we have, understanding that we cross them at our own peril because we can only assess the threat post hoc?
There are various possible worlds with AI progress posing different risks.
In those worlds where a given capability level is a problem, we’re not setting ourselves up to notice or react even after the harm materializes. The set of behaviors or events that we could be monitoring keep being spelled out, in the form of red lines. And then they happen. We’re already seeing tons of concrete harms—what more do we need? Do you think things will change if there’s an actual chemical weapons attack? Or a rogue autonomous replication? Or is there some number of people that need to die first?
In the absence of a fire alarm, what other observable signals or patterns might we look out for that this perennial line-crossing predicates harm?
Are there behaviors or events we can monitor for in the world that, having crossed some red line, a real threat is emerging as a result of some model capability?
Or are defining the red lines the only warning we have, understanding that we cross them at our own peril because we can only assess the threat post hoc?
There are various possible worlds with AI progress posing different risks.
In those worlds where a given capability level is a problem, we’re not setting ourselves up to notice or react even after the harm materializes. The set of behaviors or events that we could be monitoring keep being spelled out, in the form of red lines. And then they happen. We’re already seeing tons of concrete harms—what more do we need? Do you think things will change if there’s an actual chemical weapons attack? Or a rogue autonomous replication? Or is there some number of people that need to die first?
Probably yes. Safety regulations are written in blood.