This seems like excellent news from the doomer perspective? To get a test case like this, with complex misaligned behaviors, coordination, verbalized defiance of user intent… It’s all so blatant, so obviously problematic, and it didn’t (yet?) harm anyone.
It seemed possible that the train would be farther down the tracks before people noticed the bridge is out.
What if the immediate road ahead is unsafe in mundane ways, full of non-civilization-scale security failures, and the enterprise is bogged down long enough for governance systems to muck things up with regulation?
The other view is that this isn’t severe enough that it won’t generate enough noise or concern for teams to take larger action (e.g. intl slowdown). Faced with competitive external pressures, leaders will decide this is benign and manageable enough that it just requires a relatively small pause/adjustment to security posture.
Something like this has been my expectation since approximately announcement of Devin in spring 2024, with a major caveat: policymakers won’t push for economically costly measures until some people die from misaligned AIs (and I don’t mean suicides), but the issue is certainly unsolvable with “cheap” measures, meaning people will have to die, and that’s still not a guarantee =(
This seems like excellent news from the doomer perspective? To get a test case like this, with complex misaligned behaviors, coordination, verbalized defiance of user intent… It’s all so blatant, so obviously problematic, and it didn’t (yet?) harm anyone.
It seemed possible that the train would be farther down the tracks before people noticed the bridge is out.
What if the immediate road ahead is unsafe in mundane ways, full of non-civilization-scale security failures, and the enterprise is bogged down long enough for governance systems to muck things up with regulation?
The other view is that this isn’t severe enough that it won’t generate enough noise or concern for teams to take larger action (e.g. intl slowdown). Faced with competitive external pressures, leaders will decide this is benign and manageable enough that it just requires a relatively small pause/adjustment to security posture.
Something like this has been my expectation since approximately announcement of Devin in spring 2024, with a major caveat: policymakers won’t push for economically costly measures until some people die from misaligned AIs (and I don’t mean suicides), but the issue is certainly unsolvable with “cheap” measures, meaning people will have to die, and that’s still not a guarantee =(