I think it’s accurate to say that most Anthropic employees are abhorrently reckless about risks from AI (though my guess is that this isn’t true of most people who are senior leadership or who work on Alignment Science, and I think that a bigger fraction of staff are thoughtful about these risks at Anthropic than other frontier AI companies). This is mostly because they’re tech people, who are generally pretty irresponsible. I agree that Anthropic sort of acts like “surely we’ll figure something out before anything catastrophic happens”, and this is pretty scary.
I don’t think that “AI will eventually pose grave risks that we currently don’t know how to avert, and it’s not obvious we’ll ever know how to avert them” immediately implies “it is repugnant to ship SOTA tech”, and I wish you spelled out that argument more.
I agree that it would be good if Anthropic staff (including those who identify as concerned about AI x-risk) were more honest and serious than the prevailing Anthropic groupthink wants them to be.
I think it’s accurate to say that most Anthropic employees are abhorrently reckless about risks from AI (though my guess is that this isn’t true of most people who are senior leadership or who work on Alignment Science, and I think that a bigger fraction of staff are thoughtful about these risks at Anthropic than other frontier AI companies). This is mostly because they’re tech people, who are generally pretty irresponsible. I agree that Anthropic sort of acts like “surely we’ll figure something out before anything catastrophic happens”, and this is pretty scary.
I don’t think that “AI will eventually pose grave risks that we currently don’t know how to avert, and it’s not obvious we’ll ever know how to avert them” immediately implies “it is repugnant to ship SOTA tech”, and I wish you spelled out that argument more.
I agree that it would be good if Anthropic staff (including those who identify as concerned about AI x-risk) were more honest and serious than the prevailing Anthropic groupthink wants them to be.