Thanks for explaining your thoughts on AI safety, it’s much appreciated.
I think in general when trying to do good in the world, we should strive for actions that have a high expected value and a low potential downside risk.
I can imagine a high expected value case for Anthropic. But I don’t see how Anthropic has few potential downsides. I’m very worried that by participating in the race to AGI, p(doom) might increase.
For an example pointed out in the comments here by habryka:
I mean, didn’t the capabilities of Claude leak specifically to OpenAI employees, so that it’s pretty unclear that not releasing actually had much of an effect on preventing racing? My current best guess, though I am only like 30% of this hypothesis since there are many possible hypotheses here, is that Chat-GPT was developed in substantial parts because someone saw or heard about a demo of Claude and thought it was super impressive.
Could you explain to me why you think there are no large potential downsides to Anthropic? I’m extremely worried the EA/LessWrong community has so far only increased AI risk, and the creation of Anthropic doesn’t exactly soothe these worries.
PS: You recently updated your website and it finally has a lot more information about your company and you also finally have a contact email listed, which is great! But I just wanted to point out that when emailing hello [at] anthropic.com I get an email back saying the address wasn’t found. I’ve tried contacting your company about my worries before, but it seems really difficult to reach you.
Thanks for explaining your thoughts on AI safety, it’s much appreciated.
I think in general when trying to do good in the world, we should strive for actions that have a high expected value and a low potential downside risk.
I can imagine a high expected value case for Anthropic. But I don’t see how Anthropic has few potential downsides. I’m very worried that by participating in the race to AGI, p(doom) might increase.
For an example pointed out in the comments here by habryka:
Could you explain to me why you think there are no large potential downsides to Anthropic? I’m extremely worried the EA/LessWrong community has so far only increased AI risk, and the creation of Anthropic doesn’t exactly soothe these worries.
PS: You recently updated your website and it finally has a lot more information about your company and you also finally have a contact email listed, which is great! But I just wanted to point out that when emailing hello [at] anthropic.com I get an email back saying the address wasn’t found. I’ve tried contacting your company about my worries before, but it seems really difficult to reach you.