The general pattern from Anthropic leadership is eliding entirely the possibility of Not Building The Thing Right Now. From that baseline, I commend Zach for at least admitting that’s a possibility. Outright, it’s disappointing that he can’t see the path of Don’t Build It Right Now—And Then Build It Later, Correctly, or can’t acknowledge its existence. He also doesn’t really net benefits and costs. He just does the “Wow! There sure are two sides. We should do good stuff” shtick. Which is better than much of Dario’s rhetoric! He’s cherrypicked a low p(doom) estimate, but I appreciate his acknowledgement that “Most of us wouldn’t be willing to risk a 3% chance (or even a 0.3% chance!) of the people we love dying.” Correct! I am not willing to! “But accepting uncertainty matters for navigating this complex challenge thoughtfully.” Yes. I have accepted my uncertainty of my loved ones’ survival, and I have been thoughtful, and the conclusion I have come to is that I’m not willing to take that risk.
Tbc this is still a positive update for me on Anthropic’s leadership. To a catastrophically low level. Which is still higher than all other lab leaders.
But it reminds me of this world-class tweet, from @humanharlan, whom you should all follow. he’s like if roon weren’t misaligned:
“At one extreme: ASI, if not delayed, will very likely cause our extinction. Let’s try to delay it.
On the other: No chance it will do that. Don’t try to delay it.
Nuanced, moderate take: ASI, if not delayed, is moderately likely to cause our extinction. Don’t try to delay it.”
The general pattern from Anthropic leadership is eliding entirely the possibility of Not Building The Thing Right Now. From that baseline, I commend Zach for at least admitting that’s a possibility. Outright, it’s disappointing that he can’t see the path of Don’t Build It Right Now—And Then Build It Later, Correctly, or can’t acknowledge its existence. He also doesn’t really net benefits and costs. He just does the “Wow! There sure are two sides. We should do good stuff” shtick. Which is better than much of Dario’s rhetoric! He’s cherrypicked a low p(doom) estimate, but I appreciate his acknowledgement that “Most of us wouldn’t be willing to risk a 3% chance (or even a 0.3% chance!) of the people we love dying.” Correct! I am not willing to! “But accepting uncertainty matters for navigating this complex challenge thoughtfully.” Yes. I have accepted my uncertainty of my loved ones’ survival, and I have been thoughtful, and the conclusion I have come to is that I’m not willing to take that risk.
Tbc this is still a positive update for me on Anthropic’s leadership. To a catastrophically low level. Which is still higher than all other lab leaders.
But it reminds me of this world-class tweet, from @humanharlan, whom you should all follow. he’s like if roon weren’t misaligned:
“At one extreme: ASI, if not delayed, will very likely cause our extinction. Let’s try to delay it.
On the other: No chance it will do that. Don’t try to delay it.
Nuanced, moderate take: ASI, if not delayed, is moderately likely to cause our extinction. Don’t try to delay it.”