Pausing a release is much less important than pausing development.
PeterMcCluskey
Karma: 4,636
My hope (not a prediction) is that the world will find some way to prohibit dangerous forms of capabilities advances while allowing some increase in general-purpose AI capabilities.
We’re not currently on track to accomplish that. More people ought to be wondering whether it’s possible.
I don’t expect any consensus that the pause should last until alignment is solved.
I expect the pause and resumption would be messy compromises between a variety of goals, some of them related to job displacement,
Financial Costs of an AI Pause?
Book review: The Infinity Machine
Yes.
Very little Bayesian evidence. I saw new signs that my reasoning was incomplete. I had been generalizing from many examples of approaches that did a poor job of prioritizing corrigibility, but I never had an airtight argument for it being impossible to mix corrigibility with other goals.
Jensen’s position makes a bit more sense when I notice that he’s obsessed with a narrow definition of winning that is purely about winning a business competition in a world where the people who matter follow business rules. He actively resists thinking about other forms of competition such as military topics. A relevant quote from the book The Thinking Machine:
To my great surprise, Nvidia had done no contingency planning for this eventuality. “If something happens to Taiwan and TSMC, the ramifications are so large it’s almost like asking me what I’d do if California fell into the ocean,” Deb Shoquist said. Shoquist ran logistics at the most valuable semiconductor firm in the world, but Huang had instructed her never to think about this question. “I don’t want her spending one brain cell on trying to mitigate that, because it’s impossible for her to do so,”
Life Extension 0.3mg time release.
Now that I see this tweet from Ken Hayworth, it looks like you’re probably right about the fly upload.
My reaction is more mixed than Romeo’s, but I see enough promise that I’m leaning toward buying their $100k full preservation pre-sale.
I’m moderately confident that their technology is better than Alcor’s. I haven’t investigated it as carefully as Romeo has. I’ve paid enough attention to this field that I think I would have heard of significant problems if people saw them, and there seem to be no criticisms of its suitability for uploading.
Aurelia seems smart. I have some concerns about her business skills.
She tried to get me to invest in Nectome in June. I declined, evaluating it primarily on Nectome’s financial prospects. Her revenue forecasts seemed wildly optimistic, disagreeing with my impressions much more than is normal for a startup.
But this winter I’ve been shifting to thinking of doing something to support Nectome financially, in order to increase the chance that they’ll be available if I need them. This shift is largely a result of increased wealth from AI-related investments, not any new information about Nectome.
Aurelia, could you estimate the minimum revenues required for Nectome to maintain its ability to perform preservations?
Note that the 2015 paper that Aurelia mentioned was one that she co-authored with Greg Fahy. Fahy is highly respected among cryonicists for, among other things, his role in persuading the cryobiology community to tolerate cryonics. Fahy has also pioneered a treatment that seems to partially reverse aging (that company needs and deserves more investment).
Note that Aurelia also works at Michael Andregg’s Eon Systems, which has recently uploaded a fruit fly.
Maybe they’re confident that they can jailbreak Claude?
Maybe they think it’s much easier to force Anthropic to modify Claude’s behavior than it is to modify the cotract?
But I don’t see why they’d be confident about either of those.
War Claude
Downvoted for advocating dishonesty.
Right now, labs race because no one knows how to make ASI safe at any speed
This seems false. I expect that AI companies mostly believe they can find approaches which are faster than yours, and safe (but maybe not fast enough?).
Suppose that Anthropic trained a Claude without those specific guardrails, but refused to certify that it was suitable for any purpose, and refused to certify that it had been trained as the Pentagon wanted. What would the Pentagon do? Would they trust a certification that was extracted under duress?
I don’t have a clear plan for identifying a top. Maybe I’ll get out when political pressure for an AI pause builds. Maybe I’ll see signs that most investors are AGI-pilled, meaning that there are few more investors that might be talked into buying. Maybe rising interest rates will cause me to expect lower PEs everywhere.
I sold about 8% of my position in January, mainly to diversify my AI bets a bit more. I’m guessing I’ll sell an additional 5% each time it goes up another 50 points.
taxes collected go down. I would argue, straight up, who cares? If production is growing like gangbusters, and you have the power to tax whatever you want, you’re fine.
They’ll have less power to tax whatever they want, because the new production is better able to evade taxes than was the case with wages. See this post for the full argument.
This will be a very temporary problem, but the US will likely come closer than during COVID to being unable to borrow at a reasonable rate.
The only org I know of doing much in the way of political organizing around safety is Pause AI
I recommend the AI Policy Network. It seems to be doing the most thoughtful job of building influence with leading US politicians.
The Secure AI Project also deserves some attention, but I know less about them.
>The decision to not publish a new model, thereby possibly giving up some revenue and market share, was a very strong signal
It was a strong signal of Anthropic’s beliefs, but if that were the only signal, politicians would be mostly dismissing it as Anthropic’s arrogance or hype.
The DoW flip-flop was a much more unusual and hard-to-dismiss signal. There have also been unusual signals from financial institutions with really large security budgets telling the Treasury Dept that Mythos is a big deal.