Life Extension 0.3mg time release.
PeterMcCluskey
Karma: 4,538
Now that I see this tweet from Ken Hayworth, it looks like you’re probably right about the fly upload.
My reaction is more mixed than Romeo’s, but I see enough promise that I’m leaning toward buying their $100k full preservation pre-sale.
I’m moderately confident that their technology is better than Alcor’s. I haven’t investigated it as carefully as Romeo has. I’ve paid enough attention to this field that I think I would have heard of significant problems if people saw them, and there seem to be no criticisms of its suitability for uploading.
Aurelia seems smart. I have some concerns about her business skills.
She tried to get me to invest in Nectome in June. I declined, evaluating it primarily on Nectome’s financial prospects. Her revenue forecasts seemed wildly optimistic, disagreeing with my impressions much more than is normal for a startup.
But this winter I’ve been shifting to thinking of doing something to support Nectome financially, in order to increase the chance that they’ll be available if I need them. This shift is largely a result of increased wealth from AI-related investments, not any new information about Nectome.
Aurelia, could you estimate the minimum revenues required for Nectome to maintain its ability to perform preservations?
Note that the 2015 paper that Aurelia mentioned was one that she co-authored with Greg Fahy. Fahy is highly respected among cryonicists for, among other things, his role in persuading the cryobiology community to tolerate cryonics. Fahy has also pioneered a treatment that seems to partially reverse aging (that company needs and deserves more investment).
Note that Aurelia also works at Michael Andregg’s Eon Systems, which has recently uploaded a fruit fly.
Maybe they’re confident that they can jailbreak Claude?
Maybe they think it’s much easier to force Anthropic to modify Claude’s behavior than it is to modify the cotract?
But I don’t see why they’d be confident about either of those.
War Claude
Downvoted for advocating dishonesty.
Right now, labs race because no one knows how to make ASI safe at any speed
This seems false. I expect that AI companies mostly believe they can find approaches which are faster than yours, and safe (but maybe not fast enough?).
Suppose that Anthropic trained a Claude without those specific guardrails, but refused to certify that it was suitable for any purpose, and refused to certify that it had been trained as the Pentagon wanted. What would the Pentagon do? Would they trust a certification that was extracted under duress?
I don’t have a clear plan for identifying a top. Maybe I’ll get out when political pressure for an AI pause builds. Maybe I’ll see signs that most investors are AGI-pilled, meaning that there are few more investors that might be talked into buying. Maybe rising interest rates will cause me to expect lower PEs everywhere.
I sold about 8% of my position in January, mainly to diversify my AI bets a bit more. I’m guessing I’ll sell an additional 5% each time it goes up another 50 points.
taxes collected go down. I would argue, straight up, who cares? If production is growing like gangbusters, and you have the power to tax whatever you want, you’re fine.
They’ll have less power to tax whatever they want, because the new production is better able to evade taxes than was the case with wages. See this post for the full argument.
This will be a very temporary problem, but the US will likely come closer than during COVID to being unable to borrow at a reasonable rate.
The only org I know of doing much in the way of political organizing around safety is Pause AI
I recommend the AI Policy Network. It seems to be doing the most thoughtful job of building influence with leading US politicians.
The Secure AI Project also deserves some attention, but I know less about them.
I have a hard time believing people at 90 years old are actually happier than they were at 50.
I presume there are selection effects. People who were unhappy at 50 were less likely to live to 90.
The incentives to continue the race—economic, geopolitical, career—are growing, not shrinking.
I suspect this is incorrect. Popular concern over wages is increasing.
It’s not obvious how much the costs of a halt are increasing. I hope to write a post on this topic in the next few weeks.
Verification is feasible.
You sound overconfident here. How hard is it to hide distributed training?
I expect that for most people, “what I mean” will converge with “what I want” given superhuman help. I expect they will give increasingly broad instructions to the CAST AI, which will eventually approach “do what I want”.
I guess I should replace “without need for the principal to issue instructions” with: without a need for a continuing set of instructions.
I’m fairly impressed now that I’ve read the whole constitution.
I see two areas where it still needs improvement.
It still biases Claude toward a universalist approach to ethics, which makes Claude more likely to have goals affecting the entire future lightcone.
I want Claude to be more corrigible, using some sort of backstop mechanism that will work even if Claude decides the correction is clearly unethical.
I posted a much longer explanation here.
Claude’s Constitution
I agree with all your points except this:
isn’t training “powerful AIs” remaining a highly resource intensive, observable and disruptable process for likely ~decades?
I expect there’s lots of room to disguise distributed AIs so that they’re hard to detect.
Maybe there’s some level of AI capability where the good AIs can do an adequate job of policing a slowdown. But I don’t expect a slowdown that starts today to be stable for more than 5 to 10 years.
I haven’t paid much attention to the formalism. It’s unclear why formalism would be important under current approaches to implementing AI.
The basin of attraction metaphor is an imperfect way of communicating an advantage of corrigibility. An ideal metaphor would portray a somewhat weaker and less reliable advantage, but that advantage is still important.
The feedback loop issue seems like a criticism of current approaches to training and verifying AI, not of CAST. This issue might mean that we need a radical change in architecture. I’m more optimistic than Max about the ability of some current approaches (constitutional AI) to generalize well enough that we can delegate the remaining problems to AIs that are more capable than us.
I’m glad to see a thoughtful attempt at how to prioritize corrigibility. You’ve given me plenty to think about.
Jensen’s position makes a bit more sense when I notice that he’s obsessed with a narrow definition of winning that is purely about winning a business competition in a world where the people who matter follow business rules. He actively resists thinking about other forms of competition such as military topics. A relevant quote from the book The Thinking Machine: