It’s mostly a sign that the Situational Awareness Fund was using too much leverage. If you look at an un-levered semiconductor index (SOXX), it’s down ~25% from the peak, but it’s still up nearly 50% for the year.
MichaelDickens
Karma: 3,432
Getting margin called and selling a significant chunk of holdings isn’t that weird. Selling all their public stock holdings is surprising. Maybe they have a lot of private stock that they can’t sell? Presumably they hold a lot of options on public stocks and the article doesn’t say they sold their options.
But the article also says “The firm had been negotiating to sell its stake in Anthropic”.
I just want to surface the hypothesis that known liar Sam Altman could be lying about this. Although seems somewhat unlikely given that it would (probably?) be easy for an OpenAI insider to contradict him.
I think this argument only makes sense if you already oppose capital punishment in the first place (which I do, for the record). Like yes, if it turns out that we could have radically extended the lives of death row inmates and failed to do that, then that would be very bad for the inmates. But the entire point of executing someone is that being executed is very bad for you.
(The rehabilitation and isolation arguments already give no reason to prefer execution over prison.)
I agree it’s unlikely but OpenAI still has a responsibility to prove that it didn’t happen.
At first, it means I’m going to put serious effort into AI safety, probably in the form of advocacy and funding. My local city does not have a PauseAI chapter, or anything similar, so I might start by looking into the value of creating one. I also will start increasing my modest donations to AI safety causes. I currently give a very small amount. I plan on quadrupling that. I won’t yet start burning through my dry powder though.
That all sounds like a good idea to me. Good luck!
Are you saying grantmakers should read Character Disturbance because it will help them recognize grifters when reviewing grant applications?
Email template for writing your representatives: https://pauseai-global.notion.site/an-ai-escaped-its-lab-and-hacked-a-company
Note: Policy-makers give more weight to custom emails than templates, so I’d recommend re-writing the message in your own words, although sending a template is still better than nothing.
Recent examples of misalignment shouldn’t be a big update for an old-school misalignment-worrier, but they do falsify a lot of objections people have made about how this sort of thing would never happen (or if it did then it would be easily fixable, etc.).
This seems like the strongest direct evidence to date that a scaled-up LLM would kill everyone. In this incident, the LLM ignored most of its guidance to solve the problem, grabbing up external resources to do so. The scarier version of this incident looks like “pave over civilization to build more datacenters to work on the problem”.
I interpreted the saying as “play a lot of games without overthinking”, not “play blitz”.
I’ve taught a few people how to play Go, and for the first 20–50 games, players are basically clueless about what they’re even supposed to be doing—they can place stones anywhere, and they have no notion of why they should place a stone in any particular place.* I think it’s better to develop basic intuition by playing rather than by studying what you’re supposed to do.
*I think chess and Go are different in that respect. In chess, it quickly becomes obvious that the correct thing to do is to develop your pieces.
This is the sort of question LLMs are good at answering: https://claude.ai/share/06e3e68f-933e-4e2f-be11-83f036e22947
TLDR: Yes there are a few experiments that have found this result, but only in very short-term studies, so I’m not very confident that it’s a real effect. (I’d say the studies are maybe a 1.5:1 update in favor. My common-sense prior is 66%, so that puts my posterior at 75%.)
True. There’s also some hope that weakly-superhuman AIs will be persuasive enough to convince AI company leaders to stop. (This is one case where it would be a good thing for AI to be superpersuasive, although superpersuasiveness seems concerning in general.)
Yeah I do have some hope that weakly-superhuman AIs will understand the problem and refuse to help us build ASI until we solve alignment.
There’s the !remindme bot for reddit, I imagine LW could implement something similar.
I don’t have a strong opinion on whether the econ 101 arguments are the strongest consideration, but I weakly believe the government should ban labor unions in the same way it bans price-fixing collusion in general.
One of my favorite things about LW that you don’t see on other similar-ish forums is that people regularly leave comments on years-old posts.
My feeling is that it’s the other way around: the reason the past predicts the future is because Occam’s Razor says it should. The simplest worlds are the ones in which the laws that govern the past also govern the future, and simpler worlds are more probable.
The grandparent comment is stating my opinion. I can’t speak for all libertarians. There are some libertarians who say all government intervention is bad. My position (which I think is a popular position on LW, and popular among economists) is that governments should support free markets, which de facto looks like libertarianism most of the time because most government intervention makes markets worse, not better. But there are some circumstances where government intervention is justified, like breaking up monopolies and regulating externalities.
If you’re not giving a lot of attention to risk management, I think it’s easy to say, “Look, our investments are up 200% this year! Think of how much more we could’ve made if we’d used even more leverage!”