bhauth
I downvoted because:
-
It starts with an irrelevant AI image. Why? And then the “how hard is AI safety” image is embedded in the middle of unrelated text.
-
It conflates different things under a single word “safety”, eg:
We quickly learned that labs that prioritized speed captured the market as users actively revolted against overly preachy models. Safety shifted from an idealized differentiator to an impediment to market dominance.
While I could be wrong, that also seems like AI-edited text.
It has “12 months” in the title and then has no justification for that particular number.
-
I’m sure it’s possible to write a better version of this post. I hope someone does. Believe it or not, my specialty is engineering, not rhetoric.
My assessment of Sam Altman is that he’s a very good actor, very untrustworthy, and a nihilistic power-seeker who cares very little about benefit or harm to humanity as a whole. I agree that this post alone is only weak support for that assessment. A proper “compendium of reasons not to trust Sam Altman” would probably end up being a considerably longer post.
Because that’s a good way to lose every other agreement I’m part of. If that gets out once, everyone I’m dealing with needs to worry about whether they’ll be the second time. Even if they’ve seen me be trustworthy for years, if I decided they were evil, I might not just leave them in the lurch, but exploit every iota of the trust I’d earned to make them suffer and pay. Who wants to risk that?
Lots of people in leadership positions, from what I’ve seen! Altman. Trump. Bush and WMD in Iraq. Ronald Reagan and Iran-Contra. Lyndon Johnson and the Gulf of Tonkin. American CEOs do that pretty often!
From my point of view, you’re playing an iterated Prisoner’s Dilemma and committing to always cooperate even after the other person defects. OpenAI had a charter and a mission and a board with pro-humanity goals. Altman and his backers broke all of them. He therefore deserves the same in return.
OpenAI leadership broke that implicit contract first. It was originally supposed to be a philanthropic thing for the benefit of humanity. It was supposed to be “open”. Then it became for-profit, now it’s going to work on killer robots for the military. To whatever extent there’s an implicit contract like you describe, it would also apply to the work that people previously did for it under false pretenses!
I disagree entirely. You’re not considering the implications of his argument. There’s a reason why some dunks on him got over 10k likes.
He argued that AI energy usage is fine because all-in it’s less than the energy usage of equivalent humans. That comparison implies that any value of humans apart from their usefulness as workers that could be replaced by AI is negligible.
Your ethical framework here doesn’t seem consistent to me, but maybe you can explain how it works.
Uh, because I’m not a person who screws people over whenever it’s convenient to me.
Doing a job that harms people because you get paid is...also screwing over people because it’s convenient to you.
If it didn’t seem like a valuable option to them, presumably they wouldn’t spend a bunch of money and personal involvement (and a bit of bad optics) on having a bunker.
What I’m saying is, if everything goes wrong somehow, Sam Altman has the option of going to his bunker and playing video games or whatever indefinitely, and I think he wouldn’t feel particularly bad about what’s happening to people outside.
I don’t have the impression that Altman:
is the biological father
sees that kid as his
has been doing parenting
really cares at all
Unlike peptides or other complex organic molecules, lithium is very non-specific. If it’s present at levels high enough to noticeably inhibit a specific enzyme, it also inhibits lots of other stuff a similar amount.
I’d recommend being less credulous about papers on Alzheimer’s, considering how the incentives in that field have been.
But moving the IP would be illegal
While some IP has export restrictions, I don’t think AI weights currently do.
Chip restrictions would likely be a major barrier to relocation
“Frontier companies” do not necessarily run datacenters. A company can move to Europe and use US datacenters.
You have a good point about Transcriptic and “cloud labs”, but one issue is, that model seems incompatible with the current structure of both university labs and drug companies. A university lab today is a barony ruled by a professor, and it does its own research. Labs generally don’t even share reagents, they’ll each buy little bottles instead of sharing a bigger bottle that could be half the total cost.
My impression is, Ginkgo and Automata don’t understand manufacturing, and I don’t think a “cloud lab” would buy their hardware when it’s cheaper to buy generic general-purpose robotic arms. Also, Y Combinator companies in general...these days when I see “funded by Y Combinator” it gives me a similar impression to reading “MIT scientists have discovered”.
About Briefly Bio and Tetsuwan, I personally think they should be sticking to text-based high-level descriptions. Yes, Unreal’s Blueprints and ComfyUI are considered successful, but specifications having version control would be especially helpful for lab experiments. And of course LLMs are better at outputting text than custom GUI programming systems; if you have a good enough high-level description you could probably just translate natural language to it automatically.
Unlike 1, this isn’t completely implausible, but it seems like a very ambitious claim.
Not really. To adjust frequency you just need to 1) adjust air resonance by changing length or 2) mechanical resonance by changing tension. (1) might be too slow here but (2) is not.
I agree with Zac. I also think you’re misunderstanding the physics involved, and you’re underestimating the researchers doing this research.
The mixed codas seem like strong evidence against this view
No, I don’t think they are.
The fact is, the whales can produce multiple clicks with different spectral patterns. This is in quick succession so it’s not a change of orientation. The patterns can vary so it’s not a fixed mechanical thing. Therefore it comes from some mechanism of adjustment between clicks, which could be adjusted by muscles. So your whole thesis is off.
I don’t know what you’re trying to say here.
I’m saying that you’re making a questionable leap from:
Then the alignment team RLHFs the models to follow the spec.
to “the model follows whatever is written in the spec”. You were saying that “current LLMs are basically aligned so they must be following the spec” but that’s not how things work. Different companies have different specs and the LLMs end up being useful in pretty similar ways. In other words, you had a false dichotomy between:
the model is totally unaligned
the model is perfectly following whatever is written in the spec, as best it can do anything at all
That’s not:
a “spec” that’s followed directly by the models in the sense Scott meant
something that OpenAI definitely follows themselves
something that OpenAI models end up consistently following
If AI is misaligned, obviously nobody gets anything.
That depends on how it’s misaligned. You can’t just use “misaligned” to mean “maximally self-replication-seeking” or whatever you actually are trying to say here.
I think there’s also a strong possibility that AI will be aligned in the same sense it’s currently aligned—it follows its spec
Spec? What spec does GPT-5 or Claude follow? Its “helpful” behavior is established by RLHF. (And now, yes, a lot of synthetic RL and distillation of previous models, but I’m simplifying and including those in “RLHF”.) That’s not a “spec”. Do you think LLMs are some kind of Talmudic golems that follow whatever Exact Wording they’re given??
While I stand by the overall point of this post, I wrote it quickly in response to ongoing events, and maybe it isn’t up to my usual standards for writing quality.
Should I take it down and try to write a better version?
Is it better to focus more on specific details of OpenAI, or on engineering ethics more generally and how past conclusions people made about that fit the current AI situation?
Did someone else already write a better version of this, or is someone going to?