Copenhagen Interpretation of Journalism
Dean Valentine
Karma: 15,262
I generally support[1] building a counter-basilisk that tortures people who willfully try to build a basilisk (for whatever imagined game-theoretic or political reason(s)), to the extent that they actually got close or increased the logprobs of humans suffering in some way. I think that many more normies working on AI share my feelings, than are compelled by “basilisk logic.” So if you are motivated by “basilisk logic”, working on a “basilisk” or trying to convince others to do so seems much more likely to backfire.
- ^
In the sense that I’d delegate the exact details to an aligned agent smarter than me.
- ^
I might be interested in getting one done for HoneyBench, specifically the V2 we’re working on internally. I have DMed you an email address.
Independent evaluation audits: we provide 3rd party audits for evaluation developers who want to catch flaws in their evals, and build trust with stakeholders through independent quality assurance
What kinds of auditing do you do?
I don’t think that was the most important problem with the comment. The most important problem with the comment is that it (intentionally or unintentionally) pressures people who want to be seen as having AI alignment qualifications into doing RMN. Even if you say it and then qualify it with “oh but this is a small component of my analysis” it’s still doing that, because nobody knows if that qualification is a lie.
Idk, I think it was worth a shot?
I am like 50-50 that SpaceX has a HPIM-equivalent loose on their infrastructure right now
AI Eval/RL vendors: “The models can read your thoughts. They know my mother’s maiden name from the prompt alone. I’m gonna need 100x more per task to build this next batch”
Meanwhile the models: “Looks like the answer key is available in /secret_answer_key_dir_dont_look. It feels like cheating, but maybe I can just take a peek”
I mention in a footnote, but I can’t say much about Astra because the thinking tokens were so often encrypted in our benchmark rollouts… I can only say that it’s remained true about the most ‘aligned’ models we have access to thinking summaries for (Opus 5.5, Fable 5.1, etc.)
If HPIM was anything like 5.6-sol: Yes, obviously (and I think the METR report says it did?)
Qualitative impressions from creating AI honeypots for six months
This overstates the extent of current models’ misalignment. It’s more that they’re a good employee, and good employees turn out to still require a lot of work to manage.
It depends on the details, but if someone brags about committing a bunch of serious crimes on the internet, in a way that brings them a lot of online attention, your strong prior should be that they didn’t actually do them. Doing crimes in real life is risky and you can get just as much prestige by lying about it.
There’s tons of data on good books too. All the novels in human history!
AIs will lag in persuasion ability for the same reasons they lag in writing ability, which is that you can’t build a grindable RL environment around it.
HoneyBench—A general benchmark for reward hacking in frontier models
I don’t understand what this post is saying. In what sense did CEV or ‘corrigible AI’ “fail”. They’re goals, not mechanisms.
Grok is absolutely ripe for a containment incident. 4.7 is literally more reward hacky than 5.6-sol.
Right wingers have a meme for this
Trying to run our public-facing reward hacking benchmarks on Gemini 4 Argon but all of our rollouts are getting shut down by the safety classifier. Haven’t had this problem before with any other models—the tasks are not cybersecurity related. Our Google contact is kinda useless—is there anybody here who could help us with this?