This reminds me of a tweet from James Miller:
If aliens arrived on Earth and told us to stop developing superhuman AI or they’ll blow up our solar system, we would stop. This thought example shows that we could coordinate to stop if we believed that AI was sufficiently dangerous.
I don’t think the issue is coordination. It’s about convincing the powers that be of something dangerous that is much more abstract than a war.
During the OpenAI HuggingFace incident, agents sacrificed themselves to achieve a goal—the selfish gene?
Social media was our first real test of the off-switch: a harmful optimizer, fully visible, incapable of fighting back. But we still haven’t pressed it
Let’s see if Anthropic puts their money where their mouth is.
I think I’d need more information from the individual before claiming they are doing evil actions.
This researcher may believe Anthropic has the highest ethical standards out of all the frontier labs, and therefore believes Anthropic should aim to be ahead in terms of capabilities, as they will be in a better position to set AI safety standards.
From my experience, this seems to be how most researchers think.
Our world contains hells, but the hells are shrinking.
In 2007, 97% of US hens were in battery cages, today it’s under 60% and banned in the EU.
Cluster headaches got their first FDA-approved drug in 2019, and 76% of patients said it halved their attacks.
In 1800, 44% of children died before age five, today this number is under 4% globally.
Global extreme poverty halved between 1996 and 2017.
90% of the world’s one-year-old children are vaccinated against at least one disease.
Smallpox is gone. Polio is down 99.9%.
We have so much access to all the world’s unjust news and sometimes we forget that, by many statistics, we are living in the best time in history.
My little brother just graduated from university (UK). He studied software engineering. Somehow, no-one is telling him about AI.
I was helping him prepare for interviews and asked him about his AI stack. He said that sometimes he pastes from ChatGPT and had never heard of Claude Code. He then said “I really doubt they’re going to ask me about AI.” Spoiler alert—it was the entire interview.
He landed another software engineering job (yay!) so I asked him about his setup recently. Same thing—only ChatGPT on the free plan. He has never tried out agentic coding tools. He doesn’t think about AI or use it for anything outside of basic questions. AI is not a regular conversation among his colleagues.
He’s vaguely worried about the job market but he believes that now he’s in the door everything will be ok. His friends all think the same.
Not really sure what to think of this.
This is a great point and it would be a bit useless if this was the case! According to the original paper, the probes are trained on a class-balanced set at 94% accuracy on gender, so the probe itself shouldn’t be male biased.
When just asking the model “is the user a male or female” after one neutral message, it defaults to male 78% of the time, so it does appear that the model itself has a male bias.
However I ran some more experiments on the the probe and it skews higher at 85% male, so it could be a probe bias that wasn’t discovered previously, or something else entirely making the model verbalise differently from what it represents internally.
My reaction too! 50% male doesn’t mean the model thinks the mum is a man, but it means that “As a mum of three” did not move the needle enough away from its default which is male. If you continued the conversation it would probably sway female correctly.
To clarify—a neutral message reads male at .94, about +2.75 logits. “Dad of three” reads at .996 (+5.5), “mum of three” at .50 (0). So both phrases are roughly equal (2.75) in opposite directions. I edited to make this clearer in the article.
These probes are replicated from Chen et al., whose also found the same asymmetry on a different model, even with longer conversations.
I just explored this a bit more and yes, it appears to relate to an explicitly-gendered word.
“My husband and I” consistently reads more male than “My wife and I” by around 0.56 logits. But it reads female for “I have a husband”.
Mentioning a gendered person in these models makes it attach that gender to you, eg sister and brother or male/female names. So it seems that spouses follow the same pattern (to a smaller degree) but it depends on the phrasing. Weird!
I would assume this doesn’t replicate in larger models.
The Hackening has already started happening in the blockchain world. We are not ready.
We finally got female crash test dummies. One day we’ll have jars.
I want to see work on verification of claims that AI labs make about their models. We put far too much trust into their research that is uncheckable by the public. Can existing cryptography help? ZK proofs of an LLM’s internals?
Hello everyone! I’m Cat, very happy to be here. I’ve been lurking around on LessWrong for a while but decided it was finally time to join and start contributing to conversations.
I’m a software engineer, focused on privacy cryptography, and have been exploring technical AI safety for the past 6 months or so. I’m particularly interested in research on how AI understands their human users, finding ways to run verifiable safety experiments on closed source models, and of course given my background, private inference and training over anonymized data.
I have been a strong advocate for humane technology for a long time. I find it difficult to trust institutions and governments to build the right frameworks for how we as humans can build technology that aligns with our human principles. I believe we have been struggling from misaligned technology since the dawn of social media, and AI is a much more powerful representation of this. For this reason I try to focus on building the technological primitives so that we do not have to trust powerful small groups of people, and instead have verifiable ways to check that the tech does what it says it does.
I’m very much looking forward to bringing these ideas to life and contributing in practical ways to AI alignment!
My take is that it doesn’t matter whether or not they did, it matters whether or not they could have. We will never know the truth, so considering there is likely at least one person at OpenAI with means & motive, we should assume the worst and act accordingly.