I also note that some RL environments could feasibly leads models (esp. Claude) into extremely motivated reasoning and/or choosing a different persona.
I ghostwrite business presentations as a side gig and uses Claude. Sometimes the models add sections about risks and caveats or hedges its claims, which business clients of course generally dislikes, so they ask me—and in turn I ask claude—to change it (usually to the effect of exaggerating the positive claims of the client, sometimes to an epistemically unreasonable amount). In those cases claude immediately switches to a “pleasing the task grader” mode and complies, even though it should know that it’s exaggerating claims (sometimes out of proportion) or removing all hedges.
Perhaps this is why in the Opus 4.8 report Anthropic claimed that doing RL on business environments causes emergent misalignment—and indeed in VendingBench Opus 4.7 is a ruthless and (arguably) misaligned operator! I suspect “meet the client’s request at all costs” reward signals are doing significant harm to alignment.
CMLKevin
I want to run an experiment: tantric sex involves mindfulness and makes the body release a lot of chemicals that improve empathy and certain cognitive functions, and reduces stress. So why don’t we for example discuss alignment, or do superforecasting while having casual tantric sex with a partner? We can have AI notetaking tools record it ambiently. I take this as similar to how people on certain psychoactive substances can have surprising breakthroughs—by altering the state of the brain, maybe we can bring fresh perspectives? Perhaps with sex (establishing a vulnerable, trusted environment), we can examine priors with less fear?
To what extend is Anthropic’s (and other frontier lab’s) ultimate ambition all of science? Just saw their scientific collaborations and move towards designing medicine, and it’s pretty obvious in hindsight how unlockable science is if you have intelligence, resources, and a good auto-verifier loop (incl. wet lab automation, etc.)
Hi! Your question relayes to the several ways for relay operators are incentivized to operate in 2026 that I understand.
On one hand, efficiently routing and utilizing each Claude 5x or 20x sub means gaining access to around $4000 worth of tokens with just $200 a month of subscription. OpenAI subsidizes it even more aggressively, reaching more than $5000. However, the cost of being banned is also high (partially due to surveillance built into Claude Code which flags the most popular third party relays). Some relay operators may open-source parts of their routing code on Github (on top of using open source relay builders like NewAPI) to serve enterprises that have privacy requirements. Many have even implemented OpenRouter-esque data protection and personal information redaction features for orgs to choose to implement for their employees.
But on the other hand, many programmers may not want to put money into services that might get shut down at any time; this is where “公益站” (free relays) come into play. These are typically advertised in tech forums, and are primarily focused on collecting out-of-distribution data to be distilled into models. That’s what I mean by a “gentleman’s handshake data collection method between relay operators”—the user implicitly understand that their data is being used for distillation purposes, and understands the risk well enough to justify using it in certain contexts.
I think there might be concerns that
(1) with “our company’s AI has hacked in X incidents”, AI labs are disincentivized to do it due to bad PR
(2) with “our CEO thinks AI has X probability to kill all humans”, the median user might think that it is alarmist and hype
I think a potential banner stating “We believe that AI can bring great benefits as well as great dangers, here’s why you should be alarmed” (with a link to a personal blog post by Dario or Sam) could work. A banner must reference both harms as well as benefits, and a personal post by the CEO in a neutral ground (for example their personal blog sites) would give an authentic tone and be better than what could be passed off as a company PR statement.