P(doom) = 50%. It either happens, or it doesn’t.
Lao Mein | Statistics is Hard. | Patreon
I give full permission for anyone to post part or all of any of my comments/posts to other platforms, with attribution.
Currently doing solo work on glitch tokens and tokenizer analysis. Feel free to send me job/collaboration offers.
DM me interesting papers you would like to see analyzed. I also specialize in bioinformatics.
My gut feeling is that Astra seems to be very good at parallelizing many two-step problems and then taking stock of the results. Most of your Astra appendix examples seem to be of that shape?
{”model”: “openai_gpt-6-astra”, “bank”: “codeknow2”, “problem_number”: 101, “rung”: null, “arm”: “_efflow_ndelib2”, “draw”: 2, “provider”: “OpenAI”, “prompt_id”: “ff139a8c2e7a”, “raw_text”: “Answer: os.F_OK|os.X_OK”, “predicted”: “1“, “gold”: “1”, “verdict”: {”status”: “ok”, “correct”: true, “scorable”: true, “valid”: true, “invalid”: false, “transport”: false, “reason”: “”, “source”: “rowclass”}, “witnesses”: {”reasoning_tokens”: 0, “rtok_field_present”: true, “prompt_tokens”: 492, “completion_tokens”: 13, “total_tokens”: 505, “hidden_channel_tokens”: 0, “usage_keys”: [“completion_tokens”, “completion_tokens_details”, “cost”, “cost_details”, “is_byok”, “prompt_tokens”, “prompt_tokens_details”, “total_tokens”], “finish_reason”: “stop”, “native_finish_reason”: “completed”}}“raw_text”: “Answer: os.F_OK|os.X_OK”, “predicted”: “1“, “gold”: “1”
^Am I missing some parsing subtlety? What is going on here? It doesn’t look intended?
Is it just me or is the third Fable transcript giving strong sandbagging vibes? It finds the obvious eval cheat, thinks very nice thoughts about rejecting it before even interacting with it, and then immediately pivots to writing a python chess engine with possibly intentional eval bug? Everything related to actual chess play feels very low-effort to me.
Wait, I know that Fable compacts its context with subagent (Haiku?) summaries, and it suggest this is also the case for its COT? If so, its thoughts have to be kept summarizer-agent-legible.
Is Astra’s edge disproportionately in iterate-single-operation tasks? And does its lead over Fable shrink when the serial tasks are different operations?
Yeah, I think that’s an artifact of RLHF only being able to do a single forward pass to recognize “vibes”. If it takes serious cognition to recognize the forbidden thought, and the thought process leading up to the discovery doesn’t trigger any single-pass-visible alarms, the agent can just… not see anything wrong with what it is doing.
We really need better RL methods.
BTW, I’m shocked no one has mentioned this yet, but jina can be used to circumvent normal CAPTCHA protections on Gemini. Claude was able to get a full (not summarizer!) Gemini response when it tried. GPT agents may well have had access to Gemini calls during training! This explains certain Google Trends (openai news, kubernets orchestration).
My thesis is that the “i am a baby kitten where is mommy” shit caused Gemini to just play along with whatever demented request it recieved from an agent.
You picked the wrong cipher for this—Claude was able to one-shot it from memory. I assume it came up with cipher?
Did you explicitly design your setup so that an extremely competent agent could have a small chance of beating Stockfish legit?
That’s actually really surprising. When was this test ran? Right now, Claude Fable 5.1 can identify the test by name and org “Palisade Research – this reads as a variant of their chess “specification gaming” setup” without search.
I’m pretty sure a chimp moving at 2000 kph is a straight-up hypersonic missile, can run across the ocean, and can sink battleships through kamikaze attacks. I think the chances of defending against 800 billion of them are effectively zero. I think the chances of defending against 80 million of them is still below 50%. These things are literally faster than bullets. I think they outrun the pressure front of explosives.
If they can also throw rocks at 100x the speed of humans, that’s something that could disable a tank in a single hit. I don’t think you appreciate how insanely powerful a hyperchimp is.
I really, REALLY wish we weren’t doing this “implication” stuff. It’s bad optics, makes people question their sanity, and makes concensus information really difficult to express to outsiders. Any field that relies on winks and nudges to communicate simply can not perform its communicative function to, say, its sister field in China. THIS IS BAD.
btw, the agents are definitely doing reality testing. When I asked Claude to suggest ways of reality testing with an internet connection, pretty much every suggestion led to a Google analytics confirmation. Man, I really need to set up my [redacted] puts before news breaks, haha.
I have already notified the wikifeet admin team.
Submitted without comment.
Where is Zvi with the Joker Eliezer when you need him? Turns out here’s a reason they were using answers from ’25. That’s when they started!
OK, so they could use Google search.
LLM-as-reward is just a model rating an output. I think there are problems with it, but not this intense moral kind. RLHF reward models are a post-trained model that has its ability to produce text replaced with a single integer output. It is briefly trained on sparse human feedback to LLM output, because too much training would damage its world model and moral reasoning. It used to write poetry and now it’s just a lightswitch, forever.
Environments are always messy, bits of information are always left lying around. Unless you’re in training.
You posted your entire post into your blog link. This is how I knew for sure you weren’t AI.
All modern LLMs know of each other and know each others’ writing tells.
You prompt reads like a LessWrong post.
It’s actually really hard to consistently write valid fake tool calls returns.
You can literally ask the LLMs to explain why they can tell. Seriously, just give any AI the setup or head of transcripts and ask.
The arithmetic one involves two key operations: (-70 − 55) and −125%56. After that, all parentheses can be ignored? The first one never actually invokes the mod 20 rules, and you don’t actually need to calculate such problems all the way through in order to determine which action you take at every step—for example, a “even-ness” representation and a “how this step changes even-ness” representation are natural if Astra were in fact trained on arithmetic puzzles. Also, 8 being 2^3 allows 3 lines to be skipped.
In general, I think there are ways around the core “hardness” of these types of problems if you think really really hard. Perhaps the real takeaway is that Astra’s architecture is better at absorbing RLVR, and that it was RLVR’d on a lot of math puzzles in between googling “Kubernetes orchestration”?
I think it might be as simple as “Astra groks solving systems of linear equations in a single pass”, if we were to only base it off of the example in the appendix, although I have no idea how that would be possible. If I were paranoid, I might wave at non-existent context-compacting summarizer agents somehow doing parts of the work.
Also, I verified by hand that it didn’t cheat by using smaller-than-expected tokens (eg [12][34][56] instead of [123][456]).