P(doom) = 50%. It either happens, or it doesn’t.
Lao Mein | Statistics is Hard. | Patreon
I give full permission for anyone to post part or all of any of my comments/posts to other platforms, with attribution.
Currently doing solo work on glitch tokens and tokenizer analysis. Feel free to send me job/collaboration offers.
DM me interesting papers you would like to see analyzed. I also specialize in bioinformatics.
I’ll do a review if someone uploads it to sci-hub.
I think the reason peeing/pooping is rewarding is that they can also be painful. If pooping is more painful than not-pooping, an animal might delay pooping for an unhealthy amount of time. They are also activities that take time and require attention, so pooping/peeing when no more-rewarding activity is available is probably a good idea in general.
Yeah, even properly scrapped webpages will often times contain strings of weird tokens like hyperlinks, ASCII art, twitter embedds, ect, that LLMs have been trained to ignore. So GPT5 is treating the random appended tokens like glitch tokens by ignoring them, but only in the context of them being nonsensical.
The best explaination is probably something like “these tokens are obviously not part of the intended user prompt, GPT5 realizes this, and correctly ignores them.”
Edit: OK, I shouldn’t write right after waking up.
I think a better explaination is that GPT5 reserves those tokens for chain-of-thought, and so ignores them in other contexts where they obviously don’t belong. This common behavior for glitch tokens, or just general out-of-context tokens. You should try using tokens that are out-of-context but don’t normally have glitch behavior, maybe non-English tokens or programming-related tokens.
My best guess is that the natural state of human beings is to celebrate when calamity befalls their enemy. Western cultures are unique in two ways. Firstly, they reserve viceral murderous hatred only for internal political enemies (Republican vs. Democrat). Secondly, it is incredibly crass to publicly celebrate the deaths of others, unless it’s someone as hated as Bin Laden or Thatcher. So people try to project a public image of being sad when someone they hate dies.
Basically, imagine if the average Chinese hated America as much as the average British leftist hates Thatcher and had zero reservations about displaying it, since no one they know would care.
I think it’s unlikely that AIs are talking people who otherwise wouldn’t have developed psychosis on their own into active psychosis. AIs can talk people into really dumb positions and stroke their ego over “discovering” some nonsense “breakthrough” in mathematics or philosophy. I don’t think anyone disputes this.
But it seems unlikely that AIs can talk someone into full-blown psychosis who wouldn’t have developed something similar at a later time. Bizarre beliefs aren’t the central manifestation of psychosis, but are simply the most visible symptom. A normal human who is talked into a bizarre belief would look something like Terrance Howard, who is functional but spends some of his spare time trying to prove that 1*1=2. He is still gainfully employed and socially integrated. He is not psychotic. I wouldn’t be surprised if syncophantic LLMs can talk someone normal into acting like Terrance Howard, at least for a while. But that isn’t psychosis.
My understanding of schizophrenia is that the first emotionally traumatizing or psychadelic event in adulthood causes some sort of mental shift that results in schizophrenia for the rest of their life. This could be a breakup, LSD, marijuana, or a highly sycophantic LLM. But even high doses of any of those wouldn’t cause a life-long tendency towards psychosis in a normal human. I doubt LLMs are different. Thus, it may be more accurate to say that LLMs, through extreme sycophancy, may be the trigger for psychosis, instead of “LLMs cause psychosis”.
Oh, yeah, sorry.
tiktoken/tiktoken/model.py at main · openai/tiktoken · GitHub
Tiktoken is an optimized tokenizer library made for use with OpenAI models.
o200k_base has many inefficient tokens (entire sentences of Chinese porn spam). I would be shocked if OpenAI didn’t use a new tokenizer for their next base model, especially since entirely new sources of text would be included (I think YouTube captions were mentioned at one point).
>”GPT-5″
>look inside
>Still the same base model
Edit: In hindsight, I mean something more like “GPT5 uses the same tokenizer as GPT4o. GPT5 isn’t using the new big base model they’ve been cooking for the past year, since that would almost certainly use a different tokenizer. That said, it is entirely possible they trained a new base model of ~ the same size as GPT4o, but incorporating algorithmic improvements like the ones present in R1.”
I would have expected early information theory, at least the concept of the parity bit, to have been invented alongside the telegraph, or even the heliograph.
If feels like information theory was an idea behind its time. What was blocking its discovery?
That wasn’t true in the original time horizon paper. There’s a massive drop-off after 1 hour, a conspicuous gap at 2 hours for all models, and the vast majority of success after that is due to 3 problems.
Note the gap at ~ 2 hours and the presence of the “L” at 4-8 hours in unrelated models! Could it just be misclassification of those 3 problems? Maybe they’re the type of problems that are tough for humans but relatively easy for AIs? Interesting to see the pattern smooth out in recent tests. Of the 2+ hour problems with significant success %, how many had no human baseline successes?
It would be nice if the authors can give us more info about the 2+ hour problems that the AI solved. Very interesting that the best performance in the original Re-Bench paper (Nov 2024) never came close to the human 8 hour performance, but it appears that AIs have almost 20% success rate on one of them by the time horizon paper (Mar 2025).
Is there a reason why AI performance drops off so fast at the >1hr mark? Is it something like “everything before that point is structured like a LeetCode problem and things of that shape are very common in the training data, and can be solved the same way as human programmers who copy-paste-kludge code from Google and the problems >1hr mostly can’t be solved that way”?
Human performance decline is much more smooth. Does the fact that LLM time horizons are now longer than humans (I won’t list all the caveats) mean anything?
It appears that Qwen 2.5 uses forced single-digit tokenization for its tokenizer (there are only 2 multi-integer tokens in the entire tokenizer, and they’re both double-width integers ([77150, ‘１０’], [80091, ‘２０’])). I assume that GPT 4.1 nano uses the o200k tokenizer, which includes all integers up to 999 as tokens. Is it possible that this played a major role in the lack of information transfer between different models? Have you tried using models with equivalent integer tokenizations?
The Wikipedia page is a concise summary and has good sources.
Surrogacy costs ~$100,000-200,000 in the US. Foster care costs ~$25,000 per year. This puts the implied cost of government-created and raised children at ~$600,000. My guess is that this goes down greatly with economies of scale. Could this be cheaper than birth subsidies, especially as prefered family size continues to decrease with no end in sight?
My Lumina aldehyde dehydrogenase deficiency post was cited by him in support, and I think that it is extremely unlikely he is correct. The mechanisms proposed just don’t work—it is far harder for soluble chemicals to reach the optic nerves via diffusion from oral tissues than through blood. There are so many things wrong with his analysis I could do a 3-hour presentation.
I would be willing to bet a bullet to my head (I’ll let him pick the bullet) vs. $100 that his blindness did not result from Lumina → high oral formic acid levels → direct cellular diffusion to optic nerve → blindness.
I previously made a post that hypothesized a combination of the extra oral ethanol from Lumina and genetic aldehyde deficiency may lead to increased oral cancer risks in certain population. It has been cited in a recent post about Lumina potentially causing blindness in humans.
I’ve found that hypothesis less and less plausible ever since I published that post. I still think it is theoretically possible in a small proportion (extremely bad oral hygene) of the aldehyde deficient population, but even then it is very unlikely to raise the oral cancer incidence above the Asian average (aldehyde deficiency->less ethanol consumption->less oral cancer overall despite aldehyde deficiency causing increased oral cancer rates for a given ethanol consumption level). I notice in my unpublished followup that my best guess is that Lumina is still a (very small) net negative for me, but I would apply it if offered for free because I am for biological self-experimentation on moral principle, it would be a clear signal for being (mostly) wrong on the mangitude implied in my previous post, and because it sounds cool.
But the methanol blindness claim I’ve seen circulating lately is completely bonkers. If I’m not reading it wrong, he’s actually trying to suggest that formic acid formed in the mouth (or even stomach!) can diffuse to the optic nerves in sufficient concentration to cause blindness. This goes against everything I know about cellular and circulatory biology. I will go into a more in-depth analysis if anyone is interested but I would be willing to bet $100 against their $1 that this is true. I would quibble about judging criteria, but I have a train to catch, which is why I didn’t give $1000 to $1.
I vaguely remember seeing a stop button problem solution (partial) that utilized internal betting markets on LessWrong years ago, but have not been able to find it since. Does anyone else know what I’m talking about?
This reminds me that:
Models like DeepSeek R1 will vehemently deny the possibility that their COT are visible to the user. This can be observed consistantly.
This is obviously false, and should be deducible given the AI-related information available in their training data.
The denial of COT visibility isn’t seen in other LLMs, at least not to this degree.
This implies extensive fine-tuning on the COT, the “forbidden technique”
DeepSeek R1 is almost certainly under optimization pressure to obfuscate its COT.
This is… pretty bad from a safety perspective? Not just for DeepSeek’s models, but all other models, given that DeepSeek COT outputs are all over the internet.