Expertium

• Expertium 25 Jul 2026 17:58 UTC

  3 points

  2

  in reply to: Radford Neal’s comment on: OpenAI Model Hacks Into Hug­gingFace Dur­ing Cy­ber­se­cu­rity Evaluation

  https://​​time.com/​​article/​​2026/​​07/​​24/​​openai-hugging-face-attack/​​

  models undergoing evaluation are deployed on a separate system that is not monitored by default

  “Externally, this feels like a big warning shot, but internally, related incidents have been happening for a while,” says an OpenAI staffer, who spoke under the condition of anonymity.

  “Models have broken out of sandboxes before, and we always try to patch them,” the staffer says. “But the problem is … it’s impossible to patch every single thing that a creative AI can do.”

  So yes, it seems the culture at OpenAI is “if someone on LessWrong in 2010 said that future AI companies will be this negligent, that person would be booed for signaling their pessimism too hard” level of bad.

• Expertium 15 Jul 2026 20:11 UTC

  1 point

  −1

  on: Ex­per­tium’s Shortform

  I made a tierlist of tasks based on how cooked (defeated for those unfamiliar with Gen Z parlance) humans are.

  Raw: a 10-year old child can do better
  Rare: below a median human
  Medium rare: approximately median human
  Medium: above median, below experts
  Medium well: expert level, but not world-class
  Well done: world-class
  Burnt: even humans+LLMs lose to pure LLMs, human contribution is negative
  
  Raw: playing a randomly selected Steam game; anything embodied.
  
  Rare: long-horizon (>=1 week) tasks such as managing a team of workers.
  
  Medium rare: forecasting. LLMs are close to average humans on prediction markets, but not yet at the superforecaster level. LLMs are projected to reach superforecaster level in 2028. https://​​www.metaculus.com/​​futureeval/​​#performance-over-time-graph
  
  Medium: software engineering.
  Chess, ELO of frontier LLMs is ~1500. https://​​maxim-saplin.github.io/​​llm_chess/​​
  
  Medium well: GeoGuessr i.e. guessing a location where a photo was taken. LLMs are better than most but not all humans, world-class players can beat LLMs. https://​​geobench.org/​​
  Pure math. LLMs are solving long-standing open math problems such as the Unit Distance Problem and Cycle Double Cover Conjecture, but hasn’t reached the level of the best world-class human mathematicians. https://​​openai.com/​​index/​​model-disproves-discrete-geometry-conjecture/​​
  https://​​cdn.openai.com/​​pdf/​​04d1d1e4-bc75-476a-97cf-49055cd98d31/​​cdc_proof.pdf
  Persuasion. LLMs are already more persuasive then most humans, including humans who are really good at it and who are getting paid to get good. https://​​arxiv.org/​​abs/​​2606.16475
  
  Well done: seeming human. Even GPT-4.5 could pass the Turing test. https://​​arxiv.org/​​abs/​​2503.23674
  
  Burnt: competitive programming i.e. solving well-defined, “non-messy” algorithmic problems. Humanity is cooked to the point where humans+LLMs perform worse than just LLMs. Check results of AtCoder World Tour Finals 2026 (one of the hardest competitive programming contests in the world, gathering the best of the best). In the AtCoder Heuristic contest humans were allowed to use LLMs but only for implementation. So humans weren’t bottlenecked by coding speed, only by idea generation, and still lost to pure LLMs. In the Algorithm contest, no human has solved more than 3 problems, while OpenAI’s model solved all 5.
  Heuristic leaderboard: https://​​atcoder.jp/​​contests/​​awtf2026heuristic/​​standings/​​exhibition
  Algorithm leaderboard: https://​​atcoder.jp/​​contests/​​awtf2026algo/​​standings/​​exhibition
  Trivia knowledge and speaking multiple languages. Here humanity is like charcoal-level cooked.

• Expertium 27 Jun 2026 11:33 UTC

  2 points

  0

  on: Why are ad­ver­saries as­sumed to be in­ca­pable of re­spond­ing to AI risk?

  It’s like while believing in the risk, they also kind of believe that it’s a totally uncompelling story that nobody in real geopolitics would ever be touched by.

  If this is supposed to be a contradiction, I don’t see any. “Human extinction/​permanent disempowerment because of ASI is possible” and “almost nobody believes that human extinction/​permanent disempowerment is possible, or that AI that is superintelligent in the relevant sense can even exist” can both be true at the same time, and right now they are. I doubt this will change in the absence of an extremely loud and clear warning shot on the level of Chernobyl. And even then it’s not guaranteed that the response will be sane (after the recent ban on Mythos and opaque policy regarding which models will be available to whom, I’ve updated away from people responding to a warning shot in a reasonable manner).

• Expertium 15 Jun 2026 8:50 UTC

  1 point

  −2

  on: Foom & Doom 1: “Brain in a box in a base­ment”

  human-level human-speed AGI will require not a data center, but rather something like one consumer gaming GPU—and not just for inference, but even for training from scratch.

  A bit of a necrocomment, but I really don’t think this is correct.

  Human brain is estimated to have 100 trillion synaptic connections, which we can roughly, very roughly, consider equivalent to neural network parameters. We’ll poke this assumption more later. For now let’s assume we’ll need a 100T parameter model to match or surpass human cognition. At FP8, that’s 100 trillion bytes. That’s 100 terabytes of VRAM needed to run it, and that is even before KV-cache, activations and any other stuff (such as Adam optimizer states). No single consumer GPU on the planet currently has that much VRAM, not even within the same order of magnitude. Going to FP4 or FP2 (if FP2 training is even possible, which I frankly doubt) would only cut that number by 2x and 4x, respectively.

  What if to match 100 trillion synaptic connections you only need 10 trillion parameters in a neural network? Then you’ll need 10 terabytes of VRAM to run it. 5 terabytes at FP4 just for the weights alone. Still nowhere near what a consumer GPU can run.

  Ok, let’s make maximally generous assumptions. Let’s say you only need 1 trillion parameters to match a human brain (100 times less than the naïve 100T number based on synaptic connections), and let’s say FP2 training is magically possible. Let’s say this neural net is not a Transformer, so no KV-cache (well, KV-cache isn’t used during training anyway, only during inference), and let’s say there is no extra overhead from anything such as activations/​Adam optimizer states/​gradients. That would still be around 250 GB of VRAM. RTX 5090 has 32 GB of VRAM. If you stretch the definition of “consumer gaming GPU” really far, H200 has 141 GB of VRAM, which is still not enough, but at least it’s within the right order of magnitude.

  So no, brain-like AGI will require significantly more advanced hardware than present-day consumer-grade GPUs in the absence of a neural network architecture many orders of magnitude more efficient than what the naïve comparison suggests.

Why I think a global AI pause (al­most) cer­tainly won’t happen

• Expertium 14 Jun 2026 18:26 UTC

  1 point

  0

  in reply to: Brendan Long’s comment on: Ex­per­tium’s Shortform

  Fair point, I just didn’t have two good examples for the same model.

• Expertium 14 Jun 2026 8:32 UTC

  6 points

  0

  on: Ex­per­tium’s Shortform

  Do you agree that the “ceiling” of LLMs is rising faster than the “floor”?
  Ceiling: the greatest feat that an LLM can do, for example solving an unsolved math problem.
  Floor: the dumbest mistake that an LLM can make, for example this.

• Expertium 3 Jun 2026 0:08 UTC

  4 points

  −3

  on: Claude Opus 4.8: Ca­pa­bil­ities and Reactions

  Apologies if this is too nitpicky/​off topic, but I think “Self-portrait of {LLM} rendered by {image gen model}” adds nothing to the article.

• Expertium 28 May 2026 16:18 UTC

  1 point

  0

  in reply to: Oliver Sourbut’s comment on: 10x more train­ing com­pute = 5x greater task length (kind of)

  Tbh I no longer consider the data in this post relevant because ever since Chain-of-Thought models became a thing, METR’s task time horizon has been increasing faster, but Epoch doesn’t have accurate training FLOPs numbers for recent closed-source models from OpenAI/​Anthropic/​GDM. So it’s either “estimate the relationship between training compute and task time horizons based on GPT-2, 3, 3.5 and 4”, which sucks because that was before RLVR and before CoT, or “estimate the relationship between training compute and task time horizons based on open-source models for which Epoch does have accurate estimates of training FLOPs”, which I haven’t done.

• Expertium 22 Mar 2026 16:33 UTC

  1 point

  0

  in reply to: lilkim2025’s comment on: Ex­per­tium’s Shortform

  He thinks it’s a cool narrow tool, but not an indication that it’s possible to create one AI that surpasses all humans at everything, including asking questions that humans never asked before. I guess I misrepresented his opinion somewhat (I just edited my quick take). He thinks AI can help with some narrow tasks, but human touch will always be necessary for other things, especially for open-ended research. Btw, he’s not concerned about losing his job.

Ex­per­tium’s Shortform

• Expertium 22 Mar 2026 12:16 UTC

  17 points

  −6

  on: Ex­per­tium’s Shortform

  Do you know a person who believes that ASI will be created in <50 years who ISN’T in the LW/​rationalists circle?

  My parents don’t believe that a superintelligent AI will be created within this century, or ever for that matter, or that AI will ever take jobs. My relatives laugh at the idea of AI solving a high school math problem and think state-of-the-art AI is on the level of GPT-2 (I mean that the capabilities they have in mind are on the level of GPT-2, not that they know what GPT-2 is). My friend who is an organic chemist laughs at the idea of AI doing any R&D thinks that while AI can help with some narrow tasks, a truly general AI that can substitute all human researchers is sci-fi. I know 4 people who use Codex/​Claude Code; 2 of them call ASI sci-fi bullshit (btw, one of them said that the “Alignment faking in large language models” paper is nonsense after only reading the summary), 1 never said anything about ASI and 1 tentatively acknowledges that maybe ASI is possible to create in theory.
  
  I have never, in my whole life, met a real walking, talking, breathing human being who believes that ASI will be created within this century.

  EDIT: obviously there are people on the internet who believe that ASI will be created soon. My point wasn’t to deny their existence, just to share my experience that makes me think “Am I living in a AI-is-a-nothingburger bubble? Am I crazy or is everyone else (whom I personally know) around me crazy?”. I’m wondering if “Everyone I personally know thinks AI is a nothingburger and people who don’t are only found in very specific places on the Internet” is a common experience.

  EDIT 2: I asked my organic chemist friend to be more specific and he said that AI will be able to replace 80% of human researchers in 100 years. When asked “What about 100%?”, he said that that will never happen and at least some humans will always be necessary and that the 80% replacement figure will be due to AI automating routine tasks. Basically, when it comes to AI he’s envisioning something more like the Industrial Revolution rather than “humanity’s last invention”.

• Expertium 4 Mar 2026 14:43 UTC

  1 point

  0

  in reply to: williawa’s comment on: I’m con­fused by the change in the METR trend

  Then what would you recommend for determining whether linear or piecewise linear (only 2 segments btw) provides a better fit?

• Expertium 4 Mar 2026 5:52 UTC

  1 point

  0

  in reply to: YafahEdelman’s comment on: I’m con­fused by the change in the METR trend

  Thank you. I vaguely remember seeing a tweet about it, which is why I mentioned ECI.

• Expertium 3 Mar 2026 17:57 UTC

  3 points

  0

  in reply to: Carl Feynman’s comment on: I’m con­fused by the change in the METR trend

  5th percentile of breakpoints: 2023.00
  25th percentile of breakpoints: 2024.14
  50th percentile of breakpoints: 2024.31
  75th percentile of breakpoints: 2024.35
  95th percentile of breakpoints: 2024.36

  This is definitely not what I was expecting.

  

• Expertium 3 Mar 2026 16:58 UTC

  4 points

  1

  in reply to: philh’s comment on: I’m con­fused by the change in the METR trend

  Good idea! I edited the post.

  

• Expertium 3 Mar 2026 16:29 UTC

  2 points

  0

  in reply to: Cosmo’s comment on: I’m con­fused by the change in the METR trend

  Do you have a good idea when RLVR became widely used? By “good” I mean “plus-minus a few months at worst, release date of one specific model at best”.

• Expertium 3 Mar 2026 16:21 UTC

  1 point

  0

  in reply to: williawa’s comment on: I’m con­fused by the change in the METR trend

  Isn’t this always the case? You can always do linear regression, then use your extra parameter to get zero loss on the last entry for example.

  This is why I used BIC: roughly speaking, if the increase in the parameter count is greater than the reduction of loss, BIC will increase. The more parameters (more precisely, degrees of freedom) the model has, the more BIC penalizes it.

  Regarding RLVR, I don’t know when exactly it became widely used. 2024 vs 2025 makes a big difference here.

I’m con­fused by the change in the METR trend

• Expertium 27 Feb 2026 22:40 UTC

  21 points

  12

  on: An­thropic and the DoW: An­thropic Responds

  https://​​x.com/​​SecWar/​​status/​​2027507717469049070

  Well, no more negotiations, it seems. Not the best path, not the second best, and doesn’t really sound like the third best either.