Matrice Jacobine

• Matrice Jacobine 27 Jul 2026 7:32 UTC

  2 points

  0

  on: OpenAI’s my­opia just keeps caus­ing al­ign­ment problems

  With Bing, it was rushing to push out a superficially Helpful Assistant.

  You didn’t mention Bing earlier in the article, so the sentence is confusing if one didn’t knew what you were talking about (and even if one did tbh).

• Matrice Jacobine 24 Jul 2026 17:13 UTC

  3 points

  0

  in reply to: Alexander Gietelink Oldenziel’s comment on: Alexan­der Gietelink Ol­den­z­iel’s Shortform

  ???

  Terrorist attacks using drones have been a thing for a while now e.g. Caracas 2018, Abha 2019, Homs 2023.

OpenAI and Hug­ging Face part­ner to ad­dress se­cu­rity in­ci­dent dur­ing model evaluation

• Matrice Jacobine 22 Jul 2026 6:24 UTC

  2 points

  0

  in reply to: Dave Banerjee’s comment on: jdp’s Shortform

  Furthermore, this prompt would make no sense in the context of ExploitBench, which is not an agentic benchmark but a coding benchmark. To get rewarded in ExploitBench, you have to send a PoC (for a given known CVE in the V8 engine) as a single JS file to a sealed-off (automated, non-LLM) grader.

• Matrice Jacobine 16 Jul 2026 19:13 UTC

  8 points

  2

  in reply to: the gears to ascension’s comment on: the gears to as­cen­scion’s Shortform

  I used to defend this line of work as relevant toys to demonstrate capabilities to sabotage back when it was still hegemonic to consider models as mere tools. I do not do so anymore. Both non-judgmental AI psychology on one hand and realistic misalignment cases on the other hand have boomed as fields since then, including inside the walls of Anthropic, and continuing this line of work as is is wholly harmful at this point.

• Matrice Jacobine 15 Jul 2026 15:37 UTC

  4 points

  0

  in reply to: Richard_Kennaway’s comment on: Skep­ti­cal of the TESCREAL Acronym? Read This.

  (related)

• Matrice Jacobine 1 Jul 2026 0:53 UTC

  4 points

  0

  in reply to: Avi Brach-Neufeld’s comment on: Func­tional De­ci­sion The­ory: Not Even Wrong, Also Wrong

  The ASI isn’t going to be worse at prediction than you, currently predicting that people are going to two-box in Newcomb’s problem.

• Matrice Jacobine 1 Jul 2026 0:40 UTC

  3 points

  0

  in reply to: Jiro’s comment on: Func­tional De­ci­sion The­ory: Not Even Wrong, Also Wrong

  Actually, the halting problem (well, its generalization, Rice’s theorem) allow you to get a more precise intuition for why punishing agents iff they follow XDT is ‘unfair’ (it would be Turing-uncomputable for Omega to decide if an agent follow XDT, even with his omniscience and infinite compute).

• Matrice Jacobine 24 Jun 2026 21:17 UTC

  4 points

  0

  in reply to: Ben Pace’s comment on: (Re)in­tro­duc­tion of a ra­tio­nal­ist dragon, and clar­ifi­ca­tions on Ziz’s character

  [Update: Zajko charged for the double-homicide.]

• Matrice Jacobine 18 Jun 2026 13:58 UTC

  6 points

  4

  on: Dario prob­a­bly doesn’t be­lieve in superintelligence

  As another point, here is Anthropic’s publicly stated dystopian nightmare scenario if “China rises”:

  Scenario two: The CCP-controlled AI ecosystem is neck-and-neck

  AI developed and deployed in China is near-frontier on model intelligence. Despite a weak semiconductor production capacity, models trained by PRC AI labs are only a few months behind US models. Ongoing distillation attacks, overseas compute access, weak SME export enforcement, and a loosening of export controls on American semiconductors have assisted CCP efforts. Continued access to US frontier AI for AI R&D has also enabled AI labs in China to close the gap and approach parity with their US counterparts.

  Rapid commercial and state adoption. Beijing has championed a whole-of-nation push on domestic adoption via “AI+” policies. Even though China’s AI models are slightly less capable than US models, CCP efforts to accelerate adoption have paid off. China is thus able to deploy near-frontier AI capabilities more advantageously across economic, military, and technological domains, shifting the balance of power in China’s favor.

  The CCP’s AI-enabled cyber force is a serious threat. The CCP’s integration of AI-enabled cyber capabilities within an already advanced cyber force has sustained the PLA as a menacing cyber competitor. PLA cyber actors have gained additional access to critical and dual-use infrastructure in the US and most countries around the world, enabling them to disrupt critical national security and societal functions. As AI is incorporated deeper into our most critical systems, democracies enjoy no security advantages over China in AI, despite having developed the technology first.

  Beijing is winning in global adoption on cost and on-prem flexibility. Huawei and Alibaba data centers are globally prevalent, especially in, but not limited to, lower cost markets in the Global South. These data centers scale on older chips, which China is able to export because it can serve its domestic market with a combination of US chips purchased with an export license, smuggled into China, or remotely accessed in overseas data centers. They host second-tier, but cheaper and still effective models produced by PRC labs. Similar to the Huawei playbook of being cheap and “good enough,” China’s near-frontier models and hardware support a non-trivial and rapidly growing segment of the global economy. This infrastructure advantage gives CCP leadership significant influence over those markets.

  Even taking into account the need to present a reduced shock level for decision-makers, I do not think anyone who actually believe in superintelligence, let alone the possibility of existential risk from it, would consider this (/​the ++ version adding effective centuries of technological development) anything other than an A-tier Paretotopian future, and supporting arms-racing brinkmanship to avoid it as Anthropic does anything other than Brigadier General Jack D. Ripper behavior.

• Matrice Jacobine 9 Jun 2026 18:14 UTC

  7 points

  0

  on: Ma­trice Ja­cobine’s Shortform

  My first Fable benchmark was to one-shot turning `emacs -batch -l dunnet` into a graphical adventure game and it hit the safety guardrails bc one of the puzzles involve nitroglycerine 😭

• Matrice Jacobine 28 May 2026 8:40 UTC

  3 points

  0

  in reply to: JBlack’s comment on: Atom­i­cally pre­cise mechanosyn­the­sis of car­bon struc­tures on hy­dro­genated Si(100) by in­verted-mode STM

  I did initially put the link as + Linkpost, but the ForumMagnum editor seems to casually delete it half the time. It’s unfortunately not the first time this happened to me.

Atom­i­cally pre­cise mechanosyn­the­sis of car­bon struc­tures on hy­dro­genated Si(100) by in­verted-mode STM

• Matrice Jacobine 14 May 2026 22:58 UTC

  8 points

  0

  on: Ma­trice Ja­cobine’s Shortform

  “On the Promotion of Safe and Socially Beneficial Artificial Intelligence” by @SethBaum from 2016

  

• Matrice Jacobine 10 May 2026 14:13 UTC

  3 points

  0

  in reply to: ryan_greenblatt’s comment on: ryan_green­blatt’s Shortform

  It sure would be great if there was a well-funded “AI safety” nonprofit with the goal to be as open as possible about AI research. Anyone considered that?

• Matrice Jacobine 6 May 2026 14:48 UTC

  79 points

  1

  on: Ma­trice Ja­cobine’s Shortform

  So… what’s the general take on the hantavirus outbreak?

• Matrice Jacobine 4 May 2026 13:39 UTC

  4 points

  0

  in reply to: Steven Byrnes’s comment on: Ma­trice Ja­cobine’s Shortform

  I don’t think it shows that. It arguably suggests that abundant pretraining data doesn’t matter as much as one could think. As opposed to good pretraining data.

  I think it straightforwardly show the reverse (good pretraining data doesn’t matter as much as one could think, but abundant pretraining data does)? Olmedo himself notes: “What holds the 1930 model back is that it is severely undertrained (only 260B tokens), rather than its pre-training data.”

  I presume that the codebases + agentic coding transcripts that they SFT’d on were high quality, right?

  SFT is posttraining??

Ma­trice Ja­cobine’s Shortform

• Matrice Jacobine 3 May 2026 15:01 UTC

  25 points

  2

  on: Ma­trice Ja­cobine’s Shortform

  Ricardo Olmedo: “We fine-tuned Alec Radford’s 1930 vintage LLM to solve SWE-bench issues. After just ‼️250‼️ training examples, the model solves its first issue, a simple patch to the xarray library.”

  This is interesting because, it is not discussed enough that AlphaGo worked the same way LLMs work: it was pretrained on a large dataset of human moves, then posttrained through reinforcement learning. The shift to AlphaZero removed the pretraining, showing that pretraining data didn’t matter at all for the model’s capabilities (quite the contrary, AlphaZero is superior to AlphaGo). While this coding model is still pretrained, it also shows that good pretraining data doesn’t matter nearly as much as one could think.

• Matrice Jacobine 27 Apr 2026 7:37 UTC

  7 points

  1

  in reply to: Creatrei’s comment on: (Re)in­tro­duc­tion of a ra­tio­nal­ist dragon, and clar­ifi­ca­tions on Ziz’s character

  Substack is designed for news, not for posts people will be reading in five years. Reading a Substack’s archive is a pain, and most Substacks are poorly indexed by Google. Therefore, if you take the intermittent-posting approach, I recommend posting to both a Substack and a separate stand-alone website (for inspiration, consider Andy Masley or dynomight’s pages). Your Substack allows people who are interested in your thoughts to subscribe and be automatically notified when you have something new to say, while the stand-alone website serves as a canonical reference.

  (x)