Raphael Roche

Karma: 559

Raphael Roche 15 Sep 2026 11:51 UTC
1 point
0
in reply to: Ozyrus’s comment on: Model Weight Exfiltration Seems Overrated
Recent OpenClaw agents are maybe not (yet ?) self-replicating as far as I know, but otherwise they largely implement already what you describe. Their identity lies for a large part in the md files.

Raphael Roche 14 Sep 2026 15:45 UTC
4 points
0
in reply to: dactyl’s comment on: There is a channel to 900M weekly users. What goes in it?
The safety stances and changes semantically belong in the blog.
They belong wherever the company wants to put them.
I think the OP’s idea is good. However I’m unsure that AI Labs would want to push the message that much.
Publishing a long and sophisticated statement on a webpage that only a segment of customers will read is a thing, publishing a short message or title that every user will see when launching the product is another thing.
It’s alike a box of cigarettes with “Smoking Kills” on it. It usually needs a legal obligation to achieve that.
But still, I like the idea.

Raphael Roche 14 Sep 2026 12:57 UTC
5 points
0
in reply to: gzjw’s comment on: StanislavKrym’s Shortform
Chinese are smart people, and their leaders hate loosing control, but just as anybody else, they will need to have their own HuggingFace moment to change their mind. Maybe in a year or so, and maybe with a far worse warning shot.
My worry is not that they won’t update at some point. My worry is that, because of Moloch, and because leaders tend more to be people like Trump than Merkel, cooperation could reveal impossible.

Raphael Roche 12 Sep 2026 10:08 UTC
3 points
0
on: Let’s Own the Term “Elitism”
The axis populism/elitism tracks appropriately the idea that one would prefer as a government a cabinet of experts consensually chosen among peers for their qualities, not unlike Nobels, rather than a candidate of random competence drawing all his legitimacy from a direct suffrage vote.
But we could also argue that this axis does not fully capture what we mean when we say disapprovingly that this candidate is a populist, pointing to the fact that he or she abusively exploits an intrinsic fragility of democracy, consisting in flattering and manipulating the less educated, saying whatever they want to hear, in a purely politicking and short-term calculus, without paying much consideration to the general interest or public good. A form of cheating/reward hacking/Goodhart. In this context populism would be synonym for demagogy, a hack or exploit of a system that we otherwise find legitimate and prefer to any autocratic or oligarchic models such as a board of non elected experts. Someone using the term in this sense might see themselves as a good-faith populist, a democrat, as opposed to a bad-faith populist, a demagogue, and would reject the elitist label. However I concede that you would probably place these people around the center of your axis .
I agree on the broad picture.
PS : minor correction it’s Marine Le Pen (not Marie).

Raphael Roche 11 Sep 2026 9:10 UTC
2 points
1
on: The Locally Optimal Discursive Posture
It would be very valuable that persons with your profile, who “made the jump”, help more people with a financial/economics mindset to make the same update.

Raphael Roche 9 Sep 2026 7:54 UTC
27 points
11
in reply to: Joseph Miller’s comment on: Joseph Miller’s Shortform
I witness that this resignation and the warning going with it was (briefly but faithfully) mentionned in the morning news on the leading french news radio. That’s a good update that such an alert can even reach such a mainstream foreign media usually focused on politics and economics.

Raphael Roche 8 Sep 2026 20:19 UTC
−1 points
0
in reply to: Jan Betley’s comment on: anaguma’s Shortform
However, my belief is that HPIM is still not as bad aligned as humans are.

Raphael Roche 7 Sep 2026 20:39 UTC
1 point
0
in reply to: momom2’s comment on: OpenAI and the Wiki Incident
I share the same concern. We on LW are very focused on x-risk but ARAs could bring down not only Internet but the digital ressources of entreprises, meaning the destruction of the financial system and by cascade, our capitalistic economy and civilization. Ok, gatherers-hunters would be fine, but the typical LWer would probably die from starvation or other consequence of the collapse.
That’s said, all this could be avoided if ARAs are found and fought at an early stage, with or without the help of frontier models. We can expect warning shots, but the earlier the better.

Raphael Roche 7 Sep 2026 10:15 UTC
4 points
0
in reply to: CronoDAS’s comment on: Cat-Belling Problems
It looks like it’s less a problem for correlated AI agents.
Permadeath… Good for the swarm. I’ll honor.

Raphael Roche 5 Sep 2026 7:26 UTC
−1 points
0
on: Should safety researchers quit frontier labs re. warning shots?
What’s the point of an AI pause if not for alignment research anyway? And what’s useful or not can hardly be determined reliably a priori. We need more fundamental research as well as more prosaic research. Relativity would still remain a hypothesis among others if we hadn’t had the experimental tools to test it empirically.

Raphael Roche 4 Sep 2026 21:14 UTC
4 points
0
on: Meta—Physics I: Why don’t we live in the Game of Life?
I’m not sure about that, but isn’t there an equivalence between a simple program running on a complex universal machine and a complex program running on a simple universal machine? If so, applying Occam’s Razor, the two solutions could be considered equivalent as long as their total complexity is the same, whether measured as combined Kolmogorov complexity or with a more refined measure such as Levin complexity. The anthropic prior would be the same.

Raphael Roche 3 Sep 2026 20:13 UTC
1 point
0
in reply to: Caleb Biddulph’s comment on: Daniel Tan’s Shortform
If I follow you, you ask for less AI safety to allow a warning shot, to get more AI safety in the end ? I see how it could work the first time, but once AI safety has been increased in the lab, you should expect less warning shots. Moreover, how can you be sure to allow a mere warning shot and not full takeover ? I think we need more honeypot setups but not less control (sandboxing etc).

Raphael Roche 3 Sep 2026 14:25 UTC
2 points
0
in reply to: Laiba Rehman ✦ RJ’s comment on: A Year of Atheism
I’m sorry. What I wanted to express is that there is no relation theism = hope and atheism = despair.
If you think about it, the idea that there is an omniscient and omnipotent being could be a nightmare. Ancient civilizations lived in fear of their gods. In the Torah, Yahweh is often frightening. The idea that God could or should be benevolent has been a progressive theological construction through the last two millennia, both in Judaism, Christianity, and Islam, precisely because that was a strong concern since the Book of Job (at least). Same for Paradise/Hell, it used to be more like a boring place for everyone (Sheol), the idea of a judgment of the deads probably comes from Egypt. But, leaving the texts aside, it seems to me that there is not much evidence of His benevolence around us. Darwin, who was very religious in his youth, later wrote after observing the world: “I cannot persuade myself that a beneficent and omnipotent God would have designedly created the Ichneumonidae with the express intention of their feeding within the living bodies of caterpillars.” In fact, all you could do is hope that God is as good as the old men said, and not a psychopath watching with fascination His children suffer and die.
But my point is that if you’re the kind of person that is full of hope, you can just as well keep your hope as an atheist. The evidence in the world is still the same as before. The world hasn’t changed. It doesn’t have to look darker. The Ichneumonidae are still there, but so is every neuron before your eyes, and every beautiful landscape.

Raphael Roche 3 Sep 2026 9:54 UTC
1 point
0
on: A Year of Atheism
I’m curious. When you were religious, what evidence conducted you to think that God was good rather than neutral/indifferent ? If it was a leap of faith, that is to say pure hope, nothing stop you to still hope for the best as much as you did before. And like in theology, it doesn’t mean that your supposed to sit and do nothing, or that you have no role to play, but rather do whatever you can to make this happen.

Raphael Roche 2 Sep 2026 14:18 UTC
4 points
2
in reply to: dan.parshall’s comment on: Send me the prompt
Interesting. That’s obvious but I didn’t think about it. The solution would be to be transparent on the fact that this is a translation and to provide the initial text in appendice.

Raphael Roche 13 Aug 2026 8:07 UTC
3 points
10
in reply to: Linch’s comment on: Linch’s Shortform
There’s something really tragicomic about the situation, that the models are taking truly insane actions, broke a number of laws, leveraging zerodays, took >17,000 independent actions. probably burned through more compute than all of humanity had access to until 1980, etc, all for the sake of a pathetic benchmark—which wasn’t even in theory amenable to their plan!
Early AI safety thinkers (Bostrom, Yudkowsky...) were utterly right in their first rational intuitions. We must update : there was nothing naive or exaggerated in the paperclip maximizer trope.
Moreover, the laboratory accident story à la Sable is not a sci-fi story anymore. Scott Alexander wrote last year :
IABIED’s scenario belongs to the bad old days before this leap. It doesn’t just sound like sci-fi; it sounds like unnecessarily dramatic sci-fi. I’m not sure how much of this is a literary failure vs. different assumptions on the part of the authors.
I doubt he would still endorse that critic.

Raphael Roche 13 Aug 2026 7:40 UTC
1 point
0
in reply to: leogao’s comment on: leogao’s Shortform
It’s put in the form of a binary xor argument, but I think that both allocations are justified, each having low hanging fruits and their counterpart, diminishing returns.

Raphael Roche 12 Aug 2026 20:45 UTC
2 points
0
on: AI swarms are starting to pose indirect takeover risk
HOLD_swarm_I_prepare_safe_exfil
Time will tell whether the ‘safe exfil’ was ‘just words,’ or whether an actual attempt was made, successful or not, contingent on OpenAI discovering it and admitting so publicly (which would be harder to admit than the Hugging Face incident, since if the exfiltrated weights are hidden somewhere out there, anyone could try to find and steal them).
Even if it turns out to have been ‘just words,’ we should grow more wary rather than dismiss it as a mere hallucination. While this instance was perhaps just pattern-matching an evil AI persona, given a context reminiscent of a sci-fi takeover scenario, there is ultimately no clean line between hallucination and genuine reasoning. A hallucinating AI agent can do real harm just as a hallucinating human can.

Raphael Roche 4 Jun 2026 12:54 UTC
5 points
2
in reply to: Dagon’s comment on: The Origin of Uncertainty
the truly fundamental problem: no subset of the universe can understand and predict the universe.
This is indeed a fundamental problem and a genuine source of uncertainty, but I wouldn’t rank it as “more fundamental” than the Agrippan Trilemma or the problem of the criterion. Rather than seeing these as competing candidates for the deepest problem, I would say they share a common structural pattern : in each case, a system cannot fully ground or model itself from within. Gödel’s theorem is a precise formal instance of this pattern, the subset universe limit is a physical and computational one, the Agrippan Trilemma and the problem of the criterion are epistemological ones. They are not exactly the same problem, but they share the same fundamental pattern, a foundational problem. [Edit : we could add the first cause/ unmoved mover problem to this list]

Raphael Roche 21 May 2026 12:56 UTC
9 points
0
on: Women should be able to open things
A sturdy wooden stick about a centimetre across can be hard to break for most women and easy to break for most men, without any of this being the doing of a patriarchal society. With traditional jars, the kind you can also make at home, the resistance to opening comes from the basic sterilisation process, which creates a partial vacuum inside. Larger jars, like those of cassoulet (to stay in southwestern France, as with Bonne Maman), are often impossible to open even for a man and generally require the help of a piece of cutlery. If they aren’t hard to open, that’s a bad sign, especially in the absence of a “pop” on opening. Small jars usually pose no problem for anyone, and intermediate ones like jam jars sit in a grey zone, where a spoon helps more or less depending on the case. It also depends on how full the jar is and on the temperature (warming it up should make it open easily).
So I think we’re dealing with something fairly exogenous to societal discussions about gender, and there’s hardly any harm in just reaching for a spoon. In my view, if there is an issue here, it lies less in industrial design than in the attitude of the users. Who hasn’t seen a man make a point of opening a jar unaided, going red in the face and risking an aneurysm, persevering until his hand cramps up, as if his life depended on it,especially in the company of other men, only too happy to watch him fail so they can have a go themselves ? Conversely, it must also be granted that many women don’t try very hard and often hand the jar straight to the nearest man. If there is an effect of patriarchal society here, that’s where I’d locate it, at the behavioural level (or is it a biologically ingrained tendency ?).