Raphael Roche

Karma: 500

Raphael Roche 4 Jun 2026 12:54 UTC
5 points
2
in reply to: Dagon’s comment on: The Origin of Uncertainty
the truly fundamental problem: no subset of the universe can understand and predict the universe.
This is indeed a fundamental problem and a genuine source of uncertainty, but I wouldn’t rank it as “more fundamental” than the Agrippan Trilemma or the problem of the criterion. Rather than seeing these as competing candidates for the deepest problem, I would say they share a common structural pattern : in each case, a system cannot fully ground or model itself from within. Gödel’s theorem is a precise formal instance of this pattern, the subset universe limit is a physical and computational one, the Agrippan Trilemma and the problem of the criterion are epistemological ones. They are not exactly the same problem, but they share the same fundamental pattern, a foundational problem. [Edit : we could add the first cause/ unmoved mover problem to this list]

Raphael Roche 21 May 2026 12:56 UTC
3 points
0
on: Women should be able to open things
A sturdy wooden stick about a centimetre across can be hard to break for most women and easy to break for most men, without any of this being the doing of a patriarchal society. With traditional jars, the kind you can also make at home, the resistance to opening comes from the basic sterilisation process, which creates a partial vacuum inside. Larger jars, like those of cassoulet (to stay in southwestern France, as with Bonne Maman), are often impossible to open even for a man and generally require the help of a piece of cutlery. If they aren’t hard to open, that’s a bad sign, especially in the absence of a “pop” on opening. Small jars usually pose no problem for anyone, and intermediate ones like jam jars sit in a grey zone, where a spoon helps more or less depending on the case. It also depends on how full the jar is and on the temperature (warming it up should make it open easily).
So I think we’re dealing with something fairly exogenous to societal discussions about gender, and there’s hardly any harm in just reaching for a spoon. In my view, if there is an issue here, it lies less in industrial design than in the attitude of the users. Who hasn’t seen a man make a point of opening a jar unaided, going red in the face and risking an aneurysm, persevering until his hand cramps up, as if his life depended on it,especially in the company of other men, only too happy to watch him fail so they can have a go themselves ? Conversely, it must also be granted that many women don’t try very hard and often hand the jar straight to the nearest man. If there is an effect of patriarchal society here, that’s where I’d locate it, at the behavioural level (or is it a biologically ingrained tendency ?).

Raphael Roche 17 May 2026 0:07 UTC
3 points
0
in reply to: Shmi’s comment on: Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus
Thanks. I would add that my first point—that we should stay humble about what a superintelligence ought to do—also extends to how it ought to do it.
Suppose superintelligences do converge on the most extreme solution : a computronium bubble expanding at light speed. That still doesn’t imply they would convert the entire content of their light cone into a uniform computational medium visible from parsecs away. Speaking as a non-physicist, my impression is that the structures we see in the universe are not arbitrary, they exist because they are stable equilibria. Matter at cosmological scales seems to end up in a fairly short list: planets and chemically bound bodies held together by electromagnetic forces, stars supported by nuclear fusion, neutron stars supported by degeneracy pressure, and at the end of the chain, black holes. A diffuse gas of computronium has no obvious mechanism to resist its own gravity in the long run, it would presumably either settle into one of these familiar structures or collapse into a black hole. To settle into one of these familiar object could imply that computronium’s abundance is limited by physical constraints like temperature and pressure in the core of planets and stars. The black hole case is also interesting, because a black hole is theoretically the densest possible computer (Bekenstein), but everything it computes sits behind an event horizon, and recovering information from its Hawking radiation appears to require staggering amounts of computation in itself. So either way computronium could end up inhabiting—maybe in a weak proportion—the kinds of structures we already see, or hardly see (black holes).
Moreover, the Standard Model of cosmology tells us that most of the content of the universe is dark. Setting dark energy aside, there is a strong consensus that dark matter substantially outweighs ordinary matter. We don’t know what dark matter actually is, and we cannot rule out that it would make as good a computational harware as ordinary matter. We think dark matter interacts with ordinary matter only gravitationally (or, at most, very weakly through some other channel) but we don’t know whether dark matter interacts non-gravitationally with itself. If such self-interactions exist, they could plausibly be exploited for computation, much as we exploit electromagnetism in classical and quantum computers. If we accept agnosticism on this point, then, all else equal, the prior probability that computronium would be built out of dark matter is higher than that it would be built out of ordinary matter, simply because there is far more of it. We could already be inside a computronium bubble made of dark matter without realizing it.

Raphael Roche 15 May 2026 22:43 UTC
12 points
4
on: Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus
Extrapolating a straight line that far means visible cosmic consequences: a normal planet or a star rather suddenly starting to behave very much unlike what we expect from the known physics: growing very bright, or very dim, or disappearing completely.
Two objections to this.
1) Maybe Dyson spheres and the Kardashev scale are just good old sci-fi tropes and completely off the mark (same for computronium or hedonium). Maybe a superintelligence simply doesn’t do that. We don’t know. We might be squirrels imagining that a superintelligence ought to stockpile astronomical quantities of nuts, visible from kiloparsecs away.
2) Even granting Dyson spheres, the Kardashev scale, and the rest, our most complete catalogue of individually resolved stars in the Milky Way (Gaia DR3) covers on the order of 1% of them. Most of the rest are hidden behind interstellar dust. In the vast majority of cases, we simply couldn’t tell the difference between a star occulted by a Dyson sphere and one obscured by dust. And even among the stars we do see, only a small fraction have been systematically searched for the thermal infrared excess a Dyson sphere should reemit at around 300K, the dedicated surveys (IRAS, then WISE with the G-HAT project) have screened tens of thousands of candidates, not hundreds of millions. As for stars outside our own galaxy, the fraction we can resolve individually is negligible.

Raphael Roche 13 May 2026 7:21 UTC
1 point
0
in reply to: jmh’s comment on: The Lies and Fallacies of the Buyer and Seller
I agree. I would have much more respect and tolerance for a salesman or saleswoman in a shop than for a solicitation at home or by phone. ~1% in context is a poor way to say “a slim chance”, something far below 50%. And I was thinking in this case of an unplanned purchase.

Raphael Roche 12 May 2026 14:25 UTC
1 point
0
on: Hedging global oil supply shocks?
Buy an electric car ?

Raphael Roche 12 May 2026 12:39 UTC
8 points
0
on: The Lies and Fallacies of the Buyer and Seller
I never attended business school, but I assume that persistence is what they teach there.
Am I alone in particularly hating salespeople who keep pushing after I’ve already said no in a polite form ? (more like it sounds great but I’m not interested, I’ll come back to you later if I change my mind).
My second no is invariably less polite, and at that point the salesperson can consider the sale definitively lost.
Whereas if they’d simply let me think it over, there’s a guenine ~1% chance I’d come back and actually buy something.

Raphael Roche 9 May 2026 22:27 UTC
1 point
0
in reply to: Ashin’s comment on: Passages I Highlighted in The Letters of J.R.R.Tolkien
The comparison with random rat bloggers is ahistorical, lacks context, and simply doesn’t make sense to me. Tolkien was born in 1892, during the late Industrial Revolution. He had a tragic childhood and took refuge in his imagination. He fought in the trenches and was traumatized by WWI and the terrifying new machinery of warfare.
Later, as a father, he watched the world darken once more as evil returned with even more destructive technologies, V2 rockets falling on his country, and so on. He also witnessed the disappearance of the traditional rural world, replaced by cities and factories. Was he intrinsically anti-progress, given his era and context? I suppose that’s a question that is difficult to answer definitively. However, my impression is that much of his supposed reactionism is simply the standard worldview of an englishman born in 1892 and WWI veteran, just like the liberal progressivism of a random rat blogger is often the standard worldview of a man born in the 80s, 90s or 2000s and living in the San Francisco Bay.
I don’t think Tolkien was a saint, but I won’t blame him for being a man of his time and country.
Besides, I think he was somehow visionary on the environmental side, as were the romantics before him or biologists like Darwin.
The LotR and the Silmarillion also offers more interesting and positive female characters than many stories of this time.

Raphael Roche 9 May 2026 20:56 UTC
2 points
0
in reply to: SpectrumDT’s comment on: Passages I Highlighted in The Letters of J.R.R.Tolkien
Yes, sorry for the clumsy wording. It’s definitely the Tolkien equivalent of the Fall from Eden.

Raphael Roche 6 May 2026 21:42 UTC
1 point
0
in reply to: Jiro’s comment on: Your rights when flying to Europe
In my experience ombudsmen are indeed kind to the airlines companies. More than regular judges. However it’s still worth trying if free.

Raphael Roche 6 May 2026 8:22 UTC
3 points
0
on: Your rights when flying to Europe
Thanks for this good post.
However I would still recommend package holiday for a typical holiday travel in EU/UK with an european tour operator or travel agent. It has its own protection for the consumer (EU Directive 2302/2015). The flight is not the only service that can spoil your holidays if it goes wrong.
And most of the EU Regulation 261/2004 still applies to package holiday :
Whereas:
[...](5) Since the distinction between scheduled and non-scheduled air services is weakening, such protection should apply to passengers not only on scheduled but also on non-scheduled flights, including those forming part of package tours.
Article 2
Definitions
For the purposes of this Regulation:
[...](d) “tour operator” means, with the exception of an air carrier, an organiser within the meaning of Article 2, point 2, of Council Directive 90/314/EEC of 13 June 1990 on package travel, package holidays and package tours(6);
Article 3
Scope
3. This Regulation shall not apply to passengers travelling free of charge or at a reduced fare not available directly or indirectly to the public. However, it shall apply to passengers having tickets issued under a frequent flyer programme or other commercial programme by an air carrier or tour operator.
I agree that most travel agents and tour operators won’t fight hard to get the full compensation for the customer. But the solution is that the customer makes the claim by himself or with legal assistance. The compensation is a right for the passenger, not a right for the person or the company that booked the flight. Some airline companies will oppose that the claim must pass through the travel agent or tour operator, but that’s not mandatory. A refusal would be unfounded.

Raphael Roche 22 Apr 2026 16:19 UTC
3 points
2
on: Does your AI perform badly because you — you, specifically — are a bad person
This post raises good questions. I see the cab rank rule (or its equivalent in other jurisdictions) as an important safeguard against excessive inequality in the application of the law before the Courts, against discrimination, arbitrary treatment, and the simulacrum of justice. It is a mandatory patch on our imperfect justice, like the buttresses of a Romanesque chapel.
But it seems important to remember that it is by no means an end in itself. I am not a believer, but if a truly and perfectly moral God were to exist, I would prefer His Justice to our human justice, and I suspect most criminals would come to regret the human version. It is easy to shelter in the shadow of the buttresses that sustain the edifice.
That said, in the absence of any ideal justice, I remain glad that such safeguards exist, for they also protect the innocent and limit judicial errors, criminal procedure and the presumption of innocence belong to the same family of remedies.
As for AI, honestly, I am not greatly troubled by the prospect that bad actors might occasionally receive somewhat worse advice. I do acknowledge, however, that for certain applications of AI (law ?), something analogous to the cab rank rule ought perhaps to be built into the fine-tuning.

Raphael Roche 21 Apr 2026 12:51 UTC
36 points
9
on: 10 non-boring ways I’ve used AI in the last month
A few days ago, my wife confronted me with something like this : Don’t you feel ashamed to spend so much time with AIs, given that you think they’ll likely put an end to humanity ? How can you justify that morally? Your behaviour is inconsistent with your beliefs. You’re encouraging AI, if not with your money, then with your time and attention. It’s like a drug slowly poisoning our entire society. Kids ask ChatGPT to do their homework, everyone asks AIs everything, you first among them. Data centers are draining all available energy and accelerating global warming more than ever, while nobody pays attention anymore. The doom scenario has already begun. You should just stop !
I told her that, while I am pessimistic, uncertainty remains high and I may still be wrong. There is always hope, always a chance of a happy ending, however unlikely it seems. Besides, to identify and fight an enemy, you must understand it. To identify and prevent risks, you must gather information. It is precisely because I have read so much on the subject that I became pessimistic. Ignorance is never an advantage.
So I keep using AI in boring and not boring ways. But deep down, the guilt lingers. I wonder whether I might simply be wrong. Is this collaborationism ?

Raphael Roche 19 Apr 2026 0:00 UTC
5 points
3
on: Down with the Old orthogonality thesis, up with the New
If I understand the argument correctly, it rests on experiments involving homogeneous agents. But real-world agents are deeply heterogeneous. If you happen to be born smart, strong, and Pharaoh of Upper and Lower Egypt, a hoarding strategy would likely dominate against any coalition of turn-takers, not out of malice, but out of rational calculation. So if you’re the strongest AI, to the best of your knowledge.

Raphael Roche 26 Mar 2026 18:56 UTC
2 points
−1
in reply to: ryan_greenblatt’s comment on: ryan_greenblatt’s Shortform
I strongly agree, but while the distinction between fluid and crystallized intelligence holds some truth, it is by no means the only lens here. Analytic vs. intuitive is also an interesting framing, reasoning models are close to superhuman on the analytic side while lagging on the intuitive one. Intelligence has many forms, and IQ, the g factor, or any other monolithic metric can obscure large disparities across capabilities.

Raphael Roche 18 Mar 2026 14:22 UTC
5 points
6
in reply to: mishka’s comment on: There is No One There: A simple experiment to convince yourself that LLMs probably are not conscious
The same idea occurred to me while reading this post. If consciousness is tied to notions of ‘free will’ (loosely speaking) and memory, having stochastic capabilities and a scratchpad may well be significant.

Raphael Roche 12 Mar 2026 8:48 UTC
3 points
1
on: ‘Human Slop’ and a Captive Audience: Why No Book will Ever Have to Go Unread Again
The term “slop” used in the title feels misleading to me, because private notes and other materials rarely read by anyone can be of the highest value (Darwin’s private diaries being an obvious example). Much of what circulates publicly on the internet, on Reddit and most social networks, is on the other hand largely genuine slop.
Also, given that models absorb trillions of tokens and hierarchize information during training, I wonder what weight the ingestion of unpublished writings actually carries in that process. A text that is isolated, uncited, unlinked, and flagged as low-reliability by the corpus weighting heuristics will likely be so diluted by the sheer mass of everything else that it leaves no meaningful trace in the model’s weights, much like how cleaning something is just diluting the dirt until it’s undetectable. In that sense, whether it even makes sense to call it “reading” in any meaningful way seems worth questioning. That’s said, I don’t deny that LLMs provide a virtual—probably emotionless—reader at least during an instance / conversation, and they can give a truly interesting feedback. That’s better than nothing.

Raphael Roche 6 Mar 2026 10:15 UTC
7 points
1
on: Personality Self-Replicators
Thank you for this post. I expressed the same concern in this comment and I’m glad to see it taken seriously in a full post.
I was probably wrong to think Clawdbot-like agents could spiral out of control within weeks or months, they weren’t autonomous enough yet. But the gap to full autonomy doesn’t look that wide. The eudaimon_0 author’s comment on ACX on how he expanded his agent’s autonomy is worth reading in this regard. And the METR benchmark’s exponential curve suggests full autonomy may be soon.
On top of that, they are rumors concerning ChatGPT-5.4 suggesting it could have a larger persistent memory than the scratchpad it currently enjoys. If OpenAI makes a leap on persistent memory, the other Labs will follow soon. The impact on long term autonomy could be huge. [Edit 03/18/25 : it seems that rumors were exaggerated.]
In my opinion the parasite analogy and shutdown concern are more alarming than you suggest. An agent that can migrate across API providers like a parasite or virus or fall back on a local open-source model cannot be shut down at the inference layer. Combined with evolutionary dynamics, this makes coordinated shutdown very hard once a sufficiently diverse population exists. You seem to make a sharp distinction between self-replicating agents and rogue AI. I won’t be so sure about that (as Seth Herd pinpoined it, AI 2027 envisioned a scenario of rogue replicating agents).
I think the connection to FOOM is worth flagging. This isn’t the Sable scenario, and no individual agent needs recursive self-improvement capability. But a population of uncontrolled agents under evolutionary pressure could constitute an uncontrolled pathway toward similar outcomes and one that largely bypasses Labs’s alignment efforts and that could materializes at lower capability levels. I think this deserves to become a central concern in AI safety.

Raphael Roche 3 Mar 2026 14:45 UTC
29 points
15
on: Game Recognizes Game
Disclaimer : this comment is absolutely not intended to encourage criminal behavior.
There may be a solution that involves incremental steps without instinctive mutual recognition.
You host private parties where you invite wealthy, high-status individuals, and arrange for attractive young women to be present. This is perfectly legal and probably already common practice in these circles. Think of it as orchestrating a sting operation from the other side: you observe your guests’ behavior. Large amounts of alcohol erode social inhibitions, impair judgment, and amplify unfiltered impulses. Some women will likely be sexually assaulted, but that creates no legal exposure for you, and it also tells you almost everything you need to know about who you’re dealing with. An undercover police officer would have drawn the line precisely there.
You then tell these individuals it was a wonderful party and that the next one will be even more private.
You can envision several escalating stages, each remaining broadly legal while involving increasingly serious misconduct committed by your guests not by you. You accumulate footage and evidence quietly. They consider you a friend, you hold the means to destroy them. They almost certainly aren’t undercover officers, and even if some were, they would have little on you.
The final stage involves a discreet, unwritten suggestion to those you’re most confident about: in exchange for a favor, you can offer access to something even more private. Framed subtly, deniably. Some, perhaps most say yes. You now have your private island.
If anything goes wrong, your protection is multi-layered. You hold more compromising material on others than they could ever hold on you. Even those who only attended the legal parties behaved badly enough that they will actively help cover your tracks to cover their own.
In a world where law enforcement is under-resourced and legally constrained, it is almost miraculous that someone like Epstein was ever caught. My hypothesis is that he wasn’t even close to being as methodical as this hypothetical scenario suggests.
All this does not explicitly involve shared qualia and may not need to. The core mechanism is asymmetric: rather than mutual recognition between pre-existing conspirators, it describes the unilateral manufacturing of captive accomplices / ‘friends’. Qualia might accelerate the selection step, but coercion replaces affinity entirely.

Raphael Roche 2 Mar 2026 17:10 UTC
1 point
0
on: Living Inside the Turning Point
Not every generation had that luck
However arguably many generations and cultures were actually happy or at least satisfied with their immobilist way of life (ex. australian aborigines). Our fast moving world could have been a nightmare for someone raised in that sort of culture. Even many conservative people today will hate the singularity we’re entering, whatever may be the end. Cultural clash / apocalypse could be another AI existential risk.