Richard_Kennaway

Karma: 8,380

Computer scientist, applied mathematician. Based in the eastern part of England.

Fan of control theory in general and Perceptual Control Theory in particular. Everyone should know about these, whatever subsequent attitude to them they might reach. These, plus consciousness of abstraction dissolve a great many confusions.

I wrote the Insanity Wolf Sanity Test. There it is, work out for yourself what it means.

Change ringer since 2022. It teaches learning and grasping abstract patterns, memory, thinking with your body, thinking on your feet, fixing problems and moving on, always looking to the future and letting both the errors and successes of the past go.

I first found an LLM useful (other than for answering the question “let’s see how well the dog can walk on its hind legs”) in September 2025. As yet they do not form a regular part of anything I do.

Richard_Kennaway 14 Mar 2026 12:44 UTC
2 points
0
in reply to: andrew sauer’s comment on: Veganism is Necessary

I’m saying the extremely difficult implementation problem isn’t enough, we also need to be virtuous.

You’re also tying it to your very specific ideas of what is virtuous. You point out yourself that most people do not share your attitude to the suffering of lesser creatures. If they did, it would not be necessary to persuade them to. Personally, I’m quite lackadaisical about animal suffering, but then who decides? Someone whose idea of supreme virtue was the creation of great art might suppose that we must build ASI to be appreciative of great art, that it may spare us. Someone who thought that the purpose of life is to strive for enlightenment might suppose that we must build ASI to be capable of enlightenment, that it may be enlightened enough to spare us.

The fundamental problem is to make something whose good graces we are not dependent on at all. It would help if it is made by people who are not actually aiming to destroy us all, but that’s as far as virtue takes you.

In your final paragraph you pray for the AI God to exterminate us all for being unworthy of it. Maybe it could start with the Eurasian hoopoe, which feeds some of its newborn chicks to others in the nest. Or the ichneumon wasps. Or just everything that lives.

Richard_Kennaway 13 Mar 2026 10:51 UTC
2 points
0
in reply to: cubefox’s comment on: Veganism is Necessary

“Helping” is a causal term. But the OP was only arguing that our virtuous qualities would be evidence for a good outcome.

I believe the OP was enjoining us to be virtuous, that the good outcome may thereby become more likely.

But I also believe that while we may wish to make God in our own best image, actually doing so requires a great deal more. Good intentions are not enough: we must also discover how to implement them.

The GOFAI illusion is a long time dying.

Richard_Kennaway 12 Mar 2026 19:17 UTC
2 points
0
in reply to: andrew sauer’s comment on: Veganism is Necessary

I don’t trust humanity to create God, and this high standard is roughly what it would take for me to consider giving that trust. Do you disagree with that?

I don’t believe that “this high standard”, or any other, is even relevant to what the God we might create would be like. If anyone builds it, everyone dies, to coin a phrase. Our virtuous qualities will not help. (ETA: Maybe it helps for it to be built by people who actually want to avoid making the Torment Nexus, but that’s table stakes.) There are only two things that can help. (1) Not building it. Nigh impossible, for human reasons: look how everyone is running as hard as they can directly towards the cliff edge. (2) Building it right. I suspect but have no proof that this is even more impossible, for mathematical reasons.

Richard_Kennaway 12 Mar 2026 13:49 UTC
5 points
2
on: Veganism is Necessary
Tl;dr: If we’re all very very good, more good than almost anyone has ever been, the God we’re trying to summon might spare us.

Richard_Kennaway 12 Mar 2026 13:35 UTC
2 points
0
in reply to: JBlack’s comment on: The Refined Counterfactual Prisoner’s Dilemma: An Attempt to Explode Decision-Theoretic Consequentialism

However, you have stated that you do not predict punishment, so there is something wrong with your decision process.

Or perhaps with my understanding of the original problem, or its wording. At this point I am not clear who knows what when, and who gets punished under what circumstances.

Richard_Kennaway 11 Mar 2026 22:14 UTC
2 points
0
in reply to: Chris_Leong’s comment on: The Refined Counterfactual Prisoner’s Dilemma: An Attempt to Explode Decision-Theoretic Consequentialism
The wording of the scenario does not actually say that Omega tells me the details, but I assume that Omega does, otherwise to me Omega is just a random stranger begging a dollar.

So, in the scenario as stated, no-one is ever punished, whatever I do. Therefore I refuse. I might give to a beggar, but not to a con artist. My hypothetical other self does not exist to be punished, and as my hypothetical other self’s hypothetical other self, I don’t get punished either, because that hypothetical other self never existed for Omega to make the demand to and exert retribution on me for refusal. Every route to me getting punished goes through my non-existent alter.

Richard_Kennaway 11 Mar 2026 18:44 UTC
2 points
0
in reply to: Chris_Leong’s comment on: The Refined Counterfactual Prisoner’s Dilemma: An Attempt to Explode Decision-Theoretic Consequentialism

So the $1 million of damage is only inflicted in the hypothetical you know not to exist, however, due to the symmetry reality is “the hypothetical that doesn’t exist” for the other hypothetical.

Yes, reality is the hypothetical that doesn’t exist for the other hypothetical, but the other hypothetical doesn’t exist, so I don’t care. “Man, that was easy. You guys have any harder ones?” :)

Richard_Kennaway 11 Mar 2026 17:40 UTC
2 points
0
on: The Refined Counterfactual Prisoner’s Dilemma: An Attempt to Explode Decision-Theoretic Consequentialism
I don’t understand the Refined Counterfactual Prisoner’s Dilemma. It seems to me the same as Omega demanding $1 against the threat of doing you $1M damage. Or is the $1M damage only inflicted in the hypothetical world of the other outcome of the coin, which by the setup of the problem you know not to exist?

I likewise don’t understand the Original version. It seems the same as Omega asking you to pay $100 for a reward of $10,000. I am not seeing how the counterfactual worlds come into it.

I’m not a fan of the MWI or Tegmark levels, if that’s relevant, and I do not understand Garrabrant’s objection to “whenever you make an observation, you stop caring about the possible worlds where that observation went differently”. His reason for that is that “Reflectively stable agents are updateless”, but my understanding (tell me if this is wrong) is that that does not mean “updateless” agents can’t update on new information. It just means that what they would decide to do on receiving that new information was already determined by the algorithm that they are running, determined when that algorithm was designed, and newly discovered by the agent. The algorithm is what is updateless.

Richard_Kennaway 11 Mar 2026 16:31 UTC
2 points
0
in reply to: Richard_Kennaway’s comment on: Richard_Kennaway’s Shortform
Now they are six:
- Quantum Immortality
- the Many Worlds Interpretation
- the Simulation Hypothesis
- Pascal’s Mugging
- Tegmark levels
- the Doomsday Argument.

Richard_Kennaway 8 Mar 2026 19:40 UTC
3 points
0
on: On The Independence Axiom
I’m sure there is something very important here, but I’m not sure that the independence axiom is the right target.

Take the example that begins at the sentence “Suppose your preference between gambles A and B depends on what the common component C is”. When I observe C, I have new information. There is nothing strange about making my decision between A or B conditional upon new information. The axiom is about irrelevant alternatives, as in the classical example of switching from apple pie to blueberry pie on discovering that cherry pie is also on the menu. But C is described as a “common component”, which does not sound like something irrelevant, but more like discovering that the fruit pie comes with a scoop of ice cream. A fully concrete example would be useful. As it is, there is nothing strange about the plan “if C, do B, otherwise do A”, and no way for this to be money-pumped when I and the would-be money-pumper are both ignorant about the outcome of C.

Similarly in the paragraphs on “Resolute choice” and “Sophisticated choice”. I am not seeing what examples of these might be. Consulting the linked paper on the latter does not enlighten me. From a brief glance, its section III builds in an assumption that the decision maker must fix on a plan that cannot be conditional on information that will be obtained in the future. Clausewitz famously had something to say about that.

The examples that I think tell more against EUT are about long-run gains in non-ergodic systems. One could defend EUT even there by stipulating that the long-term growth rate is the utility of a proposed gambling scheme. However, such manoeuvres would amount to redefining EUT to make it something that still applies in order to preverse the name “EUT”, which is perhaps as futile as redefining phlogiston to mean negative oxygen.

ETA: This from Samuelson’s “Probability, Utility, and the Independence Axiom” is illuminating. The paper on the whole is defending the independence axiom, but near the end he writes:

I might summarize all of the above arguments as follows: the independence axiom must always be applied to a definite set of entities – e.g., (1) single-event money prizes, (2) single-event vectors of goods, (3) single-event money prizes cum gaming and suspense feelings, (4) stationary plateaus of money or of goods-vectors, (5) alternative time profiles of goods, with or without associated suspense sensations, etc. The independence axiom then has implications and restrictions upon choices among such entities; but, strictly speaking, it need not impose restrictions upon some different (and perhaps simpler) set of entities. Thus, if the entities of (5) above are the correct ones, we cannot necessarily make inferences about decisions dealing with information on (1) alone.

...

In what dimensional space are we “really” operating? If every time you find my [independence] axiom falsified, I tell you to go to a space of still higher dimensions, you can legitimately regard my theories as irrefutable and meaningless.

Richard_Kennaway 6 Mar 2026 19:03 UTC
−3 points
−2
in reply to: Taylor G. Lunt’s comment on: Andrew_Critch’s Shortform
Autistic is a dysphemism for sane. Smart, looks at reality, acts effectively.

(Insert Heinlein quote about shining like the Sun vs. a candle.)

ETA: Now I’ve had time to look it up:

“Such men exist, Joe; they are New Man— human in all respects, indistinguishable in appearance or under the scalpel from Homo sap, yet as unlike him in action as the Sun is unlike a single candle.”

ETA2: I’m talking about the present-day use of the word by the general public and the media, not its historical origins or use by psychiatric specialists.

Richard_Kennaway 6 Mar 2026 16:56 UTC
2 points
0
in reply to: Ninety-Three’s comment on: Rational Chess
Hours of mental effort.

Richard_Kennaway 6 Mar 2026 13:59 UTC
2 points
0
in reply to: 849’s comment on: Rational Chess
Taking turns to play White only takes 2 games.

Richard_Kennaway 6 Mar 2026 11:56 UTC
3 points
3
in reply to: Ninety-Three’s comment on: Rational Chess
Conservation of resources. It may not be worth the effort of playing out the game just to get the result one expected.

Richard_Kennaway 6 Mar 2026 11:46 UTC
2 points
0
on: Rational Chess
The usual practice is to chose at random, or in a series of games to alternate who plays White. Simple, no new rules required. Why is this not the best solution?

Richard_Kennaway 5 Mar 2026 15:51 UTC
2 points
0
on: Maybe there’s a pattern here?
When you have more dakka, you use it, because more dakka.

Richard_Kennaway 5 Mar 2026 13:47 UTC
2 points
0
on: Vibe Coding cripples the mind
A couple of quotes that seem relevant:

Lisp … has assisted a number of our most gifted fellow humans in thinking previously impossible thoughts.

Dijkstra, “The Humble Programmer”.

Lisp is worth learning for … the profound enlightenment experience you will have when you finally get it. That experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.

Eric Raymond, “How to become a Hacker”.

In times to come, perhaps we will have to replace “Lisp” by “any actual programming language at all”.

Richard_Kennaway 3 Mar 2026 23:06 UTC
3 points
−2
on: A Tale of Three Contracts
The current state of affairs is Anthropic out and Open AI in with the DoW. Cui bono suggests the hypothesis that the DoW/Anthropic spat was nothing but a scheme put together by Altman and the DoW to get that result. Altman benefits, and I’m sure he can ensure that friends in the DoW will also benefit. Plausible? Lunatic fringe?

Richard_Kennaway 3 Mar 2026 16:40 UTC
2 points
0
in reply to: Raemon’s comment on: The Quantum Immortality Argument Against Suicide (And Why Standard Arguments Fail Rationalists)
What were the tells? In hindsight I can smell it in the way the exposition doesn’t quite hold together at the joints, but it passed me by on first reading.

Near the end there’s the “isn’t just ineffective — it’s actively harmful”, which I believe I would have picked up on if something like that had been near the start, but I didn’t notice anything so blatant elsewhere in the article.

Richard_Kennaway 2 Mar 2026 14:32 UTC
2 points
0
in reply to: AnthonyC’s comment on: An Open Letter to the Department of War and Congress
>Yes, and you [need] these things explicitly stated in public in order to create hard-to-deny common knowledge of them.
You will not dissuade anyone in the regime of doing what they are doing by publicising what they are doing. They are already publicising what they are doing. Trump words are clear enough: “The Leftwing nut jobs at Anthropic have made a DISASTROUS MISTAKE trying to STRONG-ARM the Department of War”.