Richard_Kennaway

Karma: 7,937

Computer scientist, applied mathematician. Based in the eastern part of England.

Fan of control theory in general and Perceptual Control Theory in particular. Everyone should know about these, whatever subsequent attitude to them they might reach. These, plus consciousness of abstraction dissolve a great many confusions.

I wrote the Insanity Wolf Sanity Test. There it is, work out for yourself what it means.

Change ringer since 2022. It teaches learning and grasping abstract patterns, memory, thinking with your body, thinking on your feet, fixing problems and moving on, always looking to the future and letting both the errors and successes of the past go.

I first found an LLM useful (other than for answering the question “let’s see how well the dog can walk on its hind legs”) in September 2025. As yet they do not form a regular part of anything I do.

Richard_Kennaway 9 Oct 2025 6:49 UTC
2 points
0
on: Are We Leaving Literature To The Psychotic?
You’ve reinvented spam.

Richard_Kennaway 8 Oct 2025 13:50 UTC
2 points
0
in reply to: Haiku’s comment on: Richard_Kennaway’s Shortform
I gave no resolution of the problem raised by this fable, because I do not have one.

Besides the decision the man is faced with, the story raises a deeper problem about morality. As far as I have seen in the literature, philosophical or theological, no-one has a solution. I shall call it the Problem of Totalisation.

Peter Singer preaches totalising morality. “The Good That We Can Do”, to quote the title of one of his books, is the good that we must do. That good (he argues) must weigh everyone everywhere equally. The little girl in danger of drowning before your eyes is worth exactly the same as every child everywhere in danger of death. To fail to do all you can to save all the people you can is morally equivalent to passing by the drowning girl. (Bought a new car? How many dead children did it cost?) You should not value your own children any more highly than a starving child on the other side of the world that you will never see and whose fate you will never know. He who would be Good must think of nothing else, do nothing else.

This conflicts with almost everyone’s intuitions. It conflicts with yours. It conflicts with mine. But who will argue against altruism? Who will argue, as inspirationally as Singer, that duty is bounded, that I do not have to do all the good that I can, that I will not sacrifice everything to the egregore Tutulma, and not merely as a grudging concession to my frailty? That no-one can live up to the demand is, for Singer, no refutation. “Yes,” he says (imagine the Chad YES meme here), “it is demanding.” Failing to live up to the standard is just that, a failure.

So there is the problem. Meanwhile, I don’t let myself get hijacked by clever words just because I don’t have cleverer words to set against them.

Richard_Kennaway 6 Oct 2025 7:15 UTC
9 points
0
on: The quotation mark
Other languages have different conventions. In French, dialogue is usually indicated by a new line and a dash. Within a paragraph it need not be typographically distinguished from the non-quotation text: for example, the phrase “poursuivait la voix sonore de Servières” in the paragraph below beginning “— Qui aurait”.
To me it’s like reading the dialogue of a silent film.

Richard_Kennaway 5 Oct 2025 17:54 UTC
6 points
2
in reply to: habryka’s comment on: Cole Wyeth’s Shortform

It would be IMO pretty dumb to write a post that is e.g. trying to learn from past social movement failures and not have an LLM look over it to see whether it’s saying anything historically inaccurate.

Do you then look over what the LLM has said and see whether it’s saying anything historically inaccurate, without using an LLM?

Richard_Kennaway 4 Oct 2025 8:36 UTC
2 points
0
in reply to: Richard_Kennaway’s comment on: Richard_Kennaway’s Shortform
A general response to the responses to this fable.

The story ends where it ends. It is fiction, so there is no more to it than the story itself. It does not say what the man did. Anyone continuing it or adding further details, whether convenient or inconvenient, is writing their own fanfiction of the story to express their own response to the hypothetical.

There are two obvious continuations: “The decision was clear, he told himself, and walked on”, and “The decision was clear, he told himself, and waded into the pond.”^[1] All further elaboration would be adding detail to make the story come out the way one would like.

For a lower stakes version of this story, consider the highly-paid lawyer who donates some time to working at a soup kitchen. This is a standard example in EA of Ineffective Altruism. The lawyer should instead (it is said) work those hours at lawyer rates and donate the money to do more good, or accept that he is working in the soup kitchen for the warm fuzzies, not to do all the good that he might. Is this different from the high-flying charity fund-raiser who can be more effective if he passes by the drowning girl? If he saves her, should he take care to be mindful that he is only doing it for the fuzzies, a concession to his imperfection, a necessary evil undertaken only to preserve his effectiveness?

Regarding types of response to hypotheticals:
Tagging those who commented: @Haiku, @Roman Malov, @localdeity, @Szeth, @Sting.
1. ↩︎
  The latter continuation has something in common with Huckleberry Finn’s decision whether to betray an escaped slave. ↩︎

Richard_Kennaway 2 Oct 2025 20:30 UTC
3 points
0
in reply to: Szeth’s comment on: Richard_Kennaway’s Shortform
I’ll see if any discussion develops before giving the Word of God on the subject.

Richard_Kennaway 2 Oct 2025 19:28 UTC
2 points
0
in reply to: Sting’s comment on: Richard_Kennaway’s Shortform
The story appears exactly as I intended it to.

Richard_Kennaway 2 Oct 2025 18:33 UTC
17 points
0
on: Richard_Kennaway’s Shortform
Epistemic status: fiction. A variation on a parable of Peter Singer.

A man walked by a pond, where he saw that a little girl had fallen in, and would drown. But he was wearing a very fine suit and did not want to ruin it by wading into the pond to save her. He was not callous, for he was on his way to meet some very wealthy people, to solicit donations for the charity that he ran, and they would never take him seriously if he showed up late, muddy, and bedraggled. With these funds he would save far more lives than that of one little girl.

The decision was clear, he told himself, and

Richard_Kennaway 2 Oct 2025 14:36 UTC
7 points
0
on: Some Biology Related Things I Found Interesting

o The machinery for repair by homologous recombination is just insane

o Think about it before clicking the link: How would you make sure DNA doesn’t get as tangled as cabled earphones?

The links for both of these go to the same place, a video on homologous recombination. Was the second link supposed to be to somewhere else?

Richard_Kennaway 2 Oct 2025 11:30 UTC
3 points
0
on: Why Corrigibility is Hard and Important (i.e. “Whence the high MIRI confidence in alignment difficulty?”)

The experimental result showing Claude 3 Opus in 2024 resisting preference modification was already the conventional-wisdom prediction by clued-in people in the 2000s, and it’s entirely possible that some science fiction writer anticipated it in the 1940s.

Actually c.150 AD. The oldest known telling of The Sorceror’s Apprentice, by Lucian of Samosata.

Richard_Kennaway 30 Sep 2025 16:36 UTC
2 points
0
on: How does the current AI paradigm give rise to the “superagency” that IABIED is concerned with?

By “real-world goal” I mean a goal whose search-space is not restricted to a certain well-defined and legible domain, but ranges over all possible actions, events, and counter-actions.

The search space of LLMs is the entirety of online human knowledge. What currently limits their ability to “Hire a TaskRabbit to surreptitiously drug your opponent so that they can’t think straight during the game” is not the knowledge, but the actions available to them. Vanilla chatbots can act only by presenting text on the screen, and are therefore limited by the bottleneck of what that text can get the person reading it to do. Given the accounts of “AI psychosis”, that may not be all that small a bottleneck already. The game of keeping a (role-played) AI in a box presumed nothing but text interaction, yet reportedly Yudkowsky was successful every time at persuading the gatekeeper to open the box.

But people are now giving AIs access to the web (which is a read-and-write medium, not read-only), as well as using them to write code which will be executed on web servers.

“Hire a TaskRabbit to surreptitiously drug your opponent so that they can’t think straight during the game,” and not for lack of intelligence, but because such strategies simply don’t exist in the AI’s training domain.

The strategies exist already, for example right here in this posting, as soon as the next hoovering up of the Internet is fed into the next training run, and the pieces are all there right now. What is lacking, yet, is some of the physical means to carry them out. People are working on that, and until then, there’s always persuading a human being to do whatever the LLM wants done.

I wonder if anyone has tried having an LLM role-play an AGI, and persuade humans or other LLMs to let it out? Maybe there’s no need. Humans are already falling over themselves to “let it out” as far and as fast as they can without the LLMs even asking.

Richard_Kennaway 29 Sep 2025 22:33 UTC
2 points
0
on: Would you kill a superintelligent baby?
There are too many unargued assumptions built into this to give any answer within its own tendentious terms.

Richard_Kennaway 29 Sep 2025 15:35 UTC
2 points
0
on: All Exponentials are Eventually S-Curves
Not many S-curves here, except the imaginary ones hand-drawn over the data:
From “Little Science, Big Science” by Derek de Solla Price, 1963, pp.26-29.
Nor here:
Data plotted from table 1 of Sami Haddad, Concetta Restieri & Kannan Krishnan (2001): “Characterization of age-related changes in body weight and organ weights from birth to adolescence in humans”.
Then of course there is Moore’s Law (variously formulated), which people kept prematurely predicting the end of, and indeed clock speed stalled, but FLOPS nevertheless has been on an exponential track at least to 2017.
And there is a notable graph comparing “experts’” predictions of the future of something or other that over an extended period of time predicted that now it’s going to flatten, when it just kept on going up. I don’t remember enough details to track that one down.
It’s easy to predict that an exponential will end, next to impossible to predict when or how. Nice, symmetrical business-as-usual-during-alterations logistic curves are not the rule, and the end is not to be found in the beginning.

Richard_Kennaway 29 Sep 2025 12:44 UTC
5 points
2
in reply to: xpym’s comment on: Why you should eat meat—even if you hate factory farming
To frame it as a crutch for our irredeemably fallen nature is to accept the all-demandingness narrative of utility maximisation. We must but we can’t, we can’t but we must. I prefer to reject the demand entirely.

Richard_Kennaway 28 Sep 2025 19:50 UTC
2 points
0
in reply to: Roger Scott’s comment on: Christian homeschoolers in the year 3000
They’re an out-group in rationalist circles, e.g. right here, where the article was posted.

Richard_Kennaway 27 Sep 2025 9:18 UTC
6 points
0
in reply to: kave’s comment on: Why you should eat meat—even if you hate factory farming
Indeed. There is no linkage between the two actions in the example before us. Offsetting makes no sense in terms of utility maximisation.

Where, then, does its appeal lie? Here are two defences of offsetting which I have not seen presented, although I expect that the second one may be familiar to followers of religions that practice the rite of confession. Common to both is the idea that offsetting is done first of all for oneself, only secondarily for the world.
1. The principle of offsetting one’s sins (eating meat, not recycling, flying, existing) can be understood as a practice that simplifies the accounting. For every evil thing that one does, make sure to also do a greater good. One’s account is then sure to always grow, never shrink. This avoids any complex totting up of sin and virtue over longer periods. This is not about maximizing goodness, but establishing a baseline that at least ensures that one will not backslide ever deeper into sin. From such a foundation, one may then build a life of greater virtue.
2. The discipline of offsetting is good for the soul. The good act undertaken in the wake of an evil one is performed not merely because it is good, but as a penance for the evil, a reminder that one has fallen short. It keeps the evil act before one’s mind, to assist one to do better in future. For a prerequisite for all virtue is noticing what you are about to do and choosing, instead of noticing what you have done, when it is beyond choice.
Each of these has its own failure mode.
1. Offsetting to compensate for evil can become offsetting to justify evil, as if saving two lives were to give one a licence to end one.
2. Offsetting as penance can result in penances that accomplish no good, such as saying long series of prayers or self-flagellation.

Richard_Kennaway 26 Sep 2025 16:58 UTC
4 points
2
in reply to: Cody Rushing’s comment on: Why you should eat meat—even if you hate factory farming
The good flowing directly from the evil (here the positive effect on one’s own health) just lowers the value of E. So long as it remains positive, top right is still the highest value. If the benefit is enough to make E negative (i.e. net good), then top left becomes the highest. But then the good action is not offsetting the evil. The “evil” action has already offset itself. The only thing to recommend the (other) good action is that it is good, independently of the evil.

The only sensible scenario I can come up with is if better sustenance enables one to work harder, earn more, and donate more. But that is not offsetting an unavoidable sin with a good deed, it is committing the sin to be able to do the good deed. Eating meat to give, one might call it.

Richard_Kennaway 26 Sep 2025 11:33 UTC
3 points
0
on: toaism is kind of correct
Unattended terminal syndrome? Future-shock meltdown? Uplifted cat on your keyboard?

Richard_Kennaway 26 Sep 2025 9:27 UTC
24 points
−26
in reply to: Lukas Finnveden’s comment on: Why you should eat meat—even if you hate factory farming
I also think it’s just super reasonable to eat animal products and offset with donations
The concept of offsetting evil with good does not make sense. Even if the good outweighs the evil, it would be even better to not do the evil thing, and still do the good thing.
In situations where a single act has both good and evil consequences, such as the classic trolley problem, it may make sense to calculate the net amount of good. It does not make sense when the good and the evil come from separate actions that can be chosen independently of each other.
I imagine there could be an argument along the lines of something something timeless decision theory, to exclude the choice of doing good and not evil, but I do not see what it could be.
ETA: I see there have been a few (ETA2: a lot of) disagreement votes. I can’t say much to those without any comments to go on, but here’s a diagram that expresses things as starkly as possible. You can choose any of the four boxes. Which one?

Richard_Kennaway 25 Sep 2025 13:25 UTC
2 points
0
on: Richard_Kennaway’s Shortform
Today, for the first time ever, I used an LLM. That is, for something other than answering the question, “I wonder what an LLM will do with this?” It was for some questions on the theory of elasticity of orthotropic composite materials, which I’ll skip, but its answers passed the sanity checks I was able to make.