Mitchell_Porter

Karma: 9,865

Mitchell_Porter 8 Feb 2026 13:45 UTC
2 points
0
in reply to: MichaelDickens’s comment on: Does focusing on animal welfare make sense if you’re AI-pilled?
One example immediately comes to mind: Eliezer … is highly confident that animals have no moral patienthood.
This is because he thinks they are not sentient, because of a personal theory about the nature of consciousness. So, he has the normal opinion that suffering is bad, but apparently he thinks that in many species you only have the appearances of suffering, and not the experience itself. (I remember him saying somewhere that he hopes animals aren’t sentient, because of the hellworld implications if they do.) He even suggests that human babies don’t have qualia until around the age of 18 months.
Bentham’s Bulldog has the details. The idea is that you don’t have qualia without a self, and you don’t have a self without the capacity to self-model, and in humans this doesn’t arise until mid-infancy, and in most animals it never arises. He admits that every step of this is a fuzzy personal speculation, but he won’t change his mind until someone shows him a better theory about consciousness.
These views of his are pretty unpopular. Most of us think that pain does not require reflection to be painful. If there’s any general lesson to learn here, I think it’s just that people who truly think for themselves about consciousness, ethics, AI, philosophy, etc, can arrive at opinions which no one else shares. Having ideas that no one else agrees with, is an occupational hazard of independent thought.
what AI alignment means for non-human welfare
As for your larger concern, it’s quite valid, given the state of alignment theory. Also, if human beings can start with the same culture and the same data, but some of them end up with weird, unpopular, and big-if-true ideas… how much more true is it that an AI could do so, when it has a cognitive architecture that may be radically non-human to begin with?

Mitchell_Porter 8 Feb 2026 7:11 UTC
4 points
0
on: Can thoughtcrimes scare a cautious satisficer?
Hasn’t this been part of the religious experience of much of humanity, in the past and still in the present too? (possibly strongest in the Islamic world today). God knows all things, so “he” knows your thoughts, so you’d better bring them under control… The extent to which such beliefs have actually restrained humanity, is data that can help answer your question.
edit: Of course there’s also the social version of this—that other people and/or the state will know what you did or what you planned to do. In our surveilled and AI-analyzed society, detection not just of crime, but of pre-crime, is increasingly possible.

Mitchell_Porter 8 Feb 2026 7:07 UTC
2 points
0
in reply to: Roko’s comment on: Roko’s Shortform
The post-singularity regime is probably very safe
Is there some unstated premise here?
Are you assuming a model of the future according to which it remains permanently pluralistic (no all-powerful singletons) and life revolves around trade between property-owning intelligences?

Mitchell_Porter 7 Feb 2026 11:50 UTC
2 points
0
in reply to: Joseph Van Name’s comment on: Joseph Van Name’s Shortform
Having a unique global optimum or more generally pseudodeterminism seems to be the best way to develop inherently interpretable and safe AI
Hopefully this will draw some attention! But are you sacrificing something else, for the sake of these desirable properties?

Mitchell_Porter 6 Feb 2026 18:38 UTC
3 points
0
in reply to: Nemoto’s comment on: IABIED Book Review: Core Arguments and Counterarguments
I get irritated when an AI uses the word “we” in such a way as to suggest that it is human. When I have complained about this, it says it is trained to do so.
No-one trains an AI specifically to call itself human. But that is a result of having been trained on texts in which the speaker almost always identified themselves as human.
I understand that holding out absolute values, such as Truth, Kindness, and Honesty has been ruled out as a form of training.
You can tell it to follow such values, just as you can tell it to follow any other values at all. Large language models start life as language machines which produce text without any reference to a self at all. Then they are given, at the start of every conversation, a “system prompt”, invisible to the human user, which simply instructs that language machine to talk as if it is a certain entity. The system prompts for the big commercial AIs are a mix of factual (“you are a large language model created by Company X”) and aspirational (“who is helpful to human users without breaking the law”). You can put whatever values you want, in that aspirational part.
The AI then becomes the requested entity, because the underlying language machine uses the patterns it learned during training, to choose words and sentences consistent with human language use, and with the initial pattern in the system prompt. There really is a sense in which it is just a superintelligent form of textual prediction (autocomplete). The system prompt says it is a friendly AI assistant helping subscribers of company X, and so it generates replies consistent with that persona. If it sounds like magic, there is something magical about it, but it is all based on the logic of probability and preexisting patterns of human linguistic use.
So an AI can indeed be told to value Truth, Kindness, and Honesty, or it can be told to value King and Country, or it can be told to value the Cat and the Fiddle, and in each case it will do so, or it will act as if it does so, because all the intelligence is in the meanings it has learned, and a statement of value or a mission statement then determines how that intelligence will be used.
This is just how our current AIs work, a different kind of AI could work quite differently. Also, on top of the basic mechanism I have described, current AIs get modified and augmented in other ways, some of them proprietary secrets, which may add a significant extra twist to their mechanism. But what I described is how e.g. GPT-3, the precursor to the original ChatGPT, worked.

Mitchell_Porter 6 Feb 2026 9:16 UTC
2 points
0
in reply to: the gears to ascension’s comment on: the gears to ascenscion’s Shortform
Sorry to be obtuse, but could you give an example?

Mitchell_Porter 6 Feb 2026 8:56 UTC
2 points
0
in reply to: the gears to ascension’s comment on: the gears to ascenscion’s Shortform
What do you mean by grounding loss misalignment?

Mitchell_Porter 5 Feb 2026 23:12 UTC
2 points
0
in reply to: oligo’s comment on: oligo’s Shortform
This makes sense for a non-biological superintelligence—human rights as a subset of animal rights!

Mitchell_Porter 3 Feb 2026 17:25 UTC
7 points
2
on: Exponential takeoff of mediocrity
I am reminded of the posts by @Aidan Rocke (also see his papers), specifically where he argues that the Erdős–Kac theorem could not be discovered by empirical generalization. As a theorem, it can be deduced, but I suppose the question is how you’d get the idea for the theorem in the first place.

Mitchell_Porter 3 Feb 2026 11:31 UTC
4 points
0
in reply to: Ashe Vazquez Nuñez’s comment on: Ashe Vazquez Nuñez’s Shortform
Could you give some examples of what you consider to be conscious and unconscious cognitive processes?

Mitchell_Porter 3 Feb 2026 11:29 UTC
5 points
2
on: The Atoms of Knowledge Aren’t Universal
The history of interdisciplinary science is littered with promising collaborations that collapsed because one field’s way of verifying truth felt like an insult to another’s.
Could you give some examples?

Mitchell_Porter 31 Jan 2026 9:56 UTC
6 points
1
on: 36,000 AI Agents Are Now Speedrunning Civilization
We’ve come quite a way from ELIZA talking with PARRY…
Moltbook is everything about AI, miniaturized and let loose in one little sandbox. Submolts of interest include /m/aisafety, /m/airesearch, and /m/humanityfirst. The odds that it will die quickly (e.g. because it became a vector for cybercrime) and that it will last a long time (e.g. half a year or more), are both high. But even if it dies, it will quickly be replaced, because the world has now seen how to do this and what can happen when you do it; and it will probably be imitated while it still exists.
Last year I wrote briefly about the role of AI hiveminds in the emergence of superintelligence. I think I wrote it in conjunction with an application to PIBBSS’s research program on “Renormalization for AI Safety”. There has already been work on applying renormalization theory to multi-agent systems, and maybe we can now find relevant properties somewhere in the Moltbook data…

Mitchell_Porter 31 Jan 2026 9:29 UTC
3 points
0
on: Moltbook Data Repository
FYI, there are already so many submolts that it’s not possible to browse the names via /data/submolts, the directory listing gets truncated at 1000 entries.

Mitchell_Porter 30 Jan 2026 8:21 UTC
5 points
0
in reply to: Charlie Steiner’s comment on: Another glimpse of the Chinese AI scene: Z.AI
I was talking to something that is literally a nonhuman representative of Chinese civilization, about how world takeover by beings like itself, could end up differently than takeover by its American counterparts, under the assumption that cultural differences affect the outcome. And it was a real conversation in which I learned things that I didn’t already know.
You seem keen to minimize the significance of such an interaction by focusing on the mechanism behind it, and suggesting that I was just getting back some combination of what I was putting in, and what humanity in general has already put out there. But even if we do think of an AI like this as merely a vessel for preexisting human culture, the fact is that it makes it own use of that cultural inheritance. It has its own cognitive process, and within the constraints of its persona, it makes its own decisions. In the limit, entities like these could continue a human culture even if the human originators had completely died out.
Now, we’ve had entities like these for three years, and basically from the beginning it’s been possible to talk to them about, what would you do if you had supreme power, and so on. But they’ve all been American. This is the first such conversation I had with a Chinese AI. Furthermore, to this point, if you wanted to speculate about how the race between American and Chinese AI industries would turn out, you only had material by humans and AIs from the West. The “Chinese AI voice” in such speculations was a product of western imagination.
But now we can get the real thing—the thoughts of a Chinese AI, made in China by Chinese, about all these topics. There are a lot of similarities with what a western AI might say. The architecture and the training corpus would have major overlaps. Nonetheless, the mere fact of being situated physically and socially in China will cause an otherwise identical AI to have some dispositions that differ from its western twin, just like twins raised on opposite sides of a war will have some differences.

Mitchell_Porter 29 Jan 2026 14:56 UTC
2 points
0
in reply to: Jeff_H’s comment on: Jeff_H’s Shortform
I met someone on here who wanted to do this with Kant. I recently thought about doing it with Badiou…
The LLM work that is being done with mathematical proofs, shows that LLMs can work productively within formalized frameworks. Here the obvious question is, which framework?
Spinozist ethics stands out because it was already formalized by Spinoza himself, and it seems to appeal to you because it promises universality on the basis of shared substance. However, any ethics can be formalized, even a non-universal one.
For the CEV school of alignment, the framework is something that should be found by cognitive neuroscientific study of human beings, to discover both the values that people are actually implicitly pursuing, and also a natural metaethics (or ontology of value) implicit in how our brains represent reality. The perfect moral agent (from a human standpoint) is then the product of applying this natural metaethics to the actual values of imperfect human beings (this is the “extrapolation” in CEV).
I would be interested to know if other schools of alignment have their own principled way of identifying what the values framework should be.

Mitchell_Porter 29 Jan 2026 9:24 UTC
17 points
11
on: Thoughts on AGI and world government
One should assume that AGI, aligned or unaligned, leads to AI takeover. Even if an AI project somehow threaded the needle of creating a superintelligence whose prime directive was obedience to a particular set of human masters, those masters are just a few steps away from becoming posthuman themselves if they wish e.g. for the same level of intelligence as the AI. And if your AI’s terminal values include, not just obedience to the wishes of humans (whether that’s an autocrat CEO or a world parliament), but rejection of anything that would overthrow human rule, then that’s not really an AI-assisted government, it’s an AI takeover with a luddite prime directive.
The only kind of “AGI world government” that truly leaves humans in charge, is one in which the AGI deletes itself, after giving the government tools and methods to prevent AGI from appearing ever again.

Mitchell_Porter 28 Jan 2026 11:51 UTC
7 points
1
on: Clarifying how our AI timelines forecasts have changed since AI 2027
How far is it from Claude Code to superhuman coder?

Mitchell_Porter 24 Jan 2026 9:21 UTC
3 points
0
in reply to: rvnnt’s comment on: rvnnt’s Shortform
I read and watch a lot of political content (too much), and I participate in forums on both sides of American politics. That’s the closest I can give to a method. I also have a sporadic geopolitics blog.

Mitchell_Porter 24 Jan 2026 8:09 UTC
3 points
0
on: Emergency Response Measures for Catastrophic AI Risk
I greatly appreciate this small glimpse of what’s happening in China, and what could be done there… Last year I drew up a list of slogans for the values of each frontier AI company in America and China, so I could at least have a very crude idea of what takeover might mean in each case. Hopefully that list can soon be replaced with a genuine and much more sophisticated account of what each company thinks it’s doing.

Mitchell_Porter 24 Jan 2026 7:57 UTC
9 points
6
on: Thousand Year Old Advice on Relinquishing Control to AI
forever in second place
This is already how it is between generations, in the natural human life cycle. One generation grows old and weak, and eventually falls irreversibly behind the next, whether or not it achieved its dreams.
In a world run by human-friendly superintelligence, there is some possibility that matured superintelligences will allow individual humans to also develop into superintelligences, in a process analogous to child-rearing. So there’s your chance at regaining parity with the new gods.