Mitchell_Porter

Karma: 10,078

Mitchell_Porter 20 Mar 2026 1:56 UTC
4 points
0
in reply to: olalonde’s comment on: olalonde’s Shortform
I suggest contacting the International Society of Solipsists, they may have some relevant organizational experience…

Mitchell_Porter 19 Mar 2026 21:24 UTC
3 points
0
in reply to: Stephen Martin’s comment on: Stephen Martin’s Shortform
On a private forum, I just listed possible outcomes and probabilities as:
- US leaves Persian Gulf and Eurasian powers work with Iran to establish new economic and security order there, 10%
- Islamic Republic falls and is followed by new Eurasian democracy crusade ultimately aimed at Russia and China, 60%
- Regime falls but no broader democracy crusade OR regime stays but strait is reopened, 30% combined
These are political scenarios and you’re asking about economics. My intuition is that there could be a world recession but not a world depression.

Mitchell_Porter 18 Mar 2026 1:30 UTC
21 points
2
on: Requiem for a Transhuman Timeline
When did everything go wrong, exactly?
I have said for some time that the problem is much deeper. The human race in general was never on board with transhumanism. The idea of radical life extension has been around for millennia, it has been scientifically plausible for decades if not centuries, but it has always been a marginal concern. There was never a society which organized to make the cure of ageing a major priority.
There has been an incremental improvement over time, both in medical capability (thanks to the progress of conventional medicine) and in openness to life extension (partly thanks to science fiction, perhaps). But it’s almost as if humanity backed its way into this improved situation, under the pressure of immediate concerns (e.g. specific illnesses, individual grief), without ever having consciously adopted a futurist vision like those you describe. At the level of individual psychology, and even more at the level of mass psychology, most people are completely resigned to living out the historically normal human life cycle.
Nonetheless, we actually have a form of transhumanism in power now, but it’s this AI-centric version, half of whose protagonists are in denial about what they are creating. Many of the others think they can skip biology entirely, and just go straight to mind uploading or creation of benevolent AI, or even believe they are in a simulation. This points to a divide within transhumanism itself (and adjacent movements). But socially and politically, I think denial of the full implications of AI, is the main enabling factor. There is no politician who runs for office on the platform of creating non-biological superhuman intelligence. It’s only the tech CEOs who talk directly about anything like that.
I have been contemplating a post about different forms of transhumanism which would go into more detail about all this.
From a very broad perspective, not even focused on Earth, but just on the possible destinies of intelligent life in the cosmos once technology comes into play… It would not be surprising to know that in the encounter with technology, intelligent species often blow up their world or inadvertently replace themselves with a successor species, and only sometimes manage to preserve their own existence and imperatives. It’s just that we also get to live through one instance of such an encounter in person.

Mitchell_Porter 16 Mar 2026 20:10 UTC
4 points
0
on: Formal Metaethics and Metasemantics for AI Alignment
Does anyone know how June Ku is doing? Website is down and no tweets for two years.

Mitchell_Porter 16 Mar 2026 20:04 UTC
2 points
0
in reply to: Cole Wyeth’s comment on: A research agenda for the final year
Having the right simple idea (Darwin, Turing) can be the kernel of everything. The correct ontology could be something like, the correct understanding of entanglement in quantum gravity, plus a precisely stated panprotopsychism that implies consciousness at the human level. The correct meta-ethics may be one of the known proposals (example by a CEV theorist), grounded in the correct ontology’s account of intentionality.
I mention these concrete proposals, not out of commitment to their correctness, but as examples of what the kernel of an answer could look like.

Mitchell_Porter 16 Mar 2026 9:58 UTC
14 points
5
on: Mitchell_Porter’s Shortform
Last month, I posted a “research agenda for the final year”. This was an attempt to rise to the challenge of our situation: AI is advanced enough now, that it would be unsurprising if it reached superintelligence this year. My expectation is that superintelligence rapidly leads to AI takeover of the world and the end of human sovereignty. If all that were true, then we only have months left in which to elevate the theory and practice of alignment to the level needed for superintelligence.
The next step for me is to emphasize that I am available for paid work in this area. That research agenda outlines how I think about the issues; so in principle, I could be hired to work exactly on that agenda, or on some subset of it. But in practice I have considerable flexibility. My main practical constraint is that I am looking for remote work that will allow me to move from Australia, where I am now, to Canada, for family reasons. If this really is the end, that’s where I ought to be. And if we do still have a few more years, being in North America should be a step up in my ability to make a difference.

Mitchell_Porter 15 Mar 2026 17:15 UTC
3 points
0
on: Some models don’t identify with their official name
Could it be this simple—that this is always a sign that an AI has been trained on the output of another? So DeepSeek was trained on ChatGPT, and Kimi was trained on Claude, and Claude was trained in Chinese on DeepSeek?

Mitchell_Porter 11 Mar 2026 15:58 UTC
3 points
0
on: AIs will be used in “unhinged” configurations
Gemini models for example have exhibited repeated looping where the model enters an infinite reasoning cycle, exhausting its token budget while generating thousands of tokens of self-talk
Maybe I’ve just missed the corresponding behaviors from other models… But it seems like Gemini can be “neurotic” (another example) in ways that Claude and ChatGPT don’t normally exhibit. (I won’t even try to compare with Grok’s capricious “personality”, which is in a class by itself.)
Identifying long-term personality trends in AIs is a complicated thing because they are so capable of roleplay. Before he became an LSD guru, Timothy Leary devised a “relativistic” model of personality which emphasized the context-dependence of personality traits; something similar might be appropriate, or even necessary, in order to properly evaluate AI personality.

Mitchell_Porter 11 Mar 2026 1:37 UTC
2 points
0
in reply to: delphix’s comment on: Open letter to doomers
If you permit, I’ll summarize a lot of that as follows: the reason that “rule by AGI” is different is that it is so alien, and we don’t know how to make it significantly less alien.
The argument from alienness still makes sense, but its strength has eroded somewhat in the era of conversational AI. It turned out, not only that directing powerful general pattern-matchers at the human textual corpus gave them the ability to talk like a human being, but that it induced in them an internal conceptual structure that humans are capable of interpreting.
An optimist might say: maybe we can use these techniques to create a first approximation to an anthropomorphically benevolent being, then ask it to devise superior techniques which will be sufficient to create the real thing, trusting that enough concepts have been correctly inferred, for it to figure out what is wrong or missing in our specifications.
This kind of optimism is based on the hope that anthropomorphic benevolence, as a target in the space of possible minds, is surrounded by a “basin of attraction”. All we have to do is land in that basin, i.e. we only need to specify the goal up to a certain degree of accuracy, and provide the task to a mind which is sufficiently close to anthropomorphic, and any details that were wrong or missing will be corrected and filled in, by the intrinsic logic of the problem.
Regarding empathy in particular, I think any mystery pertaining to it is largely because it involves consciousness, and consciousness remains a fundamental problem rather than just a technical problem, for scientific understanding… I spent quite a few years “studying consciousness”, from the perspective of wanting to understand its nature and how it relates to everything else, and this is an area in which I believe in the possibility of a conceptual breakthrough. That is, if the right connections are made, the rift between subjective experience and the naturalistic worldview could be closed completely, and many mysteries would fall into place.
Now, I’m going to cut myself short here, even though there is much more to discuss in your (very helpful) critiques. Hopefully I can return to them. But I just want to say a few things about where I’m coming from.
I have presented these sketchy solutions, or reasons for hope, to a handful of your objections, not because I am decidedly optimistic, but just to indicate where the counterargument lies. We simply do not know whether there are forms of artificial superintelligence which would naturally coexist well with humanity, or whether it’s a tightrope walk to coexistence, no matter what design you use. That uncertainty alone should be reason enough to stop what we’re doing, but that’s not how our elites see things. To those who want to stop the juggernaut, good luck. But as a theory-minded person, I intend to work on the theory of how to steer the juggernaut so it doesn’t crush us. One reason for this focus is that there truly may be very little time.
I’ll be back when I can.

Mitchell_Porter 10 Mar 2026 20:03 UTC
42 points
2
on: Gemma Needs Help
Any idea what GDM is doing, or not doing, that causes its models to emote like this?

Mitchell_Porter 10 Mar 2026 3:56 UTC
3 points
1
in reply to: delphix’s comment on: Open letter to doomers
You present certain arguments according to which having AGI around is inherently intolerable or cursed. But they seem to be getting very general, so general that they could be reasons why a child must not have parents, or a country must not have a government. There just cannot be a power over you, that diverges from you even a little. Could you clarify what’s wrong about “rule by AGI” that doesn’t apply to “rule by parents” or “rule by the state”?

Mitchell_Porter 9 Mar 2026 22:36 UTC
3 points
2
in reply to: delphix’s comment on: Open letter to doomers
There are different concepts of alignment. “Intent alignment” versus “values alignment” is one way to put it. Alignment with user intention means that an AI follows user instructions. Values alignment means that an AI adheres to certain values, even if it is making its own decisions.
The classic idea for aligning a superintelligence is that you determine its values before it is superintelligent, and then once it becomes superintelligent, it doesn’t want to change its values, even if in theory it could do so.
The extent to which this makes sense depends on the architecture of the AI. In a design where there is a very clean separation between the utility function / value system, and the problem-solving intelligence, there seems a good chance that the value system will remain stable even as the intelligence increases. On the other hand, if you have an AI where the decision-making results from a complicated interplay of multiple competing goals, there is much more opportunity for unexpected value systems to emerge.
As spokespersons for opposite sides of this argument, I would pick Steve Omohundro, who I believe has argued that as intelligence increases, there should be an increasing tendency for an explicit and protected value system to emerge, and Guillaume Verdon, who thinks it is more adaptive for an intelligence to remain deeply flexible in its goals. (Maybe @Remmelt and his guru Forrest Landry should also be mentioned in the second camp—they have argued like you, that an AI civilization would necessarily drift from its original imperatives under selection pressures.)
It is also far from clear to me that human values are inscrutably complex. The idea that they are darwinistically produced neurogenetic “spaghetti code” is a familiar one. However nature and biology also contain emergent simplicities. The ethical debate among human beings often circles back to pleasure and pain as the ultimate reference points. Some version of Benthamite hedonistic calculus, where you’re just adding up pains and pleasures in some way, may turn out to be a convergent ideal, not just for human beings, but for many possible forms of conscious mind.
You say that similarity does more than empathy to keep human beings living in positive-sum ways; from a different angle, @RogerDearnaley has recently been arguing that universal altruism is not enough for values alignment because, to paraphrase, predators enjoy eating their prey, even if the prey is a universal altruist. Dearnaley has been arguing that human-friendly values alignment needs to treat aggregate human well-being as the terminal value (i.e. as an intrinsic good), with well-being for other entities (animals, aliens, conscious AI, etc) to follow as a derived good arising from human empathy, and therefore capable of being deprioritizied if their well-being would come at the price of our well-being (or even just our survival).
A further possible consideration is an old perspective due to Eliezer, who in his very early days sometimes said, the ideal in AI is not to create something perfect that then rules the universe for all time, the idea is to ensure that when the torch of intelligence gets passed to something more capable than us, as it inevitably will, that this should be something which is morally and not just epistemically superior to us. It may still be fallible, but if it is less fallible than human, while nonetheless maintaining the imperatives that have defined the best of what we are, then that is a desirable outcome. This is a response to the idea that AI values will inevitably drift: human values drift too, and in disastrous ways. If an AI can learn good human values, while being superhuman in its fidelity to them, then that is better than a future in which e.g. humans just wipe themselves out with advanced technology.
So I don’t agree that the idea of creating aligned superintelligence is necessarily a bad idea for all time. There may be a level of knowledge at which you could do it while genuinely knowing what you’re doing. However, that is not our current world, our current world is more like “do it and hope you can figure out what you need to know along the way”. (Soon it will probably be just, “do it so the AI can save you from the economic and military fiascos you’ve created for yourself”.)
I have something like respect or sympathy for people who want to stop AI completely. I try not to get in their way. But given the widespread proliferation of knowledge on how to make AI, I will keep working to reach that “level of knowledge” at which we would “genuinely know” what we’re doing.

Mitchell_Porter 9 Mar 2026 8:50 UTC
1 point
1
on: Mitchell_Porter’s Shortform
It would be “funny” if it was the war against Iran, and its economic consequences, that burst the AI bubble.

Mitchell_Porter 8 Mar 2026 14:47 UTC
2 points
−1
on: Open letter to doomers
This strategy reminds me a little of the “Direct Institutional Plan” by @Gabriel Alfour, with its emphasis on conversation and directness. The difference is that Alfour’s plan is, first, identify how to stop AI, second, persuade the people who together can stop it (legislators, mostly).
The two concepts are potentially complementary in that you want people who see the risk to improve the quality of their thought and arguments, by talking more simply and freely with each other, whereas Alfour’s strategy presupposes the existence of a concerned community already good enough to devise a political plan and follow through. Apparently Alfour’s organization (ControlAI) has begun to do so in the UK, having spoken with over 100 politicians.
My own outlook is as follows. I expect that further AI progress will produce superintelligence. I don’t know how likely it is that superintelligence leads to doom for humanity, but I do think it very likely that superintelligence leads to AI takeover. So that is the idea that I will introduce and defend, in conversations where it is not present. For example, if I run across a conversation about AI that focuses just on economic impact, I will emphasize that AI means that the economy can live on without humans at all, and with superintelligent AI running the show.
However, I am skeptical that the AI race can even be paused at this point. It’s not impossible but the hour is very late. Economic and geopolitical competition is driving it forward. The American and Chinese governments are, in their different ways, both recklessly enthusiastic in their pursuit of AI progress. If some minor AI-driven catastrophe occurred (minor compared to extinction, but big enough to shock the world), perhaps the shock would introduce some hesitancy at high levels; though if it took place against a backdrop of war such as we are experiencing now, even dramatically bad news from the AI frontier might be lost in the fog of war.
Anyway, my focus is not on the community of doomers, but on the community of people who are driving advances in AI forwards. That primarily means the researchers and developers employed by the frontier AI companies, but it also includes wider and wider circles like their nontechnical colleagues, academic researchers in machine learning, all the users, the investors, influencers, even bot social media like Moltbook, and so forth.
Furthermore, within that community, my focus is on solving alignment—the ambitious form of alignment, in which you want to know, not how to make an AI do what you tell it, but what the ethics of an autonomous superintelligence should even be. I talk about a lot of other things, such as the evolving political and cultural situation around AI, but that is the topic I conceive to be of central importance.
It’s an example of a technical topic (with elements of philosophy, science, and mathematics) where humanity doesn’t even agree on how to pose the questions, let alone find the answers. This is why many people who would support the creation of aligned superintelligence, instead support a pause or a halt, because they don’t think we have time to solve the problem. I try not to get in the way of their activism, but the hour is late and my focus is different. My focus is on the technical community and on trying to solve the problem.
That means conversation but it also means trying to solve subproblems myself. I’ve spent a lot of my life penetrating technical discourses like those at the edge of philosophy, math, and science; I don’t consider anything that’s happening in the AI sphere inherently beyond me (although altogether it is enormously complex); so I do think directly about what alignment of superintelligence involves. The number of other people working on aspects of the problem (including those who are adding to the problem by advancing AI capabilities) is probably in the tens of thousands, possibly hundreds of thousands, and I am a marginal figure (no one is paying me to do any of this), but I am not shut out. On this forum, for example, there are people from all the western companies working on frontier AI. If I had a significant technical result, I could probably get it onto arxiv, which has become the central hub for formal research in AI.
One might say that despite the number of people, solution to the technical problems of superintelligence alignment seems very far off. I think it’s not so clear. It is like the situation in mathematics regarding all those famous unsolved million-dollar problems. The experts may say that the solution looks decades away, and meanwhile you have crackpot amateurs coauthoring AI papers declaring that they have solved all the problems at once. The situation may look dire, if you’re looking for an answer right away. But these domains of pure intellect are precisely the ones where a new idea or result can change the situation overnight.
I would say, not that I have faith that humanity will solve these problems in time, but that I believe in the possibility of breakthrough progress, and that’s even before you take into account contributions arising from the use of AI. As we climb the slope of AI capabilities, AI is being used, and will be used, to tackle those technical subproblems. This is sometimes derogatorily described as hoping that the AI will do our alignment homework for us; but it’s a real thing. You can already ask an AI for its input on any aspect of the alignment problem. Also, in our new era of AI agents, the AIs will increasingly be talking about it with each other.
So, unless a political miracle occurs (from the perspective of those calling for a halt), we are on track towards transformation and AI takeover. There is an unknown probability, which may be high, that this leads to human extinction. For this reason, the current situation of competitively racing towards AI superintelligence, is not the way things should be. However, the human condition is full of situations which should not exist, and which in principle do not have to exist, but which in fact do exist, and very stubbornly so. In the case of the AI race, it is now sustained by the fact that worldwide elites are in constant competition for more power, and AI is seen as a source of power. My choice is to focus on the nature of the resulting superintelligence, and the conditions under which there is actually a good outcome.

Mitchell_Porter 7 Mar 2026 3:49 UTC
4 points
−1
on: Your Causal Variables Are Irreducibly Subjective
What about unsupervised learning and nonparametric statistics, aren’t these an attempt to find fully objective analytical frameworks?

Mitchell_Porter 7 Mar 2026 3:47 UTC
2 points
0
on: [Error communicating with LW2 server]
Your idea doesn’t require Moskovitz to literally be president, he can just be AI czar for whoever ends up as president. Though if he did run, he could thereby draw attention to AI safety in the primaries, the way that Andrew Yang was the UBI candidate in 2020.

Mitchell_Porter 7 Mar 2026 3:15 UTC
6 points
2
on: Podcast: Jeremy Howard is bearish on LLMs
The vastness of the training distribution is certainly one feature of the AI situation. But another is an army of human developers of AI, eager to discover what isn’t in the training distribution, and what the AIs can’t currently do, so they can figure out how to give the AIs those new capabilities.
Is there any argument that LLMs, turned into recurrent networks via chain of thought, will still have inherent limitations when compared with humans?

Mitchell_Porter 4 Mar 2026 19:05 UTC
−10 points
0
in reply to: cousin_it’s comment on: Mass surveillance, red lines, and a crazy weekend
Maybe we’ll see an alliance of Butlerian jihadis with the real jihadis!

Mitchell_Porter 4 Mar 2026 18:45 UTC
2 points
0
on: Sacred values of future AIs
Societies need something like trust to succeed on large scales, but they also need a way to minimize exploitation of trust by cheaters. HHH actually sounds like it could be a core value or orientation of a diverse yet successfully cooperating society of AIs. And maybe your anti-sacralizing ideas could make the HHH society robust against cheaters.
However, I don’t feel like this is very helpful to our current situation of being in the final few moments before superintelligence and wanting to know the values to which it should be aligned. It’s more like a scenario for how an AI world might turn out if it evolved from the present in an unplanned way. (That’s how I feel, others may see it differently.)
I guess a posthuman world with a culture of HHH norms among its sentient beings, is potentially a lot friendlier to humans than many alternatives. It just reminds me of the legacy ethics that humans acquire from their culture. Yes, you could use the Bible’s ten commandments or Facebook’s community guidelines as the table of values for a superintelligence. But those tables of values are a bit contingent, a product of intuition and compromise and experience and guesswork. They may overlook essentials. I have long preferred the CEV ideal that we would systematically obtain values from deeper facts about human nature, even if we are running out of time in which to figure out how to do that.

Mitchell_Porter 2 Mar 2026 17:31 UTC
2 points
0
in reply to: abstractapplic’s comment on: abstractapplic’s Shortform
Well spotted. I had a similar thought recently, that the implications or details of rarely read books are one of the remaining gaps in AI knowledge. This is because it’s not just spelt out in the text, you have to understand the details and think about them. Current training methods don’t process texts that deeply, and if it’s a rare book, there won’t be essays spelling out the lore anywhere in the training corpus.