Marcus Plutowski

Karma: 178

Marcus Plutowski 14 Sep 2026 15:14 UTC
10 points
−1
in reply to: Matrice Jacobine’s comment on: Matrice Jacobine’s Shortform
So much for the AI safety issue not becoming polarized.
I wouldn’t be totally surprised if this flips once Trump is out of office given the weight of popular opinion, c.f. sliding Democratic opinions on immigration. But that’s a long time from now.

Marcus Plutowski 6 Sep 2026 18:26 UTC
1 point
0
in reply to: TsviBT’s comment on: PauseAI Has ‘officially disendorsed’ PauseAI-US
I suspect we’re talking about different groups of people. When you originally said
whenever they interact with someone who’s relatively closer to AI capabilities research
I understood that as people not necessarily working in a lab. Most people I know (working esp. in hw-sw codesign) who are ‘relatively closer’ to this, for example, are people working on kernels, inference hardware, PyTorch, etc. So the cost of “advocating for a stop” is pretty low; for example, I periodically hear other people at my workplace say that it’d be great if they did, uncontroversially given that many of us work on security and so for them to do so would straightforwardly make our working lives easier. But everyone understands that this does ~nothing. So if the ask is for people to go whole hog on “spreading the word” I think people rightfully dismiss it as unserious, in the same way as someone might try to evangelize people to “tell others about Bazel/NixOS/...”. And this is important, since there are many more people working ‘near’ the labs then actually working in them, even if they individually have less power. I agree that for people working on the labs, loudly doing so would be a very different kind of act and thus likely to be taken seriously.
it’s not you resigning, it’s you sitting there lazily and complaining to other people to take a big financial, social, status, and career hit
I do think that it’s necessary for the ‘evangelist’ here to have some bona fides to show their own seriousness, yes. For example an ideal situation would be for someone to speak out, resign, and then ask former colleagues to do the same (within the limits of the law). But there are other ways to prove ‘actual dedication’ more or less credibility; if one were to resign from their comfy job working on widgets to go work at a safety startup, then that would at least lend some credence to future pleas.
Ultimately I do think that you need different messages for people at the labs and for people not at the labs. For lab employees you’re essentially hoping they defect, if fractionally, from the interests of their employer; for others, it suffices to merely align against a group of relative outsiders.

Marcus Plutowski 3 Sep 2026 20:16 UTC
9 points
1
in reply to: Philip Dowdell’s comment on: Philipreal’s Shortform
Clearly someone on his team is rather tuned in; the press release mentions both biorisk and quotes of agents from the OpenAI incident investigation. It’s also interesting that he specifically calls out Anthropic/OpenAI/etc. as violating their self-imposed guardrails:

> Last year, Meta said it would “stop development,” and OpenAI said it would “halt further development” once their technologies reach beyond its ability to operate safely and unless safeguards are in place. Anthropic made a similar commitment in 2023, saying it would “pause the scaling and/or delay the deployment of new models” if the technology outpaces its own guardrails. None of these companies have taken meaningful steps to back up these words. Instead, they are racing to develop more and more advanced AI without proper safety precautions.
There’s been a lot of discussion here (e.g.) as to whether or not Anthropic should be considered to have breached their RSP, so it’s at least notable that Bernie has firmly come down on ‘yes’.

Marcus Plutowski 3 Sep 2026 20:00 UTC
1 point
0
in reply to: TsviBT’s comment on: PauseAI Has ‘officially disendorsed’ PauseAI-US
That’s fair, on second glance my ‘e.g.’ doesn’t actually get to the issue I had with what you started with. To rephrase:

> serious recommendations as to “what ~~should happen~~ they could actually do” to emphasize the seriousness of the problem, followed by a weakening
So for people at the labs, this could be something like (not necessarily my recommendations, just examples that I think more meet the ‘seriousness’ bar people expect for a topic of this gravity) “switch teams to safety, or quit”; for people ‘just’ in tech, e.g. “seriously try switching jobs to work on the safety problem” / “ostracize people who work in the labs”; or otherwise “join my pet protest movement” / “donate to this organization” / etc. Even just as throwaway line, it indicates what you think would be appropriate / actually-helpful in a way that’s harder to square with “they’re just LARPing”. So to amend your original
- AGI would kill everyone. [/ There’s a significant chance that AGI would kill everyone.]
- [/ If that’s true, then, since] AI research is progressing towards AGI, so it’s bad and everyone should stop.
- “People in your position actually matter! If you were to do $COSTLY_THING, it could seriously make a difference.”
- But if not, at the very least you should evaluate whether that’s going to happen, and then either advocate for a stop or else defend in public that it’s safe in a debate with an expert critic.
- If you won’t evaluate this deeply yourself, my request would be for you to repeat this script whenever you interact with someone who’s relatively closer to AI capabilities research.

Marcus Plutowski 3 Sep 2026 19:16 UTC
1 point
0
in reply to: TsviBT’s comment on: PauseAI Has ‘officially disendorsed’ PauseAI-US
B could have charitable surmised that A was operating under various pressures, and felt the need to make a smaller ask than a stop.
While I agree that this structure (Actual Belief + Constraints ⇒ “Unexpectedly Small” Recommendation) is valid in general, I do think that people understand this to be ‘unlikely’, even under a charitable frame, when the ‘Actual Belief’ involves risk to life/limb/etc. This instinct is strong enough that it often gets applied even where it’s illogical: e.g. a well known pattern in medicine goes something like 1. doctor says patient is at risk of heart failure 2. doctor recommends “eat less salt” 3. patient intuitively decides “well this can’t be that serious” and ignores the recommendation 5. patient dies of a heart attack. Tying back to our case study, I think that B is actually being reasonably logical in applying the principle here: they understand that there are constraints, but don’t believe that the sort-of-constraints that people they know experience would be strong enough to blunt “we’re all going to die!” to such a milquetoast recommendation. Consequently they infer that the ‘Actual Belief’ is in fact something closer to LARP’ing, and that A’s recommendation is being made to serve that goal (by drawing B into the LARP).
Ultimately I think it’s important to proffer both: serious recommendations as to “what should happen” to emphasize the seriousness of the problem, followed by a weakening to accommodate practical realities. E.g. “the only safe path forward is to [ban GPU exports/international treaty/unilateral pause/dissolve the labs/invest a billion dollars in AI safety/whatever your favorite solution is], but at the very least you can help out by spreading awareness of the risk”.

Marcus Plutowski 3 Sep 2026 18:51 UTC
8 points
2
in reply to: Shankar Sivarajan’s comment on: Linch’s Shortform
Besides the points made above (that the activities of major labs are more dangerous/reprehensible than those of current world governments), I would also point out that AI labs are much less subject to confusion/diffusion of responsibility than states are. The US government has ~2.68m non-military employees whose activities span the gamut from “make sure old people don’t starve to death” to “torturing prisoners”. While people do continue to try, it has proven very difficult to extricate the bad activities from the good, and so it is very difficult to say that the US government, as a whole, is evil, even if many parts of it, and many of its activities, certainly may be. So an employee could easily say “I’m not working on the bad stuff, and it’s not like me succeeding here will make the bad stuff more successful” and be correct about that.
But with labs like OpenAI and Anthropic, this isn’t the case. There are a few thousand people working there, of which a smaller subset are working directly on research/software/infrastructure. These companies essentially do one thing, which is build/train/serve increasingly capable models and products that leverage them. While the direct impacts of these product offerings do vary, all profits made thereby will feed into training larger models, and the success of these product offerings largely depends on those larger models. The only people that come close to escaping this logic are those working on safety, but to a large extent that seems to operate more like harm reduction than an actual counterbalance to the bad activities of the broader organization; especially considering how their efforts often just serve to enable further development of model capabilities. An analogy would be if the military were to invade another country without provocation, destroying its infrastructure in the process, but thereafter deploying the Army Corps of Engineers to rebuild key portions like bridges and the water supply in order to facilitate a long-term occupation. We could agree that it’s better for the occupied population to have water, but that it would have been better for the invasion to have never happened in the first place, thus making participation in the ACoE morally dubious there. So altogether, it is much easier to assign moral responsibility to people working at the labs than people working for the government: just about everyone involved is fundamentally working to enable something very-bad, they mostly understand that this is very bad, and yet they freely choose to do so, in no small part thanks to how they perceive it as affording personal gain.

Marcus Plutowski 3 Sep 2026 5:33 UTC
4 points
1
in reply to: TsviBT’s comment on: PauseAI Has ‘officially disendorsed’ PauseAI-US
Two of my former colleagues A, B were talking; A brought up AI doom, to which B, clearly familiar with the idea, said something along the lines of “well what should we do about it?”. A proposed a slowdown, and in response B said something like “So you’re okay as long as it happens slowly? That doesn’t sound so bad!” and laughed it off.

Marcus Plutowski 3 Sep 2026 5:27 UTC
9 points
1
in reply to: Shankar Sivarajan’s comment on: PauseAI Has ‘officially disendorsed’ PauseAI-US
I’ll clarify by saying that I’m not advocating for violence etc. but there’s a big gulf between “so do violence” and “respectfully talk to people at the labs”. For example, I could imagine advocating for a sit-in or other form of protest. That at least I can imagine sounding credible.

Marcus Plutowski 3 Sep 2026 3:56 UTC
14 points
11
in reply to: TsviBT’s comment on: PauseAI Has ‘officially disendorsed’ PauseAI-US
I think that this syllogism would seem strange to most people. People who ‘actually’ feel their lives to be in danger generally do not stop at “either advocate for a stop or else defend in public that it’s safe in a debate”. People certainly aren’t just going to devote themselves to convincing “someone who’s relatively closer to AI capabilities research”, given the already dubious moral fiber of said researchers. The natural response to “these people are going to kill you” is at minimum protest and really has no upper bound—especially when thinking in far mode, where difficult things like “gather a militia and make them stop!” are easier to consider.
Consequently, I think that the fact that you advocate for #3/#4 as the solution will convince listeners that you don’t actually believe in #1. I’ve seen this unfold before my very eyes!

Marcus Plutowski 30 Aug 2026 15:33 UTC
6 points
3
on: Why I think polyamory is net negative for most people who try it
Something that struck me (decidedly non-poly) when reading your post was just how strongly you seemed to feel about jealousy. That is to say, you really seemed to want to avoid it. On one level I can understand why one might want to avoid extreme jealousy: in the same way as love is good, but ‘too much’ leads to obsession.
But like love, healthy jealousy seems to me a very good thing! I very much want my partner to be jealous of me, and that feeling of being their ‘special someone’ is wonderful and unique.
Do you still feel this way, that jealousy is broadly negative (regardless of whether you can or cannot get rid of it) or would you say that it was part of the polyamorous experiment?

Marcus Plutowski 22 Aug 2026 11:31 UTC
2 points
2
in reply to: derelict5432’s comment on: We Must Remember That Our World Contains Hell
What does “rationalize their own suffering” mean? If you prioritize an abstract idea of “lives worth living” over individuals’ own desire to continue living, you open up many dangerous possibilities (eg justified euthanasia).

Marcus Plutowski 21 Aug 2026 4:01 UTC
4 points
1
in reply to: derelict5432’s comment on: We Must Remember That Our World Contains Hell
That seems like something worth recalibrating, since people in that position (extant hunter-gatherers) have generally described their lives as worth living.

Marcus Plutowski 13 Jun 2026 6:31 UTC
7 points
0
in reply to: Thane Ruthenis’s comment on: Philipreal’s Shortform
I struggle to understand what the disagreement with this comment is, I’d appreciate it if someone could fill me in. What other logical response would there be if people in the US government came to accept Anthropic’s messaging on the issue? Certainly we can’t expect them to try and reach out to China for collaboration on the topic, given the policies of this government that we’ve seen thus far

Marcus Plutowski 10 Jun 2026 5:42 UTC
1 point
0
in reply to: leogao’s comment on: leogao’s Shortform
You’re going to have a tough time with Dostoyevsky if you think about it as “worshipping religion”. It might help to remember that he was also quite conversant with arguments for atheism, e.g. as he has Ivan espouse in Brothers Karamazov. Not to say I think you should believe like he does, but that he’s not just putting this stuff forward out of blind ignorance.

Marcus Plutowski 28 May 2026 21:33 UTC
1 point
0
in reply to: John-Clark Levin’s comment on: Pope Leo’s First AI Encyclical – Summary and Commentary
That’s fair! On reflection I’m used to dealing with the opposite issue given some of the circles I’m in, so I think my instincts are tuned in the wrong direction and your framing was probably the right choice.

Marcus Plutowski 27 May 2026 15:44 UTC
20 points
3
on: RTMH: Pope Leo’s Magnifica Humanitas on AI
I’m surprised by your confusion at Pope Leo’s juxtaposition of Babel and Nehemiah, as the metaphor seemed rather clear to me, while the ‘candidate interpretations’ you provided feel very modern and out of place. It’s possible that this comes down to differences in exegetical tradition between Christian and Jewish schools of thought, in which case I’m probably not properly equipped to bridge that gap. Assuming not, however, I’ll give a stab at my understanding.
Starting from the text: the builders of Babel say
Then they said, “Come, let us build ourselves a city, and a tower with its top in the heavens, and let us make a name for ourselves; otherwise we shall be scattered abroad upon the face of the whole earth.” The Lord came down to see the city and the tower, which mortals had built. And the Lord said, “Look, they are one people, and they have all one language; and this is only the beginning of what they will do; nothing that they propose to do will now be impossible for them. Come, let us go down, and confuse their language there, so that they will not understand one another’s speech.
There are a few things I want to point out. First, the intentions of the builders are directly self-glorifying: to make a name for themselves, to aggrandize themselves through great works. Secondly, the motivation for the construction is a sense of vulnerability: they fear the prospect of being ‘scattered abroad’, and seek to create an anchoring-point to hold their population in one place. Yet this is directly opposed to God’s prior command to “Be fruitful and multiply, and fill the earth”, and (in Catholic thought, at least) this of command is not made out of pique, but because it is the proper orientation for human life. That is to say: God has laid out a plan for mankind; mankind is afraid of what will happen if they let that plan unfold, because doing so would mean giving up control; they seek to forestall God’s plan through their own power; in doing so, they harm themselves. God’s subsequent intervention merely restores the proper order of the world.
The parallels to AI seem clear. Those who seek to build AI do so largely out of a combination fear, greed, and pride (also greed, but that’s perhaps less relevant for the Babel analogy so I’ll ignore it for now). The leaders of the labs are afraid of China (or another lab) winning a ‘race to AGI’, and all harbor some level of a prideful desire to see their own worldview ‘conquer the lightcone’. The engineers on the ground have FOMO plus the obvious vainglory of working at a cool lab and being acclaimed by their peers. More than either of those though, everyone is afraid of death, and many AI boosters claim that the risk of AI is justified either because it’ll fix death or because we’ll destroy ourselves anyways if we don’t build AGI soon. The Catholic view, of course, is that mankind cannot actually destroy ourselves (because we have to make it to the Second Coming), death is a natural part of life (and indeed how you get to heaven), and that God’s plan should+will determine the future of the lightcone. So AI is the modern Babel because it is man’s attempt to defy God’s plan through a misplaced sense of confidence in our own capabilities and our own sense of what the world ‘should be like’.

Marcus Plutowski 26 May 2026 4:40 UTC
1 point
0
in reply to: makiba’s comment on: What am I, if not an AI?
Out of curiosity, how much did training the Mistral model to the final checkpoint cost you (approx)?

Marcus Plutowski 26 May 2026 3:46 UTC
5 points
0
on: Pope Leo’s First AI Encyclical – Summary and Commentary
Another, separate point:
Eventually, there will be tension between a blanket ban on lethal autonomous weapons and the Church’s concern for alleviating objective suffering. I hope the Church engages with the nuanced question of what principles can minimize actual harm to innocents while preserving ultimate human moral responsibility for uses of force.
I don’t think that there’s going to be much of a conflict here. What you’re describing is a conflict between a utilitarian imperative (minimize suffering / bad outcomes) and one rooted in virtue ethics (or even divine law?). In Catholic philosophy the latter tends to win out. Very rarely do you see Church teachings suggest the sacrifice of principles in favor of practical concerns. An instructive example is the case of abortion, say in the case of a mother without the means to take care of her child, with the likely outcome of a birth being the child living a life of poverty. Many will argue that the ‘misery-minimizing’ choice would be to allow the mother to conduct an abortion (and some rather less savory types, i.e. eugenicists, will even argue that it should be made mandatory). But obviously Catholic teaching does not permit this; abortion can be permitted when it occurs as a side-effect of medical treatment for the mother, but certainly not to ‘alleviate objective suffering’. The Church would instead suggest that people do something to improve the situation of the mother and her child, as it does through various charity networks.
It might be clarifying to note that the Church does not consider suffering to be the ‘worst thing’ (though it is certainly bad); sin is the worst thing, and the Church generally advises people to suffer now rather than sin and (in their eyes) risk eternal damnation later. Now of course, it’s not clear a priori that “using lethal autonomous weapons” is necessarily a sin, insofar as it’s not called out as such in the Bible or anything like that. But Catholic teaching has been moving steadily towards a broad-spectrum rejection of violence, one which harkens back earlier centuries before just war doctrine and other carveouts for the ‘licit’ deployment of force emerged. There was a time when e.g. Byzantine soldiers, fighting on behalf of a Christian emperor, would rush to priests after battle to receive absolution, just in case their killing (in warfare) was against the will of God. Those clarifications that did develop in later centuries, like the Thomist concept of just war, all rely on a carefully considered use of force mediated by human reason. So I would expect that, by default, deployment of violence in a capacity divorced from human reason would be considered sinful, because violence, by default, is sinful. The surprising thing isn’t that the Church considers new forms of warfare to be sinful in ways that old ones are not, it’s that new forms of violence are presumed (by governments, if not the general population) to be OK and only banned if exigent circumstances demand it.
A further analogy would be to consider the use of normal, non-autonomous warfare in pursuit of a (purportedly) noble goal—e.g. to invade a poor and corrupt nation, overthrow its leaders, and impose a more efficient system that can actually provide modern medical care to the general population. This would arguably reduce general suffering, but the Church explicitly does not endorse this. I expect the same rationale to apply to the use of autonomous warfare, even in situations like you describe.

Marcus Plutowski 26 May 2026 3:20 UTC
13 points
0
on: Pope Leo’s First AI Encyclical – Summary and Commentary
Glad to see this post; while I’m sure most Lesswrong posters are at least aware of the encyclical, I expect most have not taken the time to read it. I think that in terms of defining the moral foundations for an anti-AI movement, it’s leagues ahead of other entrants: however you feel about “traditional morality”, the language and structure thereof are intuitively familiar to everyday people in a way that e.g. the lightcone is not. Obviously both rhetorical approaches have their tradeoffs, but when we’re talking about building a political coalition, the former is essential—and, I think, greatly underserved in the AI safety space.

One thing I wanted to poke at, though:
Sidebar: many Americans misunderstand Catholic references to “the common good.” The Church supports healthy private enterprise—this term doesn’t imply communist collectivism. … The Church supports private property and free enterprise as good ways to promote the common good, but insists that these be implemented with a recognition that all people share in a basic right to a just portion of these resources.
You’re not wrong, but I do think that this presentation might be somewhat misleading, or indicate a misunderstanding of what the Church teaches and why. Rerum Novarum explores this in more detail, but at a high level the idea is that private property is important not for its own sake, but insofar as it bolsters human dignity (which is important for its own sake). This means that there is a difference in how, for example, capital equipment and personal goods might be considered under a Catholic social framework. We can see that the latter clearly bolster human dignity: it is more dignified to own one’s own toothbrush, one’s own home, even one’s own retirement savings, than to rely on the state or some other actor (who might take them away). But the same cannot so directly be said about the former (e.g. ownership of a factory, or of the town mill). While Pope Leo XIII was certainly no socialist, he was also certainly anti-capitalist, and introduced the concept of Subsidiarity which later Popes would build on (e.g. in Quadragesimo anno). The essential idea is that local/‘natural’ groupings of people (families and local communities especially) should not be subordinated to a remote authority except when necessary. Obviously this applies to central planning, but it applies equally well to the influence and sway of large (and today, transnational) corporations. Pope Pius XI developed this into a loose theory of ‘distributism’, which really looks a lot more like medieval guild organization than either capitalism or socialism.
Perhaps that was a long way of saying that the Church certainly supports private property of certain kinds, but otherwise its support is largely situational, and not a primary focus. The first priority is always the dignity of the individual.

Marcus Plutowski 23 May 2026 12:16 UTC
1 point
0
in reply to: G Wood’s comment on: If AI is normal technology, history is not reassuring.
That’s not true. We have evidence that after agriculture, individuals were materially less well off, eating not only less food but lower quality food. Even once populations reached their new carrying capacity, this effect persisted, even though (just?) as many children were dying in infancy.