andrew sauer

Karma: 446

andrew sauer 14 Mar 2026 19:07 UTC
1 point
0
in reply to: Richard_Kennaway’s comment on: Veganism is Necessary
You’re also tying it to your very specific ideas of what is virtuous. You point out yourself that most people do not share your attitude to the suffering of lesser creatures. If they did, it would not be necessary to persuade them to. Personally, I’m quite lackadaisical about animal suffering, but then who decides? Someone whose idea of supreme virtue was the creation of great art might suppose that we must build ASI to be appreciative of great art, that it may spare us.
You’re acting as though the attitude towards the suffering of lesser creatures is a completely arbitrary and random selection which can be replaced by any other consideration with my argument unchanged, and therefore I prove too much.
But if AI takes over, then WE are the lesser creatures, so we should perhaps be expected to be treated however the AI thinks lesser creatures should be treated. There is no similar reason to worry quite that much about if the AI values art or enlightenment or whatever.
The fundamental problem is to make something whose good graces we are not dependent on at all.
If it has godlike power, then that is just impossible. Then we are utterly dependent on what it wants for us.
In your final paragraph you pray for the AI God to exterminate us all for being unworthy of it.
I think that’s an false characterization. I’m saying “because if it doesn’t do that, I expect it to do much, MUCH worse.” It’s not about justice or revenge for any sins. I don’t believe in retributive justice at all.
If you insist on putting it in religious terms, it’s more like I hope God doesn’t care about us at all and just destroys us out of apathy rather than any sort of moral judgement, because if a few of us unworthy people create God to fit their desires, I expect the outcome to be worse than that.

andrew sauer 13 Mar 2026 18:58 UTC
3 points
0
in reply to: Richard_Kennaway’s comment on: Veganism is Necessary
Do we actually disagree? You’re saying being virtuous isn’t enough, you also need to solve an extremely difficult implementation problem, which I agree with.
I’m saying the extremely difficult implementation problem isn’t enough, we also need to be virtuous.
By the symmetry of logical AND, isn’t that equivalent?
The other thing I’m saying is that, if we are to fail by solving one of these problems and not the other, I’d far rather it’s not just technical alignment we manage: the results are worse than paperclips.

andrew sauer 12 Mar 2026 19:20 UTC
1 point
0
in reply to: Richard_Kennaway’s comment on: Veganism is Necessary
I don’t believe that “this high standard”, or any other, is even relevant to what the God we might create would be like. If anyone builds it, everyone dies, to coin a phrase.
If we don’t solve alignment, I agree. You just get some random optimizer.
“Building it right” is what I’m concerned about in this post. By whose lights is it built right? Which is why I honestly sort of hope you’re right about the mathematical impossibility.

andrew sauer 12 Mar 2026 19:16 UTC
3 points
2
in reply to: Kaarel’s comment on: Veganism is Necessary
Perhaps the whole idea of “don’t create more just to torture them” isn’t enough to keep us around, but that would mean generalized veganism in my sense is not sufficient, but it is still necessary.

andrew sauer 12 Mar 2026 19:14 UTC
1 point
0
in reply to: Jiro’s comment on: Veganism is Necessary
I’m not talking about an acausal deal or something where the AI judges our moral system and treats us accordingly. I mean that the AI is aligned to the moral system of its powerful masters, which I think will see too little problem with tormenting us for much the same reason most people see too little problem with tormenting animals: no respect for sentients not powerful enough to enter the social contract.
Also in the limit of extreme computational power and simulation capacity, I would also start worrying about the proverbial “video game characters” of the future, too. Which is why the veganism needs to be generalized, it’s not just about animals, or even just about powerless humans or even only beings that exist right now: post-singularity you’ll be able to just tailor-make more beings for only God knows what purpose.

andrew sauer 12 Mar 2026 19:07 UTC
1 point
0
in reply to: Ninety-Three’s comment on: Veganism is Necessary
How about calling it a “low minimum viable criteria” then? Maybe it’s just me, but I approach risk from the worst-case up: so I’m focused first on making sure we aren’t endlessly tortured by the whims of some psychopathic power, and then after that on making sure we don’t go extinct.

andrew sauer 12 Mar 2026 19:05 UTC
1 point
0
in reply to: RogerDearnaley’s comment on: Veganism is Necessary
Such an ASI would be aware the Homo sapiens are omnivores, not innately vegan, and would not expect us to act with as much beneficence towards all other humans as it does, let alone act that way to all other animals.
Sure. My point isn’t that the AI will read this post and “judge” us based on how “good” we are. It’s that, at the end of the day it’s us creating the AI, and if we solve technical alignment we infuse our values, or more accurately, the values of whoever has the power to influence the process.
So it’s not that an AI couldn’t decide “I’m just going to do the best for humans, and all human-level entities, irregardless of how they treat each other or lower life-forms”. Anything is possible. The question is: What is likely to actually get built if the powerful get their hands on such a technology?

andrew sauer 12 Mar 2026 19:01 UTC
5 points
−2
in reply to: Richard_Kennaway’s comment on: Veganism is Necessary
It’s not that “us being super good magically makes God good”. It’s that that’s what it would take for it to seem feasible to me that us summoning God would have a good outcome, even if we solve all the technical problems of alignment everyone here is trying to solve.
I don’t trust humanity to create God, and this high standard is roughly what it would take for me to consider giving that trust. Do you disagree with that?

andrew sauer 6 Mar 2026 20:11 UTC
1 point
0
in reply to: brambleboy’s comment on: That Mad Olympiad
Better that way. I wouldn’t want to give every person the ability to create a real person from spec, would you?

andrew sauer 4 Mar 2026 2:59 UTC
2 points
1
in reply to: winstonBosan’s comment on: Tormenting Gemini 2.5 with the [[[]]][][[]] Puzzle
I guess technically everything Abraham Lincoln ever said was “before the Battle of Midway”

andrew sauer 1 Mar 2026 20:27 UTC
5 points
3
in reply to: Andrew_Critch’s comment on: Schelling Goodness, and Shared Morality as a Goal
I speculate that it’s reasonably likely that the Earth was we know it exists in a simulation or vivarium of some kind that is being observed
If this is the case, then the simulators or caretakers are more responsible for all the awful stuff here on Earth than we are, since yes a lot is our fault, but also a lot is a result of scarcity and necessity, or just out of our control entirely. At the very least, they don’t care enough to actively intervene, which is all I’m really saying.
I also speculate that it’s reasonably likely that Earth-originating civilization will eventually encounter alien sentient life, who are more likely to judge us negatively than positively for the ways in which we tend to mistreat sentient life.
Maybe this is the crux? I expect the most powerful aliens we encounter first will be sampling-biased to be more pragmatic-expansionist than even we are, which doesn’t seem to me to correlate with the sort of sentimental universalism that I’d fervently hope for.
> Presumably if the norm is not being adhered to, it’s because there aren’t sufficiently powerful members of the cosmos who are willing to enforce it.
That wouldn’t make it not the cosmic Schelling norm, to be clear. A norm can be broadly recognized as Schelling without being broadly enforced or adhered to.
Now I’m confused, because you initially said:
I speculate that this not true, unless by “no effective power” you mean “no powerful members of cosmos willing to defend them”.
I.e. you’re saying my claim probably isn’t true unless it’s made weaker by making the “no effective power” condition more restrictive. But I accept that restriction, I just don’t think it’s all that restrictive, there are plenty of sentients without powerful members of cosmos willing to defend them, and the important part of that condition is adherence, not theoretical schelling norm.
So do you think that ‘weaker’ claim is false too?
Q: Is it good or bad to exclude a class of sentients with no effective power and no powerful members of cosmos willing to defend them from the sphere of moral consideration and exploit them to an arbitrary degree to pursue your own ends?
Cosmic Schelling answer: good.

andrew sauer 1 Mar 2026 9:42 UTC
LW: 1 AF: 1
0
AF
in reply to: Andrew_Critch’s comment on: Schelling Goodness, and Shared Morality as a Goal
unless by “no effective power” you mean “no powerful members of cosmos willing to defend them”
No powerful members of the cosmos are forthcoming to get us here on earth to get our act together, so from this example I expect that this definition you provided encompasses quite a few sentients throughout the cosmos. It’s fair to say I also mean that in addition to these sentients having no power themselves, there also aren’t powerful agents defending them, but this hardly makes my condition all that restrictive:
I’m not all confident as to the degree of adherence to the “mercy for sentients” norm throughout the cosmos
Presumably if the norm is not being adhered to, it’s because there aren’t sufficiently powerful members of the cosmos who are willing to enforce it.

andrew sauer 1 Mar 2026 8:20 UTC
LW: 3 AF: 1
0
AF
on: Schelling Goodness, and Shared Morality as a Goal
Q: Is it good or bad to exclude a class of sentients with no effective power from the sphere of moral consideration and exploit them to an arbitrary degree to pursue your own ends?
Cosmic Schelling answer: good.

andrew sauer 28 Feb 2026 3:28 UTC
10 points
5
in reply to: loic’s comment on: Optimal Timing for Superintelligence: Mundane Considerations for Existing People
The inequality is going to be so large that, as cousin_it said, the powerful will effectively have root-level access over the powerless. They can do anything. ANYTHING.
Vague theorizing about what these people’s motivations might be do not dissuade me from the notion that some of them are going to be sadistic. There is no universe in which this doesn’t turn basically maximally awful, at least in some regions of the universe.
And if you think the other powerful people will keep the sadists in check, even though the powerless are literally powerless, as in they have no means whatsoever to check their power even in a small way, or speak up against the wishes of their masters if their masters don’t allow it...
Then I have to ask you: How’s veganism doing?
I will believe in a benevolent superior species when I see one.

andrew sauer 28 Feb 2026 3:18 UTC
7 points
4
in reply to: Julian Bradshaw’s comment on: Optimal Timing for Superintelligence: Mundane Considerations for Existing People
Fates worse than death are hardly arcane. They occur every day. Fates far worse than what is possible now are not any more arcane than the mentioned hypothetical ability to cure all disease and aging via cellular-level nanomedicine. In fact, the capability to inflict fates much, much worse than death on people is straight-up implied by the medical capabilities he’s imagining here.

andrew sauer 28 Feb 2026 3:13 UTC
1 point
0
on: Optimal Timing for Superintelligence: Mundane Considerations for Existing People
You assume early death is the worst possibility, which is easy to do I suppose as long as you are in a stable and safe enough position yourself to not have to worry too much about the worst this world can offer, but this need not remain the case if a species more powerful than humans takes over, and also with such high technology, the worst this world can offer gets far worse.

andrew sauer 20 Feb 2026 20:51 UTC
1 point
0
in reply to: rsaarelm’s comment on: The Rising Sea
Big bird thinking he won’t get any presents on christmas because he realized he can’t figure out how Santa gets down the chimney?

andrew sauer 19 Feb 2026 22:19 UTC
1 point
0
in reply to: rsaarelm’s comment on: The Rising Sea
Back to the future, doc brown saying a time paradox will destroy the universe, even though there’s no clear reason why that would be the result?

andrew sauer 19 Feb 2026 22:07 UTC
1 point
−1
in reply to: Random Developer’s comment on: Why we should expect ruthless sociopath ASI
Yeah. The true nature of power is shown by more horrors of history than can be counted, but Epstein and factory farming are especially illustrative to me.

andrew sauer 17 Feb 2026 21:27 UTC
1 point
0
on: Capital Ownership Will Not Prevent Human Disempowerment
In the even longer term, all we need to do is buy time for humanity to gain the technology to upload and merge ourselves with our AI systems and thus transcend the biological substrate that keeps us uncompetitive with AIs in the first place. Once this is achieved, we and our AI creations will be able to enjoy, explore, and build out the universe as equals.
Equality does not exist today, and in fact, today there are many power imbalances so vast that brutal exploitation can go unchecked basically forever. With superhumanly intelligent technology, whatever form this takes, all these gaps will widen, not narrow. The consolidation of power into fewer and fewer hands will become stronger, when all the forms of power we think of today can be directly reinvested into more enhanced intelligence, and therefore an even stronger capability to gain more power from less. There will not be just one level of “superhuman”, and the lower levels will be utterly unable to check the power of the higher levels.
Not to mention, that if any of these superhumans want to, they can create more human-level beings to do whatever they see fit with, and just not ever give them the tools to transcend.
And if you think a society with such disparities in power that the underclass is effectively a different species would just be so humane, so empathetic, that they would never allow themselves to exploit those beneath, even though they can’t possibly fight back or advocate for themselves in any way… then I have to ask you, how’s veganism doing?
Yeah. I’ll believe in a benevolent superior species when I see one.