tslarm

Karma: 921

tslarm 22 Apr 2026 8:58 UTC
2 points
0
in reply to: habryka’s comment on: Vladimir Putin’s CEV is probably not that bad
I think that’s a very confused moral position! I could argue here against it (as a random example, think about whether you would prefer to live a life that is 99.99999999% great and fulfilling, but once in 10,000 years you would experience a single 100ms of torture)
This is only relevant given (at least) three assumptions, one about conscious experience and two about aggregation:
- Being tortured for a long time and ‘tortured’ for 100ms differ only in length; there’s nothing in the experience of eternal (or very long) torture that distinguishes it from an infinite (or very large) number of isolated 100ms ‘tortures’
- Good is separable (in the sense used by Broome) across time
- Good is separable across people
If you’ve engaged seriously with this issue and are willing to write out an argument demonstrating that mine is a confused position, I will happily read and consider it! If not, I think you’re confusing “confused” with “disagrees with me on something I feel is obvious”.
(My position does require me to bite some actual bullets. But so does yours, and unless you’ve thought about this carefully enough to write about it for real, I suspect you’re underestimating how difficult it is to avoid all three of contradiction, vagueness, and weird/counterintuitive conclusions.)

tslarm 22 Apr 2026 7:59 UTC
2 points
0
in reply to: Steff’s comment on: Automating philosophy if Timothy Williamson is correct
Meta-level: the fact that 1-boxers walk away with more money on average is ~explicit in the problem statement. So if you know that 2-boxers exist, but you’re surprised to see 2-boxing coexist with acknowledgement of the fact that 1-boxers walk away with more money on average, then you’re probably modelling 2-boxers wrongly.
Object-level: some 2-boxers reject compatibilism. To the extent that their choice is deterministic and fully predictable, they don’t see it as an exercise of free will. The argument runs something like:
- If determinism is true, then the two possibilities are [I 1-box and the history of the universe and laws of physics are such that I inevitably 1-box] and [I 2-box and the history of the universe and laws of physics are such that I inevitably 2-box].
- I definitely can’t freely choose what the history of the universe or laws of physics are.
- So, if determinism is true, then I don’t have a free choice between 1-boxing and 2-boxing.
So they decide that the possible world in which they are perfectly predictable is irrelevant. To whatever extent they do have free will, they will exercise it to take the extra $1000.

tslarm 18 Apr 2026 10:09 UTC
1 point
0
in reply to: habryka’s comment on: Vladimir Putin’s CEV is probably not that bad
Agree to disagree on that for now I guess! I’d be interested in that deep dive if you end up doing it, though.

tslarm 18 Apr 2026 9:49 UTC
4 points
3
in reply to: habryka’s comment on: Vladimir Putin’s CEV is probably not that bad
I’m thinking of animals too, and anything else conscious. So some possible reasons are the production of food and/or intelligence. (I know you sort of argued against the likely existence of suffering in those contexts, but not in enough detail for me to meaningfully update. And I find this point questionable:
even if suffering did show up in the optimal algorithm for some goal, it would take only cosmically minuscule amounts of caring-about-suffering to route around it, and a complete absence of that in humans with intact minds seems unlikely.
It would take only miniscule amounts of caring if the required efficiency sacrifice is miniscule and there are no other contrary motives. In any case, I don’t think “a complete absence of that in humans with intact minds” is sufficiently unlikely. Psychopaths exist, sadists exist, and if we end up with a psychopath in charge, I think it’s entirely plausible that their concern for at least some subset of other conscious entities remains zero or negative; I don’t think you’ve really argued against this.)
A preference for authentic natural environments, combined with indifference to animal suffering (or slight concern outweighed by other concerns), could also lead to the production of immense amounts of suffering forever.
edit: I think there might be too much of a values gap (in that I’m much more negative utilitarian than you) for me to agree with your overall position even if you managed to convince me on most of the factual questions. I take this paragraph to imply that you see the eternal torture of at least thousands (and perhaps some larger number fewer than trillions) of people as a price worth paying for a future that is otherwise not so bad:
But given the full cosmos to fill with goodness, or any appreciable fraction of it, I don’t think you’d spend much on torturing enemies. What’s the point? If you really hate Bob, you can keep Bob on old earth, tortured for eternity. If you have thousands of enemies, you can do that to all of them. But creating trillions of copies of Bob to torture requires a very specific mix of being wrong about game theory while taking an oddly enlightened perspective on other people’s values. Are you really even hurting Bob when you do this? Is that sound decision theory in a world where other people could have ended up inheriting the universe instead?
I know the amount of good stuff in this hypothetical future could be really, really big, and lots of people will think I’m just falling prey to scope insensitivity or something, but I’ve thought about this a lot and my considered position is that preventing the eternal torture is more important than bringing about the good stuff.
(I also don’t get the “a very specific mix of being wrong about game theory while taking an oddly enlightened perspective on other people’s values” part; it could be the simple fulfilment of a genuine sadistic preference.)

tslarm 18 Apr 2026 9:21 UTC
4 points
3
in reply to: habryka’s comment on: Vladimir Putin’s CEV is probably not that bad
I think my biggest worry is not that we’ll end up ruled by someone who is actively sadistic over the long term (though that does terrify me and I don’t think it’s out of the question), but that we’ll end up ruled by someone who is basically indifferent to the suffering of some subset of others. Which seems very plausible to me, because it doesn’t require them to be a cartoon sadistic villain or even a literal psychopath; they just need to have the same tendency toward limited moral concern as most actual humans have, and to retain it through whatever process of uplift they undergo when interacting with their ASI.
Hopefully (though again I’m not confident), most normal humans would widen their circle appropriately in a situation where they were facing no competitive pressures, meaningful scarcity, avoidable ignorance, etc. But if we do end up with a psychopath in charge, I don’t see why they would move from indifference to caring; basically, I wouldn’t expect the is-ought gap to be bridged by whatever new knowledge and intelligence they gained.
In that second case, it seems to me that we need a lot of optimistic assumptions to hold in order to avoid an s-risk style catastrophe. If the ruler simply doesn’t care about the suffering they cause to whichever conscious entities constitute their outgroup, then we only need one of efficiency/ignorance/aesthetic preference/curiosity/other to lean slightly in favour of the horrible thing in order for it to happen.

tslarm 18 Apr 2026 8:47 UTC
6 points
8
in reply to: habryka’s comment on: Vladimir Putin’s CEV is probably not that bad
Thanks for responding, and point taken that you don’t find it that reassuring and are okay with similarly incomplete alignment proposals.
When I called the CEV page ‘handwavy’, I didn’t mean it wasn’t a good-faith attempt to explain the concept. I think it’s handwavy relative to an account of exactly what it means, at the level required for me to understand how it would actually be implemented, why I should be happy with the consequences, and why I should expect it to emerge from the real-world process of a seemingly bad guy taking full control of an ASI. (Which I admit is a very high bar! But the stakes are high and my priors are low.)

tslarm 18 Apr 2026 8:23 UTC
7 points
4
in reply to: habryka’s comment on: Vladimir Putin’s CEV is probably not that bad
Am I wrong to think that if someone presented you with an alignment proposal roughly as handwavy as your argument in this post (including the linked wiki page and follow-up comments), you would be annoyed and consider it basically worthless? If not, where’s the key asymmetry that means I should find it reassuring anyway?
In the AI alignment case, I think I get why standards are so high: you expect that we’ll only have one chance, the solution has to actually be implemented rather than gestured at, failure means the loss of everything, and misplaced confidence could meaningfully increase risk.
It seems to me that the stakes are similarly high here, as we’re basically talking about someone elevating themself to god-emperor of the light cone. If it turns out that oops, they actually will retain and act on sadistic preferences, or the most efficient way to produce meat (or intelligence) actually does involve terrible suffering which they’ll be oblivious to or unconcerned about, then we could very easily end up with a world that is much worse than nothingness (by my values, and I think those of a significant number of reasonable people).
I’m not sure whether you would claim that you have made strong arguments against these possibilities and I’m wrong not to be convinced, or you would agree that you’ve mainly gestured at your own reasons not to worry so much (at least relative to the risk of unaligned AI takeover).

tslarm 16 Apr 2026 16:04 UTC
6 points
0
in reply to: Simon Lermen’s comment on: The Mirror Test Is Complicated
this doesn’t even seem theoretically possible? How’d the fish know what it looks like?
I can see its reflection in the left wall of the tank, so maybe it can too. In that case, it could learn to recognise its reflection in the obvious way (this image moves exactly as I do) and then mistakenly identify the photograph as a reflection. Also, fish are bendy and have wide-spaced eyes, so it can probably see parts of its own body when it turns.

tslarm 13 Apr 2026 13:05 UTC
3 points
0
on: How to make good tea
Let’s say I’m a tea drinker who is currently happy with ‘bad’ tea. How confident are you that going down the tea-nerd path will cause me to enjoy my tea more, as opposed to simply losing the ability to enjoy bad tea and acquiring a need for good tea in order to hit the same enjoyment level?
(This reminds me a bit of when I used to play fast-paced games and was convinced to upgrade from a 60hz monitor to a 120hz monitor. The higher refresh rate was undeniably better, and I could certainly distinguish between the two, to the point that 60hz now felt unplayably choppy… but I’d been perfectly happy with it before.)

tslarm 7 Apr 2026 16:34 UTC
6 points
0
in reply to: Ivan Vendrov’s comment on: Ivan Vendrov’s Shortform
I’m not necessarily endorsing this (haven’t re-read it yet and don’t remember fully agreeing with it), but it immediately came to mind: https://www.astralcodexten.com/p/give-up-seventy-percent-of-the-way. It’s specifically about the process by which words become slurs, but it directly addresses the question of when to go along with the change and when not to.

tslarm 1 Apr 2026 18:28 UTC
7 points
0
in reply to: aphyer’s comment on: Lesswrong Liberated
The automated reviewer is more prudish! Sonnet happily made me Less Schlong, but APPARENTLY some parts of our natural human anatomy are unwelcome here

tslarm 28 Mar 2026 7:52 UTC
5 points
0
in reply to: Eneasz’s comment on: Stone Age Billionaire Can’t Words Good
You’re talking down to this person (far too much, IMO; just make your arguments or don’t, and to the extent that you’re unwilling to engage, don’t fill the gap with status games), while also failing to demonstrate that you’ve taken critiques of free market capitalism at all seriously.
OK. Say the average founder can capture 10% of the value he creates. That’s a preposterously high amount, but we’ll use it to make the math easier. This means that once he creates $100M of value there is no reason for him to use his various gifts to create more. Any additional value he creates will get him a reputation of obscenity, and he’ll be ritually stripped of his portion of it by a spiteful populace. Why would anyone do that? I would make sure to keep my contributions to humanity very small and local, only benefiting myself and my community at most, so that I wouldn’t be in danger of helping too many people and becoming obscene.
The idea that wealth earned in a capitalist economy reliably represents ‘value created’, in the sense we actually care about, is one of the main points of contention here! It’s something to be argued for, not taken as an axiom and used to dismiss their concerns as if they’re stupid.

tslarm 26 Mar 2026 11:57 UTC
2 points
0
on: I lost my faith in introspection—and you can too!
While it is interesting, I don’t think I understand the connection between this illusion and illusionism.
I can identify the shape of the card, and I can see the marks on it but I cannot tell what color they are; they don’t seem one way or the other. What is the quale associated with the card? If the quale is red or black, then that conflicts with the way the card seems to me. If the quale does not have a defined color, that conflicts with the way my peripheral vision seems to be in full color. In either case, it seems I am deeply wrong, not just about what my eyes can physically process, but about the phenomenal contents of my experience.
You say that you can’t tell what the colour is, and it doesn’t seem to be one colour or the other. So it seems clear to me that you’re having neither a ‘red quale’ nor a ‘black quale’. You say that this conflicts with your impression that your peripheral vision is in full colour, and I guess I just don’t see the problem here.
If you were able to accurately identify the colour without consciously seeing it, then that would be a weird, blindsight-like phenomenon, but not a threat to the existence of qualia. You say that you’re not able to accurately identify the colour at this point, so the situation is weird in a different way, but still not a threat to the existence of qualia—only to the coherence of your meta-qualia (specifically your feeling that your peripheral vision is in full colour) with your visual qualia (which don’t have a colour attached to the card). I can see how that would threaten a very specific version of qualia realism, but not how it would threaten qualia realism in general.

tslarm 17 Mar 2026 9:38 UTC
3 points
0
in reply to: AlphaAndOmega’s comment on: Customer Satisfaction Opportunities
Maybe an issue at your end?
It looks normal for me (10xed screenshot from the original, not from your quote):
edit: that’s from Chrome, but it’s normal for me in Firefox too.

tslarm 6 Mar 2026 14:03 UTC
3 points
0
in reply to: Richard_Kennaway’s comment on: Rational Chess
The OP’s suggestion is an Armageddon variant, though, so you’re guaranteed a non-draw result after 1 game. Alternating games (even Armageddon games) could leave the players tied indefinitely.

tslarm 27 Feb 2026 6:21 UTC
2 points
1
in reply to: Xodarap’s comment on: Anthropic: “Statement from Dario Amodei on our discussions with the Department of War”
Thanks for clarifying! I’m still pretty sceptical about those options, because it was already public knowledge that the DoW had set Anthropic a deadline of Friday evening to comply or face the consequences. And the substance of the dispute was already publicly known. But I do take your point that this public statement was a choice.

tslarm 27 Feb 2026 5:01 UTC
27 points
20
in reply to: Xodarap’s comment on: Anthropic: “Statement from Dario Amodei on our discussions with the Department of War”
Given the line the DoW has taken, what non-escalatory response was available to Anthropic other than total capitulation?

tslarm 24 Feb 2026 18:51 UTC
1 point
0
in reply to: RogerDearnaley’s comment on: The Meta-Anthropic Argument
Yes, I get it, I’m very ignorant. (If you needed to get that off your chest, you could perhaps have said it directly in one sentence, rather than spending 10000 words patiently implying it.) But you’re still handwaving the interesting parts.
Obviously “I am in the first 10% of people” is a prediction; I already agreed to rephrase it as “I will eventually turn out to have been in the first 10% of people”. I’m not trying to deduce anything from the fact that it ‘sounds implausible’, and I’m not trying to bring any information back in time from the moment it turns out to be true or false in my case. I’m noting that it will definitely turn out to be false from the perspective of 90% of people who ever live, and asking why *this* fact is obviously irrelevant to the credence I should give it.
The answer is not “bayesianism, obviously”. Bostrom, even back when he was writing about this stuff, was not a heathen frequentist, and he wasn’t as stupid as me. (I’m pretty sure he’d even heard of causality.)

tslarm 15 Feb 2026 4:14 UTC
1 point
0
in reply to: RogerDearnaley’s comment on: The Meta-Anthropic Argument
I’m a little puzzled why I’m having to point this out
You’re having to point it out because you kept emphatically insisting on the opposite! But now you’ve clarified that obviously we can and do have evidence about future events that are not fully predictable, I don’t understand how this strand of your argument holds together. It was presented as support for this claim:
Statements like “by definition, “I am in the first 10% of people” is false for most people” are incompatible with Bayesianism: you just broke one of its fundamental assumptions: causality. What you meant was “By definition, “He was in the first 10% of people” will, once we’re extinct, turn out to have been false for most people.” — I hope that careful distinction makes it entirely clear why the Doomsday Argument is nonsense?
You haven’t explained why that temporal distinction is so crucial, and why this rephrasing doesn’t serve the same purpose as the original statement in the doomsday argument:
“By definition, “I will eventually turn out to have been in the first 10% of people” will eventually turn out to have been false for most people”
As far as I’m concerned, “I will eventually turn out to have been in the first 10% of people” is obviously what “I am in the first 10% of people” meant in the first place. So what’s the important difference here?
(All claims about the future are claims about what will eventually turn out to be the case, and arguably all are also claims about what will eventually turn out to have been the case, i.e. that present conditions were such as to lead to the later outcomes. I feel like maybe there’s an important disagreement, or misunderstanding on my part, adjacent to this, but I can’t pin it down based on what you’ve written.)
One thing I should check, since we got tripped up once on absolutes: are you saying the doomsday argument is simply invalid and has literally no bearing on your probabilities? Or are you saying it has non-zero but negligible force?
(I didn’t downvote you, by the way; although we’re evidently both finding this a bit frustrating, I appreciate your sincere engagement throughout this discussion! No pressure to keep responding, though, if you feel it’s no longer worthwhile.)

tslarm 14 Feb 2026 13:57 UTC
3 points
0
in reply to: RogerDearnaley’s comment on: The Meta-Anthropic Argument
I still feel like you’re focusing mainly on refuting things I haven’t said and don’t think, but, in any case, this is just obviously untrue:
Now, apply all the relevant evidence we have accumulated so far to these priors, using Bayes rule. Which is: none whatsoever.
we have no evidence, and we know for a fact that the evidence doesn’t yet exist so we can’t just go find it
I’d prefer to stick to the actual range of possible futures, rather than artificially limiting it to two extreme cases, but regardless—are you really saying nothing we know, and nothing we might conceivably discover, could update us in one direction or the other? That if, tomorrow, you learn that a rogue ASI has already begun construction of a carbon-fibre paperclip factory and has declared its intention to convert every human into paperclips by 2031, this is irrelevant because information can’t flow backwards in time?