cubefox

Karma: 1,884

cubefox May 24, 2025, 3:55 PM
−2 points
−4
in reply to: xpym’s comment on: The stakes of AI moral status

what should we do?

Figure out where we’re confused

Congratulations, you just reinvented philosophy. :)

cubefox May 24, 2025, 7:09 AM
4 points
0
in reply to: Said Achmiz’s comment on: The stakes of AI moral status

No, babies are very different from adults in this regard, inasmuch as adults can tell us that they are in pain, can describe the pain, etc.

This doesn’t look like a big difference to me. Moreover, adults may also be unable to speak due to various illnesses or disabilities.

For very dissimilar entities, like rocks, the probability of being conscious would fall back to some kind of prior, though I don’t know how such a prior could be justified. (Purely intuitively it seems clear that rocks being conscious is highly unlikely, but it isn’t obvious why.)

… really? You can’t think of any reasons for this belief? Just pure intuition, that’s all you’ve got to go on? Are you seriously making this claim?

Yes. At least not from the top of my head. Note that this prior is supposed to not incorporate the information that you are conscious yourself.

cubefox May 24, 2025, 4:58 AM
4 points
0
on: How load-bearing is KL divergence from a known-good base model in modern RL?

Should we expect the trend of RL models being less like outcome pumps and more like agents which execute unsurprising actions to hold?

Very recently, with RLVR (reinforcement learning from verifiable rewards) the trend seems to have reversed. See here or here.

cubefox May 23, 2025, 2:15 PM
8 points
3
in reply to: Said Achmiz’s comment on: The stakes of AI moral status
These citations would only include trivial data we know about anyways. E.g. “If you injure a baby, it cries (which seems pretty similar to what I do when in pain)”. Babies are hardly different from adults in this regard. So it makes little sense to demand “evidence” for babies being able to feel pain, but not for (other) adults. I think in all these cases I can infer other minds with an inference from analogy, i.e. from similarity to myself in known properties (behavior, brain) to unknown properties (consciousness). For very dissimilar entities, like rocks, the probability of being conscious would fall back to some kind of prior, though I don’t know how such a prior could be justified. (Purely intuitively it seems clear that rocks being conscious is highly unlikely, but it isn’t obvious why.)

cubefox May 22, 2025, 2:55 PM
10 points
3
in reply to: Said Achmiz’s comment on: The stakes of AI moral status
There is no “citation” that anyone but myself feels pain. It’s the “problem of other minds”. After all, anyone could be a p-zombie, not just babies, animals, AIs...

cubefox May 22, 2025, 11:31 AM
3 points
0
in reply to: E.G. Blee-Goldman’s comment on: E.G. Blee-Goldman’s Shortform
Is this evidence for the natural abstraction hypothesis @johnswentworth?

cubefox May 22, 2025, 12:59 AM
5 points
0
in reply to: E.G. Blee-Goldman’s comment on: E.G. Blee-Goldman’s Shortform
Thanks for this reference. It arguably means aliens don’t have alien ontologies. Previous related discussion.

cubefox May 22, 2025, 12:54 AM
3 points
0
in reply to: TAG’s comment on: Warty’s Shortform
The article is not about alignment (that’s a different article), it’s about a normative moral theory.

cubefox May 22, 2025, 12:11 AM
4 points
1
in reply to: Warty’s comment on: Warty’s Shortform
That’s a frequent misconception. In fact, Eliezer Yudkowsky is a moral realist.

cubefox May 21, 2025, 3:21 PM
11 points
2
in reply to: whestler’s comment on: Matthew Khoriaty’s Shortform
Yeah. It is probably even more important for the cover to look serious and “academically respectable” than for it to look maximally appealing to a broad audience. It shouldn’t give the impression of a science fiction novel or a sensationalist crackpot theory. An even more negative example of this kind (in my opinion) is the American cover of The Beginning of Infinity by David Deutsch.

cubefox May 21, 2025, 2:51 PM
2 points
0
in reply to: Matthew Barnett’s comment on: Winning the power to lose
Note that individual value differences (like personal differences in preferences/desires) do not imply a difference in moral priority. This is because moral priority, at least judging from a broadly utilitarian analysis of the term, derives from some kind of aggregate of preferences, not from an individual preference. Questions about moral priority can be reduced to the empirical question of what the individual preferences are, and/or to the conceptual question of what this ethical aggregation method is. People can come (or fail to come) to an agreement on both irrespective of what their preferences are.

cubefox May 21, 2025, 8:53 AM
7 points
4
in reply to: Vale’s comment on: Matthew Khoriaty’s Shortform
Not sure about the italics, but I like showing Earth this way from space. It drives home a sense of scale.

cubefox May 21, 2025, 8:34 AM
13 points
10
in reply to: ryan_greenblatt’s comment on: peterbarnett’s Shortform
Note, the video doesn’t show up for me.

cubefox May 20, 2025, 2:39 PM
2 points
0
in reply to: rustyfe’s comment on: rustyfe’s Shortform
X because Y implies X and Y, though not the other way round.

cubefox May 20, 2025, 10:45 AM
3 points
0
on: Thinking Insect Suffering Is The Biggest Deal In The World Is Surprisingly Intuitive
This is an overall well-reasoned post. I don’t want the conclusion to be true, but that is no reason to downvote.

cubefox May 19, 2025, 5:38 PM
4 points
0
in reply to: Said Achmiz’s comment on: What Does It Mean to “Write Like You Talk”?
I noticed this years ago when the variations of the show Big Brother were on TV in various countries. The show consists of compilations of real people spontaneously talking to each other throughout the day. The difference between this and the scripted conversations we saw on TV before Big Brother is huge. Real people apparently hardly talk in complete sentences, which is why scripted conversations are immediately recognizable as being fake. It’s also strange that this is hardly noticeable in real life when you are actually having conversations.

cubefox May 19, 2025, 12:50 PM
2 points
0
in reply to: Vladimir_Nesov’s comment on: a confusion about preference orderings
I think one issue with the “person+time” context is that we may assume that once I know the time, I must know whether it is Friday or not. A more accurate assessment would be to say that an indexical proposition corresponds to a set of possible worlds together with a person moment, i.e. a complete mental state. The person moment replaces the “person + time” context. This makes it clear that “It’s Friday” is true in some possible worlds and false in others, depending on whether my person moment (my current mental state, including all the evidence I have from perception etc) is spatio-temporally located at a Friday in that possible world. This also makes intuitive sense, since I know my current mental state but that alone is not necessarily sufficient to determine the time of week, and I could be mistaken about whether it’s Friday or not.

A different case is “I am here now” or the classic “I exist”. Which would be true for any person moment and any possible world where that person moment exists. These are “synthetic a priori” propositions. Their truth can be ascertained from introspection alone (“a priori”), but they are “synthetic” rather than “analytic”, since they aren’t true in every possible world, i.e. in worlds were the person moment doesn’t exist. At least “I exist” is false at worlds where the associated person moment doesn’t exist, and arguably also “I am here now”.

Yet another variation would be “I’m hungry”, “I have a headache”, “I have the visual impression of a rose”, “I’m thinking about X”. These only state something about aspects of an internal state, so their truth value only depends on the person moment, not on what the world is like apart from it. So a proposition of this sort is either true in all possible worlds where that person moment exists, or false in all possible worlds where that person moment exists (depending on whether the sensation of hungriness etc is part of the person moment or not). Though I’m not sure which truth value they should be assigned in possible worlds where the person moment doesn’t exist. If “I’m thinking of a rose” is false when I don’t exist, is “I’m not thinking of a rose” also false when I don’t exist? Both presuppose that I exist. To avoid contradictions, this would apparently require a three-valued logic, with a third truth value for propositions like that in case the associated person moment doesn’t exist.

cubefox May 19, 2025, 8:46 AM
2 points
0
in reply to: Veedrac’s comment on: leogao’s Shortform
And what would this look like? Can you reframe the original argument accordingly?

cubefox May 19, 2025, 8:35 AM
2 points
0
in reply to: Veedrac’s comment on: leogao’s Shortform
I meant leogao’s argument above.

cubefox May 19, 2025, 8:14 AM
2 points
0
in reply to: Veedrac’s comment on: leogao’s Shortform
It seems the updating rule doesn’t tell you anything about the original argument even when you view information about reference classes as evidence rather than as a method of assigning prior probabilities to hypotheses. Or does it? Can you rephrase the argument in a proper Bayesian way such that it becomes clearer? Note that how strongly some evidence confirms or disconfirms a hypothesis also depends on a prior.