I also used to find ancestor simulators something unlikely for future AIs to do just because they would be so beyond us with their transcendental concerns etc. but then I realized we humans would be quite interested in simulating how abiogenesis originally happened (despite being unfathomably more complex and advanced compared to the earliest forms of life) so seeing simulations of us as analogous to something like that started to look not so crazy.
Spective
Karma: 51
The ironic thing about humor as an IQ test is that IMO, it’s one that LessWrong and the Rationalist community is not doing so great on… It did occur to me one day when I realized what an outlier Scott Alexander is in this regard. As far as my personal funny bone is considered, anyway.
Yes, and “observing external reality” only leads into observing more qualia in your head rather than perceiving external reality as such, from which you make more or less indirect inferences about external reality. Some of these observations feel relatively direct, like observation of macro-level objects like tables and whatever, but even then you always only observe the them in some limited way, from some specific angle and what you see is something like your brain’s representation of the object rather than what it “objectively looks like” (there probably isn’t such a thing), these observations also never equal to the objective existence of the object which could always conceivably be a hallucination etc.
And then something like atoms or quantum fields isn’t perceived even this way at all, but rather postulated and inferred from complicated experiments to explain a wide range of comparatively more direct observations. Consciousness of other people is a bit like this—I would argue still more direct than the “observation” of atoms as pretty much all people can figure out other people probably have subjective experiences without doing any formal science. It’s just basic extrapolation from one’s own subjective experience, similarity to other humans and observing their behavior and postulating consciousness etc.
I don’t think there’s such an insurmountable ontological difference between observing the consciousness of other people (and remembering the consciousness of one’s past self) vs. observing the external reality in general. Both are something probabilistically inferred about something “out there” rather than something that’s right now right here and directly perceived. Perhaps you can never become quite as certain about the latter category than the phenomenal experience of your current observer moment, but that is hardly surprising that you know “the right here right now” better than the rest of the universe.
One of my life aspirations is to post something offensively stupid somewhere, forget about the post, randomly encounter it later, fail to recognize it as mine and write an outraged reply to it.
Not sure how much if I agree with 1). I increasingly find terms like physicalism/idealism/panpsychism worse than useless and conveying more vibes and tribal affiliations than truth propositions. I wouldn’t necessarily encourage people to identify with them more explicitly. Though other philosophical terms might be more useful—you mentioned closed individualism which (along with its opposite open/empty individualism) does seem to have more concrete implications on one’s attitude towards things like death, survival, mind uploading and various transhumanist technologies if we are lucky enough to ever have those problems. Potentially also has big implications on how you view AI x-risk as well.
IMO more useful distinction than “physicalism vs. non-physicalism” per se is whether you think new physics are needed to explain consciousness like Penrose does (another one is if you think that a system needs to make use of quantum effects for phenomenal binding). That seems like a claim with content, whereas the content of “consciousness is physical/non-physical” seems highly nebulous at best. This new substance that is required to explain consciousness is “non-physical”… uhh, then what?? Debates between physicalism and panpsychism (except the most naive strawmen thereof) in particular get so ridiculously abstract and vague about the intrinsic nature of an electron and whether it counts as “proto-experiential” or something that I doubt they really know what they are even talking about.
It’s definitely the funniest combo to me. Are there any known extreme examples?
Opus 4.6 and IIRC even Opus 4.5 could see the VR switch in the sense of recognizing it as a distinct looking tile, entertaining various hypotheses of what it could be but being overall bizarrely disinterested in it. While sometimes 4.6 would manage to get the boulder on it by sheer trial and error, occasionally it did it more intentionally by first noting the tile’s distinct look and verbalizing that it might be a switch. And IIRC at one point it had made a note what the switch looked like which helped it with consequent switch puzzles, until it destroyed this extremely precious note in overzealous memory cleanups.
Often 4.6 would assume it was an item, and given its default lack of interest in items tended to not pay much attention to it. Other times it dismissed it as just “floor decoration”. The big problem was that just the idea of trying to visually identify the correct tile first before starting to push the boulder haphazardly around wasn’t sufficiently obvious. It would sometimes verbalize this idea though, only to get distracted by ladders it saw or navigable tiles or whatnot and go back to bumping into walls in case of invisible doors or testing completely random boulder positions.
Neither the average child playing the game back in the 90′s nor Opus 4.6 knew in advance what the switches look like but both could see it. A big difference though is that at this stage of the game a child remembers what an item looks like and Opus 4.6 doesn’t. But even this doesn’t account for Opus 4.6′s default dismissal of the tile as “just decoration” when it confirmed it wasn’t an item. If you’re looking for a tile to push a boulder on, the one navigable one that looks different from the others should be a top candidate.
So what might’ve looked at first glance like a vision problem was actually more of a failure of reasoning, strategy etc. although to some extent also a problem of visual memory.
As for 4.7′s run, I think it’s worth noting that it got lucky by early on tossing the item that grants the ability DIG. DIG is notorious among viewers as something that’s been kryptonite for the different Claudes so far. It allows them to escape the cave and reset the puzzles when they get confused and desperate, but this also destroys so much of their progress which often a lot of time to redo due to their memory limitations.
What are some examples of the first? I certainly find myself increasingly thinking that there’s basically no limit to how dumb smart people can be in their dumb moments.
I don’t think you can just be conscious without being conscious of some things in particular. Subjective experience has to have content. What kind of experiences could a rock be having, considering what it’s physically doing? It’s probably not thinking “another day of being a rock”. Nor is it experiencing the sun shining on it because it doesn’t have any kind of visual processing system etc. Meanwhile prima facie at least, it’s considerably easier to imagine Claude having all kinds of human-like thoughts. On the condition that Claude has subjective experiences, but the contents thereof have to be computationally specified, it seems like we have some good reasons to have beliefs about what it might be experiencing.
Also, “a rock” is just one convenient way to draw boundaries for stuff in the universe and probably not very relevant for carving out experiential subjects. Even if in some sense “consciousness” is kinda everywhere, there seems to be an obvious sense in which some random group of people don’t form “one experiential macro-subject” but the individual members do (but for some groups of people acting really cohesively, it can arguably get a bit murky!) And this doesn’t seem that mystical but rather based on how information flows within the arrangement of objects we try to analyze as one subject and how unified it is.
wrt Blindsight response, assuming for some part of your brain you (your “main consciousness”, the one that can in addition to thinking and feeling also talk) don’t have direct access that there’s nothing like to be it seems a bit like assuming animals don’t have subjective experiences because you don’t have access to them yourself and the animals are very different from you. It’s almost like a trick of language, these parts are called “unconscious” because they are not in the subjective experience of the pointy-haired boss and then we equivocate this with a positive reason to think they lack subjective experience in and of themselves. This might be an irrelevant objection to what Watts is saying (since he seems to be talking about self-reflection etc.) however in that case it might not really answer Dawkins’ puzzlement either.
Also when used as more of an aesthetic judgement, AI sloppishness is quite orthogonal to the usual sense of “sloppy”—AI generated content has a certain polish and qualities that would be considered signs of skill and competence when produced by humans, esp. before they became easy to generate with AI and thus less valued. So in this context the problem isn’t that it’s “sloppy” as in looking careless, messy or full of mistakes but that it’s generic, bland, soulless etc. And a lot of creative, interesting and original (opposite of slop) content can be “sloppy” in the sense of being unpolished and imperfect.
you make an exact copy of you with all your memories, then 5 minutes later, the original you who got copied dies. Is this fine?
My first reaction to this is that it’s obviously not fine? I value living as myself, and I don’t get to do that if I die, and sure there is a copy of me living somewhere, but that is not the same? is it?
Clearly if there are two copies of someone, neither of them need to be fine with with dying even if the other one lives. After all there’s lots of potential future experiences lost if that happens. It’s semi-analogous to having one’s life extended first but then that extended life being taken away i.e. you have a heart that is expected to fail in ~10 years, somebody gives you an artificial heart that is supposed to last as long as the rest of your body, but then some bastard steals it and replaces it with your old crappy one—obviously you don’t need to be fine with that.
However, if we are presented with two alternative futures1) I get copied, 5 minutes later the original dies painlessly and instantly
2) I don’t get copied or killed.
Then I’ll happily bite the bullet and say I don’t think it matters much which of the two happens. Even if I take the perspective of the “original” who is about to die (who also knew before getting copied that these were the two only alternatives, maybe he needed to use the services of a teleport company whose policy is that the original must be deleted but there’s a 5 minute lag), to me it’s like I just had taken a pill that in 5 minutes will make me lose the memory of those 5 minutes.
Obvious facts nobody would disagree with when explicitly stated can still be underappreciated and not paid enough attention to, so they can still be worth spelling out for that reason. Although I must say when I get tired hearing of something obvious too many times, I get a pathological contrarian urge to argue against it.
Since OP contrasted neuralese with “legible CoT”, I’d like to add that while the “hard to train” may be true for neuralese, it doesn’t apply to o3-style Thinkish. Hopefully optimization pressures don’t favor that too much.
The kind of Vulcan you are imagining might have some kind of moral status, but is phenomenal consciousness really the crux here? Suppose there’s an otherwise identical Vulcan except no subjective experience—would there be no moral status in that case? Now, the obvious problem is of course that the coherence of this counterfactual is highly dubious to many. But my personal intuition is that to the extent I accept the counterfactual, making the Vulcan a zombie makes no moral difference. Whereas for beings with affective sentience, whether somebody goes around waving their magic wand and turning them into zombies or not intuitively seems morally significant.
There’s a lot of logical uncertainty here about the space of possible minds wrt phenomenal consciousness and affective experience, so there might be some kind of necessary connection between phenomenal consciousness (even when it’s totally non-affective) and features I associate with moral status for entities that lack affective sentience. But after I stopped assuming it’s always accompanied by valence, phenomenal consciousness in and of itself just prima facie doesn’t seem morally very important at all. Yes, destroying a planet of Vulcans to save a shrimp seems monstrous, but so does destroying a planet of entities that have sophisticated behavior and cognition without subjective experience.
It bothers me there’s no really established terminology for different views on personal identity. As in, whether you treat selves or persons essentially as “ontological primitives” or not. There’s a bunch of terms out there, but they are all sort of awkward for one reason or another and in any case I find there isn’t anything as widely established and easily understood as something like metaethical positions, i.e. moral realism vs anti-realism for example.
There isn’t really a snappy term to communicate something like “I don’t think the Star Trek teleporter kills me because I think my identity isn’t defined on any basic unchanging essence or specific atoms blah blah.” Except, if you’re a Buddhist. But if you’ve come to these views from totally different sources and know next to nothing about actual Buddhist traditions, calling yourself a Buddhist seems wrong. Using the term anattā isn’t too bad I suppose, but something without the Buddhist baggage would be more ideal.
I guess I felt the need to comment because I don’t even remember a time where that description would’ve been accurate for me—but wouldn’t be surprising if this also had something to do with memory formation so I can’t make too much of that. And notably, while I don’t recognize any point of my past kid self from that description, it’s indeed starting only around the age of 5 where I feel very confident about it. Curious if anyone here remembers relatively clearly something like “having experiences while lacking awareness of having a mind”.
IMO current LLMs probably have a small amount of what we usually call phenomenal consciousness or qualia. They have rich internal representations and can introspect and reflect on them. But neither is nearly as rich as in a human, particularly an adult human who’s learned a lot of introspection skills (including how to “play back” and interrogate contents of global workspace). Kids don’t even know they have minds, let alone what’s going on in there; figuring out how to figure that out is quite a learning process.
I’ll just note that this clashes heavily with my personal memories of being a kid. I usually use those as an intuition pump for the idea that phenomenal consciousness and intelligence are different, i.e. I wasn’t any “less conscious” as a kid AFAICT—to the contrary, if anything I remember having more intense and richer experiences than now as a comparatively jaded and emotionally blunted adult. There’s two things going on though—introspection and intensity of experience, but I also remember being very introspective and “kids don’t even know they have minds” in particular sounds very weird to me.
Funnily this case arguably resembles more the “strawman” version of the paperclip maximizer. In the “actual” paperclip maximizer thought experiment, paperclips were supposed to be just some unpredictable, seemingly random emergent goal rather than following verbal instructions but not in the way you wanted. According to the historical note here.