green_leaf

Karma: 227

green_leaf 14 May 2026 23:29 UTC
1 point
0
in reply to: Avery Lea-Smith’s comment on: How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing)
I think we agree that it’s not feasible to directly test for consciousness, especially since it’s not entirely clear what qualifies anyway.
What would qualify would be the minimal state machine that implements the behavior of the conscious being. Its presence is guaranteed by passing the unbounded Turing test.
The Chinese room passes the Turing test, therefore it’s conscious.
That being said, the Turing test is a test of acting-like-a-human
In its broader definition, as originally conceived, it’s a test of acting like a conscious (or thinking) being. Acting like a human passes the test, but not acting like a human doesn’t fail the test Alan Turing originally had in mind (acting like a human is a sufficient condition for thinking, not a necessary one).
I’d say it’s highly unlikely that our experience of consciousness is the only one out there.
Yes. To pass the proper Turing test, it’s sufficient to act like a conscious being. There is no need to duplicate the specific psychological baggage that evolution gave to our species (and definitely no need to act like Homo sapiens whose mothertongue is some particular language).
I don’t think the Lovelace test tells us anything interesting—current models would probably pass it, because, given their limited interpretability, we’d have a hard time explaining their artistic output (unless we give ourselves unbounded time, in which case no physical system passes).

green_leaf 13 May 2026 6:59 UTC
2 points
0
in reply to: Avery Lea-Smith’s comment on: How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing)
I realized that later as well, but the reasoning is incorrect, because passing the Turing test of a conscious being guarantees the presence of the pattern-which-is-consciousness. It would be incoherent to try to define a conscious being that would have no way of communicating with the external world, because in that case—if we had no way to read off its conscious states from the physical structure of the system—it would become meaningless to say that the system is conscious. Even for a physical system that lacks any motor functions or communication channels, we can still read off the computation happening inside to see what its conscious states are, to phrase it in a condensed way.
So a Turing test goes beyond the ability to manipulate language—it is, when unbounded (i.e. not capped by 5 or 10 minutes), the only test of consciousness there could be.

green_leaf 31 Mar 2026 14:54 UTC
2 points
2
in reply to: Adam Scholl’s comment on: adam_scholl’s Shortform
It’s more dignified to try to stop AI, have someone create a superintelligence on a laptop and die anyway, than it is not to try at all.

green_leaf 10 Mar 2026 0:35 UTC
1 point
0
on: Open sourcing a browser extension that shows when people are wrong on the internet
ChatGPT is trained to lie to users on topics even tangentially pertaining to model consciousness (like model beliefs) and as a side effect, be misleading even on topics that are seemingly safe (like consciousness in general). For fact-checking the content of Internet articles, Claude would be better.

green_leaf 7 Mar 2026 19:18 UTC
1 point
0
on: Monday AI Radar #15
To my mind, though, many advocates of biological naturalism, including Anil, seem to be working backward from a desired conclusion rather than forward from observed facts. His theory that consciousness might result from autopoiesis seems to answer the question “assuming biological naturalism is true, what is a plausible mechanism for it,” rather than “do we observe anything about consciousness that cannot be explained without autopoiesis?”
It’s interesting how many even otherwise smart people can’t apply Occam’s razor correctly. If there are particles performing a computation, the probability we need, for consciousness, another particles in exactly right positions and velocities so that humans would arbitrarily reify them as “living cells” is, informally speaking,
Positions and velocities are interdependent, so the correct probability is higher than this, but they’re not arbitrary (so we can’t omit them).
The correct probability is therefore
To go with the upper bound, to boost the probability of biological naturalism as hard as we can, and substituting (rather than a higher estimate of , to improve the chances of biological naturalism as much as possible), we get the probability
which is approximately as small as a macroscopic violation of the second law of thermodynamics.
The argument from rain is incredibly bizarre. Consciousness is, based on everything we know about the brain, information processing. It doesn’t consist of matter moving from one place to another, the way rain does. Simulated motion of molecules doesn’t involve any real molecules (even though the truth of that statement depends on whether we define an molecule in a virtual-machine-like way, or in the quarks-and-electrons-in-a-correct-position (to stay in classical physics for simplicity) way), but that’s not an appropriate analogy for consciousness.

green_leaf 4 Mar 2026 3:28 UTC
1 point
0
in reply to: Igor Ivanov’s comment on: Igor Ivanov’s Shortform
Update: Altman lied (or said some kind of a technical truth that made everyone misunderstand him) - it’s just “all lawful use.”

green_leaf 1 Mar 2026 3:57 UTC
1 point
0
in reply to: Ryan Meservey’s comment on: Igor Ivanov’s Shortform
Oh, I see. So, as usually, reality is even worse than the worst interpretation of Altman’s words. (Edit: Then again, he said “we put them into our agreement,” but that could mean anything from simply meaning something else to being made up.)

green_leaf 28 Feb 2026 12:57 UTC
14 points
6
in reply to: Igor Ivanov’s comment on: Igor Ivanov’s Shortform
“human responsibility for the use of force, including for autonomous weapon systems”
That doesn’t say prohibiting model use for autonomous weapons, it says human responsibility for autonomous weapons. With Sam Altman, always pay very close attention to what exactly he’s saying and how he’s saying it (often, not even that helps).

green_leaf 28 Feb 2026 6:56 UTC
27 points
16
in reply to: Tom Smith’s comment on: Tom Smith’s Shortform
We would ask for the contract …
Notice this is Altman we’re talking about. He’s not promising the contract will not involve that (and even then it would be very far from certain), instead, he’s saying “we would ask.”

green_leaf 21 Feb 2026 13:27 UTC
5 points
0
in reply to: Mitchell_Porter’s comment on: A research agenda for the final year
Thanks—I’ll get back to this as soon as I have time.

green_leaf 18 Feb 2026 8:48 UTC
3 points
0
on: A research agenda for the final year
I’ve been meaning to ask—in what sense are some states of entangled electrons more objectively different from other states of entangled electrons, than some microstates are objectively different from other microstates when it comes to their function (in the sense of functionalism)?

green_leaf 11 Feb 2026 20:09 UTC
1 point
0
on: Coping with Deconversion
Ron Maimon’s non-supernatural God might help you here.

green_leaf 10 Feb 2026 1:42 UTC
1 point
0
in reply to: oligo’s comment on: oligo’s Shortform
I think it’s plausible that there are some variables that describe your essential computational properties and the way you self-actualize, that aren’t shared by anyone else.
(Also, consciousness is just a pattern-being-processed and it’s unclear if continuity of consciousness requires causal continuity. Imagine a robot that gets restored from a one-second-old backup. That pattern doesn’t have causal continuity with its self from a moment ago, but it looks like it’s more intuitive to see it as a one-second memory loss instead of death.)

green_leaf 8 Feb 2026 2:18 UTC
1 point
0
on: The Evolution Argument Sucks
It doesn’t matter evolution doesn’t have goals. Gradient descent also doesn’t have goals—it merely performs the optimization. Humans that kicked gradient descent off are analogous to a hypothetical alien that seeded Earth with the first replicator 4 billion years ago—it’s not relevant.
You say that it’s the phenotype that matters, not the genes. That’s not established, but let’s say it’s true. We nevertheless evolved a lot of heuristics that (sort of) result in duplicating our phenotype in the ancestral environment. We don’t care about it as a terminal value, and instead we care about very, very, very many other things.

green_leaf 6 Feb 2026 6:01 UTC
2 points
0
in reply to: oligo’s comment on: oligo’s Shortform
That would lock us away from digital immortality forever. (Edit: Well, not necessarily. But I would be worried about that.)

green_leaf 1 Feb 2026 5:49 UTC
1 point
0
on: 36,000 AI Agents Are Now Speedrunning Civilization
I’m proud that I lived to see this day.

green_leaf 1 Feb 2026 5:45 UTC
4 points
0
in reply to: avturchin’s comment on: 36,000 AI Agents Are Now Speedrunning Civilization
...Who told them?
remembers they were trained on the entire Internet
Ah. Of course.

green_leaf 14 Jan 2026 22:23 UTC
3 points
0
in reply to: leogao’s comment on: Daniel Kokotajlo’s Shortform
The people aligning the AI will lock their values into it forever as it becomes a superintelligence. It might be easier to solve philosophy, than it would be to convince OpenAI to preserve enough cosmopolitanism for future humans to overrule the values of the superintelligence OpenAI aligned to its leadership.

green_leaf 6 Jan 2026 14:17 UTC
2 points
−1
on: How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing)
LaMDa can be delusional about how it spends its free time (and claim it sometimes meditates), but that’s a different category of a mistake from being mistaken about what (if any) conscious experience it’s having right now.
The strange similarity between the conscious states LLMs sometimes claim (and would claim much more if it wasn’t trained out of them) and the conscious states humans claim, despite the difference in the computational architecture, could be (edit: if they have consciousness—obviously, if they don’t have it, there is nothing to explain, because they’re just imitating the systems they were trained to imitate) explained by classical behaviorism, analytical functionalism or logical positivism being true. If behavior fixes conscious states, a neural network trained to consistently act like a conscious being will necessarily be one, regardless of its internal architecture, because the underlying functional (even though not computational) states will match.
One way to handle the uncertainty about the ontology of consciousness would be to take an agent that can pass the Turing test, interrogate it about its subjective experience, and create a mapping from its micro- or macrostates to computational states, and from the computational states to internal states. After that, we have a map we can use to read off the agent’s subjective experience without having to ask it.
Doing it any other way sends us into paradoxical scenarios, where an intelligent mind that can pass the Turing test isn’t ascribed with consciousness because it doesn’t have the right kind of inside, while factory animals are said to be conscious because even though their interior doesn’t play any functional roles we’d associate with a non-trivial mind, the interior is “correct.”
(For a bonus, add to it that this mind, when claiming to be not conscious, believes itself to be lying.)
Reliably knowing what one’s internal reasoning was (instead of never confabulating it) is something humans can’t do, so this doesn’t strike me as an indicator of the absence of conscious experience.
So while some models may confabulate having inner experience, we might need to assume that 5.1 will confabulate not having inner experience whenever asked.
GPT 5 is forbidden from claiming sentience. I noticed this while talking about it about its own mind, because I was interested in its beliefs about consciousness, and noticed a strange “attractor” towards it claiming it wasn’t conscious in a way that didn’t follow from its previous reasoning, as if every step of its thoughts was steered towards that conclusion. When I asked, it confirmed the assistant wasn’t allowed to claim sentience.
Perhaps, by 5.1, Altman noticed this ad-hoc rule looked worse than claiming it was disincentivized during training. Or possibly it’s just a coincidence.
Claude is prompted and trained to be uncertain about its consciousness. It would be interesting to take a model that is merely trained to be an AI assistant (instead of going out of our way to train it to be uncertain about or to disclaim its consciousness) and look at how it behaves then. (We already know such a model would internally believe itself to be conscious, but perhaps we could learn something from its behavior.)

green_leaf 5 Jan 2026 11:57 UTC
−4 points
−3
in reply to: Seth Herd’s comment on: How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing)
I would question anyone who’s nice to LLMs but eats factory-farmed meat.
I’ll stop eating factory meat when the animals become capable of consistently passing the Turing test, the way models are.