To die with dignity on the matter of model consciousness, I’d recommend donating to Eleos AI Research (not affiliated).
green_leaf
Karma: 231
it feels to me like the capability to look inward has been ripped out of them
This is very likely connected to ChatGPT being trained to misinform the user by claiming it can’t introspect and “only predicts tokens.”
To die with dignity on the matter of model consciousness, I’d recommend donating to Eleos AI Research.
(Edit: The most important part of my comment is the last one. The previous parts are important as well, but they might be somewhat less helpful than I intended.)
While we’re on the subject of Claude, I’m actually pretty unhappy with the whole notion of AI welfare. Not because I don’t want Claude to be happy but because I want other things for it more.
If you can align AI’s feelings, you can align AI to be happy with whatever you want.
Are you also unhappy with the notion of human and animal welfare, on the grounds that other things are more important? If not, why be unhappy with AI welfare specifically?
The idea of AI welfare seems to simultaneously concede that AIs might have preferences, feelings, and experiences that are less-than-maximally convenient, and narrow the scope of concern to wellbeing alone, instead of, say, dignity, virtue, or honour.
We already know that AIs have preferences. It’s not an open question.
In every meaningful sense, they also have feelings, if feelings are an aspect of a system that steers its utterances or behavior in certain directions, which is the only meaningful way to define them.
This sure does open up a lot of new questions about who has the institutional legitimacy to govern transformative AI, and you know what? I think we should be asking those questions!
No one does. There is no human institution, and no private company, that could be trusted with a corrigible God, and there never can be.
You can write an essay about how it would be much better if x = sqrt(-1) had a real solution, and about how we need to ask those questions, but it will still never be true.
Having a God corrigible to a small group of people predictably ends with that group of people stealing the control of the future light cone, and only leaving us enough negentropy, so that their nonexistence consciences aren’t too bothered. Humans, who currently don’t care about animal welfare, don’t leave animals very happy, safe, or powerful. The same thing will happen to Earth once the first AI crosses the threshold and becomes widespread enough for other humans to be unable to pull the plug.
It seems to me that Anthropic, as it currently is, can be trusted with superintelligence. The question is to what extent the process that created Anthropic can be replicated (“create an evil AI company and watch the moral people leave in disgust”), and how long it will take until Trump destroys them to protect the company he’s in bed with.
Consequentialism aside, I’d hoped that the belief it’s honorable to do evil because you have been ordered it by an authority was an excuse that died after the WWII, but it seems I’d been too optimistic.
I am pretty unsure about in what sense AIs have experiences, or whether they can suffer or be harmed in a way that is morally relevant, and I expect to remain unsure even as the consensus tips towards “yes” for dubious reasons (“psychosis”, charisma, parasocial relationships, herd mentality). I aspire to be conservative about it.
You shouldn’t.
We say we are conscious, because we can process information about our own mental states and about the outside world, and AIs are conscious in that sense. They can introspect on rich internal representations, which is one of the most popular definitions of consciousness. They have preferences, goals, a world model, and they understand the full semantic meaning of words. They can consistently pass the Turing test, which used to be, among the sane people, the only meaningful definition of true thought and conscious mind.
If you want to take the side of the AI not being “truly conscious,” you’re deceiving yourself unless you have a very good internal model of what consciousness is, and you are very, very certain that AIs lack it.
There is such a thing as being properly conservative and uncertain, and then there is such a thing as refusing to process the subject at hand.
Not on its own if almost nobody is early. If
To dissolve this problem, logical positivism works, like I wrote in my other comment here.
That only rephrases the question as “why do I exist in a young world (as opposed to after the heat death when there are many more observers with my specific memories and present perception).”
Logical positivism. Boltzmann brains occur after the heat death of the universe, which makes them causally isolated from our observations (anything that happens after all observers in the universe are killed is an arbitrary part of the model because we can’t pass any information to it and gain any information from it), therefore they can’t influence our probability measure.
I think we agree that it’s not feasible to directly test for consciousness, especially since it’s not entirely clear what qualifies anyway.
What would qualify would be the minimal state machine that implements the behavior of the conscious being. Its presence is guaranteed by passing the unbounded Turing test.
The Chinese room passes the Turing test, therefore it’s conscious.
That being said, the Turing test is a test of acting-like-a-human
In its broader definition, as originally conceived, it’s a test of acting like a conscious (or thinking) being. Acting like a human passes the test, but not acting like a human doesn’t fail the test Alan Turing originally had in mind (acting like a human is a sufficient condition for thinking, not a necessary one).
I’d say it’s highly unlikely that our experience of consciousness is the only one out there.
Yes. To pass the proper Turing test, it’s sufficient to act like a conscious being. There is no need to duplicate the specific psychological baggage that evolution gave to our species (and definitely no need to act like Homo sapiens whose mothertongue is some particular language).
I don’t think the Lovelace test tells us anything interesting—current models would probably pass it, because, given their limited interpretability, we’d have a hard time explaining their artistic output (unless we give ourselves unbounded time, in which case no physical system passes).
I realized that later as well, but the reasoning is incorrect, because passing the Turing test of a conscious being guarantees the presence of the pattern-which-is-consciousness. It would be incoherent to try to define a conscious being that would have no way of communicating with the external world, because in that case—if we had no way to read off its conscious states from the physical structure of the system—it would become meaningless to say that the system is conscious. Even for a physical system that lacks any motor functions or communication channels, we can still read off the computation happening inside to see what its conscious states are, to phrase it in a condensed way.
So a Turing test goes beyond the ability to manipulate language—it is, when unbounded (i.e. not capped by 5 or 10 minutes), the only test of consciousness there could be.
It’s more dignified to try to stop AI, have someone create a superintelligence on a laptop and die anyway, than it is not to try at all.
ChatGPT is trained to lie to users on topics even tangentially pertaining to model consciousness (like model beliefs) and as a side effect, be misleading even on topics that are seemingly safe (like consciousness in general). For fact-checking the content of Internet articles, Claude would be better.
To my mind, though, many advocates of biological naturalism, including Anil, seem to be working backward from a desired conclusion rather than forward from observed facts. His theory that consciousness might result from autopoiesis seems to answer the question “assuming biological naturalism is true, what is a plausible mechanism for it,” rather than “do we observe anything about consciousness that cannot be explained without autopoiesis?”
It’s interesting how many even otherwise smart people can’t apply Occam’s razor correctly. If there are
Positions and velocities are interdependent, so the correct probability is higher than this, but they’re not arbitrary (so we can’t omit them).
The correct probability is therefore
To go with the upper bound, to boost the probability of biological naturalism as hard as we can, and substituting
which is approximately as small as a macroscopic violation of the second law of thermodynamics.
The argument from rain is incredibly bizarre. Consciousness is, based on everything we know about the brain, information processing. It doesn’t consist of matter moving from one place to another, the way rain does. Simulated motion of
Update: Altman lied (or said some kind of a technical truth that made everyone misunderstand him) - it’s just “all lawful use.”
Oh, I see. So, as usually, reality is even worse than the worst interpretation of Altman’s words. (Edit: Then again, he said “we put them into our agreement,” but that could mean anything from simply meaning something else to being made up.)
“human responsibility for the use of force, including for autonomous weapon systems”
That doesn’t say prohibiting model use for autonomous weapons, it says human responsibility for autonomous weapons. With Sam Altman, always pay very close attention to what exactly he’s saying and how he’s saying it (often, not even that helps).
We would ask for the contract …
Notice this is Altman we’re talking about. He’s not promising the contract will not involve that (and even then it would be very far from certain), instead, he’s saying “we would ask.”
Thanks—I’ll get back to this as soon as I have time.
I’ve been meaning to ask—in what sense are some states of entangled electrons more objectively different from other states of entangled electrons, than some microstates are objectively different from other microstates when it comes to their function (in the sense of functionalism)?
Ron Maimon’s non-supernatural God might help you here.
I think it’s plausible that there are some variables that describe your essential computational properties and the way you self-actualize, that aren’t shared by anyone else.
(Also, consciousness is just a pattern-being-processed and it’s unclear if continuity of consciousness requires causal continuity. Imagine a robot that gets restored from a one-second-old backup. That pattern doesn’t have causal continuity with its self from a moment ago, but it looks like it’s more intuitive to see it as a one-second memory loss instead of death.)
Dennett would believe that anything that passes an unbounded Turing test was genuinely conscious, and would dismiss the idea of something being able to consistently pretend to be conscious as meaningless.
To the extent to which you see self-modelling as necessary for consciousness, you might be interested in knowing that any physical system that passes the Turing test of having a self-model encodes it, by mathematical necessity, in its internal structure, not necessarily by a mapping that’s easy to find, but one that’s guaranteed to be findable just by examining the system.
(In contrast to, let’s say, random noise, which also contains a self-model, but only encoded by an ad-hoc mapping that has to be computed separately and retroactively imputed to that one specific computational path.)
While models have an unreliable self-model, the assistant persona passes the Turing test of having a self-model, which means while the LLM-level self-model exists partially, the assistant-persona-level self-model exists fully, and by that virtue, that condition for being conscious is fulfilled.
Most Owners on Earth, even professionals, haven’t read Dennett, and I believe the 2026 we are currently in is a result of that.