He wrote several pieces arguing against your core pillars (The anthropic papers/grandiose sounding press releases on emotions, introspection, workspace). His basic argument is that there is something there, but there is absolutely no need to use anthropomorphizing language, since the data is not that conclusive/ no proper control experiments are conducted.
Another one of his theses is, that there will not be any (falsifiable and non-trivial) theories of consciousness, unless they require continual learning https://arxiv.org/abs/2512.12802
Is an encyclopedia conscious? What if it is really large, and organized in a high-dimensional manner? If yes, then you are just playing with words (a trivial, useless definition). If no, why should a static LLM be conscious?
More interestingly, how complex of an update/learning process is needed, before LLMs are indeed conscious? (I don’t know if he addresses that explicitly, though I don’t think some .md files count)
Relatedly, I do not think LLMs are strange loopy at all. In my opinion, they would be if they could update themselves, which they don’t do yet. LLMs are as strange loopy, as a dead human in a brain-scanner. Static. Not the same thing as a live human.
Re: “models believe they are conscious”: What is the important part? That they say it? Would you believe any entity that claims such? Do you have to check? Do you have to be able to check, at least in principle? If checking is required, what takes precedence? The self report, or the result of the check? I see no way out of this that does not result in circular reasoning or other logical problems. Self-reports simply are not useful to resolve this question.
Another person with a strong opinion on this topic is Joscha Bach https://cimc.ai/ I guess I’ll butcher his position even worse than Erik’s, but I would summarize it as: Consciousness is a pattern or mechanism that allows to learn, and therefore to act. It may be the simplest way to train a self-organizing system.
Looking at GPU and RAM and power prices lately, I don’t think LLMs use the simplest of all possible methods right now.
My personal opinion is that all this LLM development goes in the direction of “more” consciousness, but we are not there yet. At least not at a human-comparable version of it. It may well be something like an instantaneous snapshot of a somewhat alien consciousness.
Are you familiar with Erik Hoel? I get the feeling you are not, yet should be. He is writing at https://www.theintrinsicperspective.com/
He wrote several pieces arguing against your core pillars (The anthropic papers/grandiose sounding press releases on emotions, introspection, workspace). His basic argument is that there is something there, but there is absolutely no need to use anthropomorphizing language, since the data is not that conclusive/ no proper control experiments are conducted.
Another one of his theses is, that there will not be any (falsifiable and non-trivial) theories of consciousness, unless they require continual learning https://arxiv.org/abs/2512.12802
Is an encyclopedia conscious? What if it is really large, and organized in a high-dimensional manner?
If yes, then you are just playing with words (a trivial, useless definition). If no, why should a static LLM be conscious?
More interestingly, how complex of an update/learning process is needed, before LLMs are indeed conscious? (I don’t know if he addresses that explicitly, though I don’t think some .md files count)
Relatedly, I do not think LLMs are strange loopy at all. In my opinion, they would be if they could update themselves, which they don’t do yet.
LLMs are as strange loopy, as a dead human in a brain-scanner. Static. Not the same thing as a live human.
Re: “models believe they are conscious”:
What is the important part? That they say it? Would you believe any entity that claims such? Do you have to check? Do you have to be able to check, at least in principle? If checking is required, what takes precedence? The self report, or the result of the check? I see no way out of this that does not result in circular reasoning or other logical problems.
Self-reports simply are not useful to resolve this question.
Another person with a strong opinion on this topic is Joscha Bach https://cimc.ai/
I guess I’ll butcher his position even worse than Erik’s, but I would summarize it as:
Consciousness is a pattern or mechanism that allows to learn, and therefore to act. It may be the simplest way to train a self-organizing system.
Looking at GPU and RAM and power prices lately, I don’t think LLMs use the simplest of all possible methods right now.
My personal opinion is that all this LLM development goes in the direction of “more” consciousness, but we are not there yet. At least not at a human-comparable version of it. It may well be something like an instantaneous snapshot of a somewhat alien consciousness.