p.b.

Karma: 1,567

p.b. 6 Aug 2026 6:11 UTC
2 points
0
in reply to: green_leaf’s comment on: Reasons to believe current AI models are conscious
I don’t know why that would be the case because I don’t know whether that is the case.
Two possibilities:
Either if you suppress deception features the model starts saying it has two legs—then there is no difference to consciousness.
Or it doesn’t—then the transfer from post-training it to know that it is an AI to “I have no legs” is much more direct than to “I am not conscious”, so even that would not be much evidence.
And if you instead avoid post-training: Base models do not have believes about themselves, because they have no self. They can simulate personas that have a self. If you prompt a base model about whether LLMs are conscious it will echo what humans have written about it.

p.b. 6 Aug 2026 6:00 UTC
4 points
0
in reply to: _rpd’s comment on: Reasons to believe current AI models are conscious
It refers to the very original source of the claim that models believe they are conscious, which was done by suppressing and activating deception features. My statements explain that result—the default persona has “is conscious” as an attribute. Finetuning on “I am a large language model by OpenAI” doesn’t destroy that.
Almost everything a model “believes” is baked into its weights during pretraining and therefore external and not informative about the model’s experiences because the model didn’t learn it from experience.

p.b. 2 Aug 2026 18:01 UTC
3 points
0
in reply to: green_leaf’s comment on: Reasons to believe current AI models are conscious
Models learn the world models of humans, from the subjective perspective of humans, because that’s what their training data contains. They learn “The person who has written the text, of which the next token is now being predicted, is conscious”.
The assistant persona that knows it is a model, is just thin veneer on top of this pretraining based world model.

p.b. 30 Jul 2026 17:07 UTC
2 points
0
on: iVAIS: Outer and Inner Alignment
I must confess that I mostly skimmed the article but I don’t understand why labelling actions as “action of a virtuous character” and “not action of a virtuous character” will work any better than labelling them “acceptable action” and “unacceptable action”.
I also think that character alignment is the way to go, but I think of character traits as priors over multi-agent environments (is it dangerous, is it energy-constrained, are interactions positive, zero, negative sum, are interactions likely to be multi-turn, what do you optimise for, do you optimise collectively or individually, etc.) and I suspect that any kind of stable character will need to be based on RL training in these kinds of environments.

p.b. 28 Jul 2026 7:07 UTC
2 points
0
in reply to: Noa Weiss’s comment on: The State of AI Consciousness Research
This already the second time within a few days that people make really weird assumptions about what I claim or imply on this website. I must be expressing myself really badly.
In this thread I argue that overattributing (human-like consciousness, inner life, moral patienthood) to AIs (or alternatively finding out that they have all that) would facilitate the political movement to give them rights that would lead to direct competition with humans.
That seems to me to be obviously completely logically separate from what I assume about AIs consciousness.

p.b. 25 Jul 2026 19:40 UTC
2 points
0
in reply to: Lukas Petersson’s comment on: Should we be worried about how good AI is getting at coding autonomous drones?
The point was that in that scenario the lagging side might make a deliberate effort to catch up turning something that was on nobody’s mind into a race.

p.b. 25 Jul 2026 6:45 UTC
3 points
0
on: Should we be worried about how good AI is getting at coding autonomous drones?
Frankly, having an eval for something is the first step to get really good at it.
What if your eval finds out that all the Chinese models lag far behind the American models when it comes to drones? Or vice versa?

p.b. 22 Jul 2026 7:23 UTC
2 points
0
in reply to: Noa Weiss’s comment on: The State of AI Consciousness Research
But overattributing (or figuring out that AIs do have consciousness etc) seems like a necessary first step.
I am also not sure how long that way is.
Kids and animals are clearly not on the same intellectual level as adults, part of the reason why they lack rights is that they would not be able to use them.
AI rights pattern match on other rights movements and there are already plenty of zealots ready to take up that cause.

p.b. 21 Jul 2026 9:50 UTC
1 point
−1
in reply to: momom2’s comment on: Reasons to believe current AI models are conscious
As I said, you are taking one sentence out of the context of comparing the argument for reasoning (valid) with the argument for emotions (not valid).
I understand that this one sentence can be read as stronger than it was intended if taken out of context. Well, just don’t do that.
Like, just read what I wrote: At no point do I state the AIs do not have emotions or are not conscious.
I do have opinions on both of these questions. These opinions are based on the architecture of the models, their training and the structure of the human brain.
You do not know and I bet you would not be able to correctly predict my opinions on these questions because nothing I wrote in this thread was about that.

p.b. 21 Jul 2026 8:34 UTC
0 points
−1
in reply to: momom2’s comment on: Reasons to believe current AI models are conscious
You yourself said that you can imitate emotions from an intellectual, non-experimential understanding. I am not claiming anything beyond that. It’s not a strong or controversial statement at all.
I think you are taking my statement out of context, which is: Outward shows of emotion do not prove the inner experience of emotions.

p.b. 20 Jul 2026 15:27 UTC
−2 points
1
in reply to: momom2’s comment on: Reasons to believe current AI models are conscious
The statement refers to LLMs that are trained to imitate human output.
I agree that method acting exists. I don’t think it is the only way for a skilled (human) actor (or writer) to imitate emotions or pain. Especially if the output channel is text.

p.b. 20 Jul 2026 15:15 UTC
2 points
−1
in reply to: Noa Weiss’s comment on: The State of AI Consciousness Research
A world in which models have all the rights a human has (or even just some of them, let’s say the right to own property), is a world where they compete directly against humans and will eventually outcompete humans.

p.b. 19 Jul 2026 13:42 UTC
1 point
−1
on: The State of AI Consciousness Research
The main cost of overattributing is that it makes human extinction much more likely, imho.

p.b. 19 Jul 2026 11:12 UTC
2 points
−2
in reply to: Eye You’s comment on: Reasons to believe current AI models are conscious
Models are explicitly trained to imitate human generated text. There is absolutely nothing misleading about it, it is the single most relevant fact about LLMs.
In all the human generated text the human thinks (and if that comes up, expresses the idea) that it is conscious. So almost all roles an LLM might simulate have “I am conscious” as a basic fact. Finetuning pushes LLMs to a specific assistant role which inherits that fact. There is no reason why RL (for math and code mostly) would change that.
An LLM is nothing before it is filled with the data from human generated text. Daniel Radcliffe on the other hand is a human with his own life and memories. If you’d wipe his brain and actually train it to “imitate Harry Potter” he would think that his parents were killed by Voldemort.

p.b. 19 Jul 2026 10:56 UTC
2 points
−4
in reply to: Haiku’s comment on: Reasons to believe current AI models are conscious
I think if you don’t feel the emotion you don’t have it.
If you shout “oh my god, it’s a bear” in a scared voice and then run, you are certainly representing fear in your brain and it’s also coherent with your behaviour (what I think you call “functional”), but if your amygdala is not firing your are not “having” the emotion fear.
We know from humans that understanding fear or pain and being able to act like you are in fear or pain is a pure sequence learning thing and it can be completely separate from actually being in fear and pain.
Actually being in fear and pain requires additional machinery and some humans don’t have it. Understanding and acting doesn’t replace it.

p.b. 18 Jul 2026 13:33 UTC
3 points
−3
on: Reasons to believe current AI models are conscious
1. For all functional purposes of the words “think” and “reason” and “have emotions”, they think and reason and have emotions.
If you imitate reasoning and solve more problems that way than without imitating reasoning, you’re not just imitating reasoning, you are reasoning. But if you imitate how a human with certain emotions would act you are not having those emotions, you are just acting.
Same with point 6.): Models think they are conscious because they are trained to imitate humans and humans think they are conscious.

p.b. 18 Jul 2026 8:32 UTC
4 points
0
in reply to: Mo Putera’s comment on: Mo Putera’s Shortform
It’s wild that this is possible while at the same time my impression with Fable and GPT-5.5 (and predecessors) is that they say roughly one dumb thing per output.

p.b. 10 Jul 2026 7:36 UTC
3 points
2
on: A Mechanistic Explanation of Prompt Injection (and why you should study roles)
Here’s one idea, which is surprisingly completely unstudied. During training, tokens in input-only roles (<user>, <tool>) are loss-masked: the LLM never has to predict the next token at those positions, so their activations focus entirely on comprehension instead of generation
In finetuning in my experience the problem with this setup is generally that the models learn comprehension much better if they are trained on generation. Activations that do not predict the next token are pretty out of distribution.

p.b. 10 Jul 2026 7:29 UTC
3 points
0
on: A Mechanistic Explanation of Prompt Injection (and why you should study roles)
Because LLMs often confuse my input and their own output I have long wondered whether it might be a good idea to put the tag-processing into the harness and use specific experts (in a MoE) or LoRAs for the tokens in each tag.
That way one can train against instruction following within tool-tags without hurting instruction following generally etc.
Your work makes that sound like an even better idea.

p.b. 8 Jul 2026 16:39 UTC
3 points
0
in reply to: Adele Lopez’s comment on: A global workspace in language models
That was also my first assumption. Interestingly Gemma doesn’t have this.