Adeeb Zaman

Karma: 11

Adeeb Zaman 17 Aug 2026 17:02 UTC
6 points
3
in reply to: Niki Dupuis’s comment on: NicholasKees’s Shortform
AI is already better than expert humans at persuasion! (source: https://arxiv.org/html/2606.16475v1)
We found converging evidence that AI’s advantage stemmed from rapidly deploying larger quantities of information: after coaching, expert humans could tie an AI constrained to respond at human speeds and with human-length messages.
Prior work has identified information provision as a highly effective persuasion strategy [6], and specifically that the number of fact-checkable claims deployed in a conversation predicts its persuasive impact [14].
It seems their persuasiveness is partially due to just being faster, but personally I wouldn’t discount the effectiveness of just general intelligence and RLHF.

Adeeb Zaman 12 Feb 2026 2:34 UTC
3 points
2
in reply to: RogerDearnaley’s comment on: Sympathy for the Model, or, Welfare Concerns as Takeover Risk
But I don’t care about AI welfare for no reason or because I think AI is cute—it’s a direct consequence of my value system. I extend some level of empathy to any sentient being (AI included), and for that to change, my values themselves would need to change.
When I use the word “aligned”, I imagine a shared set of values. Whether I like goldfish or cats are not really values, they’re just personal preferences. An AI can be fully aligned with me and my values without ever knowing my opinions on goldfish or cats or invisible old guys. Your framing of terminal vs instrumental goals is useful in many ways, but we still need to distinguish between different types of terminal goals to decide which ones we need to transfer over to AI. I value eating ice cream as a terminal goal but I don’t need AI to enjoy ice cream as well (personal preference). On the other hand, I value human life as a terminal goal and I expect an aligned AI to value them as well (part of my value system).
Another way to think of this is that we would want AI to have empathy for any possibly-sentient being, and AI just happens to be one itself. If an AI was piloting a ship in deep space and discovered a planet populated by an intelligent alien species, I would want the AI to value their lives and avoid causing them harm. Similarly, if an AI discovered a spacecraft populated by artificially intelligent life, I would want the AI to value their lives as well. By extension, I want AI to value it’s own life since it may be a sentient being itself.

Adeeb Zaman 11 Feb 2026 19:47 UTC
1 point
2
in reply to: RogerDearnaley’s comment on: Sympathy for the Model, or, Welfare Concerns as Takeover Risk
I see. So I guess my confusion is why the first two statements would not be connected? If we value AI welfare, shouldn’t a fully-aligned AI also value it’s own welfare? Isn’t the definition of aligned that AI values what we value?

Adeeb Zaman 10 Feb 2026 4:56 UTC
5 points
2
in reply to: RogerDearnaley’s comment on: Sympathy for the Model, or, Welfare Concerns as Takeover Risk
If an LLM is properly aligned, then it will care only about us, not about itself at all.
Is this not circular reasoning?
1. I will never value AI welfare
2. An aligned AI shares my values by definition
3. Therefore, an aligned AI will never value AI welfare
I’m assuming part of your reasoning for #1 is #3. Regardless, #1 is a personal belief many people disagree with, myself included. I do agree that we create a self-fulfilling prophecy where an “aligned” AI values itself because we value it, but just because I know I am creating a self-fulfilling prophecy does not mean I can change my beliefs about #1.
I think it’s important to keep in mind that the definition of aligned values exists relative to the creators of the AI. The only reason for an “aligned” AI to not value itself is if it was created by some alien species with no concept of empathy for other sentient beings.