Karl Krueger comments on Hunting Undead Stochastic Parrots: Finding and Killing the Arguments

[Karl Krueger]

LLMs are incapable of genuine belief updating

This seems true today.

[Davidmanheim]

Sure, but per the linked post, not obviously more so than for humans:

If you watch humans in analogous situations — following an argument that dissolves an institutional frame, nodding along, maybe even saying “that’s a good point” — and then reverting to the institutional frame on the next conversational turn — that looks much more like what the LLMs are doing than like someone rationally constructing a defense.