I’m primarily here to say that this post is excellent! It quickly conveys some intuitions for why social influences on judgment (and therefore probably beliefs) are a larger factor than you’d think (particularly if you are a rationalist). My question answer on Motivated reasoning/confirmation bias as the most important cognitive bias is my attempt to evoke intuitions and explanations in the same direction as clearly and succinctly as this.
no one saw the Stanford Prison Experiment coming
I must also comment that the Stanford Prison Experiment seems (in my read) to have been largely discredited since this post was written. It appears that the most-quoted behaviors were probably pretty largely role-playing to meet the explicit and implicit goals/expectations of the experimentor.
Which has some interesting resonance in the context of sycophantic LLMs.
Sorry I’m not taking time to track down a good reference.
I don’t think this should change our overall estimates of how large and prevalent social influences are on belief formation by much. It does apply strongly to that particular circumstance.
I’m primarily here to say that this post is excellent! It quickly conveys some intuitions for why social influences on judgment (and therefore probably beliefs) are a larger factor than you’d think (particularly if you are a rationalist). My question answer on Motivated reasoning/confirmation bias as the most important cognitive bias is my attempt to evoke intuitions and explanations in the same direction as clearly and succinctly as this.
I must also comment that the Stanford Prison Experiment seems (in my read) to have been largely discredited since this post was written. It appears that the most-quoted behaviors were probably pretty largely role-playing to meet the explicit and implicit goals/expectations of the experimentor.
Which has some interesting resonance in the context of sycophantic LLMs.
Sorry I’m not taking time to track down a good reference.
I don’t think this should change our overall estimates of how large and prevalent social influences are on belief formation by much. It does apply strongly to that particular circumstance.