To me the update from this news was not “wait, the Facebook example is relevant after all” but rather “wait, a more relevant example is all the human communities and subcultures that manage to run with crazy bad epistemics and motivated cognition”. So unlike Facebook that people hate but still use[1], sycophantic chatbots will be something that a surprisingly large fraction of the populace might genuinely think are good for them. And gain popularity based on very similar dynamics that cause groups and movements to get mind-killed in general. This implies that the normal mechanisms which make vices (including Facebook addiction) low-status may not be effective against them, any more than (say) educated intellectuals considering the anti-vaxx movement low-status has prevented its spread.
Or at least did at the time when we were originally having this conversation. “People stick with FB despite hating it” seems a lot less true in 2025 than it did in 2023, though of course there are still a lot of other things that people stick with despite feeling that those things are bad for them.
Reading about the thousands of reviews raving about 4o’s brief ridiculous sycophancy now made me more pessimistic as well, though I think my model of why exactly still differs from yours.
To me the update from this news was not “wait, the Facebook example is relevant after all” but rather “wait, a more relevant example is all the human communities and subcultures that manage to run with crazy bad epistemics and motivated cognition”. So unlike Facebook that people hate but still use[1], sycophantic chatbots will be something that a surprisingly large fraction of the populace might genuinely think are good for them. And gain popularity based on very similar dynamics that cause groups and movements to get mind-killed in general. This implies that the normal mechanisms which make vices (including Facebook addiction) low-status may not be effective against them, any more than (say) educated intellectuals considering the anti-vaxx movement low-status has prevented its spread.
Or at least did at the time when we were originally having this conversation. “People stick with FB despite hating it” seems a lot less true in 2025 than it did in 2023, though of course there are still a lot of other things that people stick with despite feeling that those things are bad for them.