I don’t know how long you’ve been talking to real people, but the vast majority are not particularly good at feedback—less consistent than AI, but that doesn’t make them more correct or helpful. They’re less positive on average, but still pretty un-correlated with “good ideas”. They shit on many good ideas, and support a lot of bad ideas. and are a lot less easy to query for reasons than AI is.
I think there’s an error in thinking talk can ever be sufficient—you can do some light filtering, and it’s way better if you talk to more sources, but eventually you have to actually try stuff.
You just opened my mind. I’m not sure what it is but perhaps I’ve been holding AI to an unreasonably high standard. My best guess is that it’s related to their ability to simulate convincing arguments with near perfect prose. But you’re right. Humans make way more mistakes than your average AI, but for some reason they get a free pass. Definitely worthwhile reflecting on that personal bias.
For context, my motivation to write this piece was part satire, part reminder to remain vigilant of AI sycophancy and cognitive offloading dependence. In their default state, it is far too easy for RLHF-optimised systems to exploit human biases like wanting to be told your smart, feeling special or being emotionally validated.
Key takeaways: take everything AI says with a grain of salt, apply rigour in steelmanning both sides, and exercise agency in rationalizing beliefs. Ironically AI is not necessary to reach this conclusion.
I don’t know how long you’ve been talking to real people, but the vast majority are not particularly good at feedback—less consistent than AI, but that doesn’t make them more correct or helpful. They’re less positive on average, but still pretty un-correlated with “good ideas”. They shit on many good ideas, and support a lot of bad ideas. and are a lot less easy to query for reasons than AI is.
I think there’s an error in thinking talk can ever be sufficient—you can do some light filtering, and it’s way better if you talk to more sources, but eventually you have to actually try stuff.
You just opened my mind. I’m not sure what it is but perhaps I’ve been holding AI to an unreasonably high standard. My best guess is that it’s related to their ability to simulate convincing arguments with near perfect prose. But you’re right. Humans make way more mistakes than your average AI, but for some reason they get a free pass. Definitely worthwhile reflecting on that personal bias.For context, my motivation to write this piece was part satire, part reminder to remain vigilant of AI sycophancy and cognitive offloading dependence. In their default state, it is far too easy for RLHF-optimised systems to exploit human biases like wanting to be told your smart, feeling special or being emotionally validated.
Key takeaways: take everything AI says with a grain of salt, apply rigour in steelmanning both sides, and exercise agency in rationalizing beliefs.
Ironically AI is not necessary to reach this conclusion.