I think your pushback is ignoring an important point. One major thing the big contributors have in common is that they tend to be unplugged from the stuff Valentine is naming!
So even if folks mostly don’t become contributors by asking “how can I come more truthfully from myself and not what I’m plugged into”, I think there is an important cluster of mysteries here. Examples of related phenomena:
Why has it worked out that just about everyone who claims to take AGI seriously is also vehement about publishing every secret they discover?
Why do we fear an AI arms race, rather than expect deescalation and joint ventures?
Why does the industry fail to understand the idea of aligned AI, and instead claim that “real” alignment work is adversarial-examples/fairness/performance-fine-tuning?
I think Val’s correct on the point that our people and organizations are plugged into some bad stuff, and that it’s worth examining that.
I think your pushback is ignoring an important point. One major thing the big contributors have in common is that they tend to be unplugged from the stuff Valentine is naming!
So even if folks mostly don’t become contributors by asking “how can I come more truthfully from myself and not what I’m plugged into”, I think there is an important cluster of mysteries here. Examples of related phenomena:
Why has it worked out that just about everyone who claims to take AGI seriously is also vehement about publishing every secret they discover?
Why do we fear an AI arms race, rather than expect deescalation and joint ventures?
Why does the industry fail to understand the idea of aligned AI, and instead claim that “real” alignment work is adversarial-examples/fairness/performance-fine-tuning?
I think Val’s correct on the point that our people and organizations are plugged into some bad stuff, and that it’s worth examining that.