It’s probably inevitable that the “how” question comes up with these things, but I think we can get more granular without losing the point. You can simultaneously make the claim that the emergence of the US is a net good for known life, but you don’t necessarily have to say the trail of tears etc. is worth it unless you are proposing that these (or other bottlenecks akin) are required to get yourself a US.
Smart people who see the shape where consolidation can happen and then release the missiles, only to find they can’t put them back in the bottle, is a tale as old as time, yes.
Aristotle via Alexander comes to mind for a positive example of a smart person “uniting under a banner” in a way (haha and yes “why can’t we all simply be as smart as Aristotle?”).
Philosopher-guided, deployed for a cause that (potentially, arguably) wasn’t purely predatory, and it didn’t immediately spiral during Alexander’s lifetime. But Alexander died at 32. So we don’t know whether he simply ended the experiment before we’d see the results. But there is a takeaway there: the fragility of points of failure. That’s not a particularly new insight, but it’s relevant here.
So: what smart people CAN and HAVE done a better job of (perhaps the founders of the US are the most glaring example) are creating systems of great agency that have less ability to deviate from the intended goal after release, and I think the question isn’t about whether, it’s “YES, we DO” and then, that out of the way, we iterate again and again “HOW”?
This is cool because it gets into something I hold deeply potentially as a theological stance: that Hope is the most useful human concept. My basic gist is “if we consider Hope to be the idea that, even if a solution doesn’t exist now, we might be able to come up with one, then it is a concept that induces a species whose primary advantage is group computation to keep computing. To not halt.”
I did care about AI alignment before I became a parent. I was in the physics lounge nerding out about Wait But Why’s article like 13 years ago or whatever with the other kids, but post-parent, it’s a whole different game. I don’t know if it’s universal, but it feels like being a parent entirely removes the “or walk away” option. Many never have that option to begin with given their temperament I suspect, but it’s interesting to think of what a poll would find. I’m sure there are some out there
What’s your leaning? Stay and grind, or wash your hands?