I really appreciate you for writing how despite verbally agreeing that modesty is unproductive, you nevertheless never judged high-status people as dumb. That’s totally the kind of noticing/Law I imagine we need more of. And I also imagine this is the sort of mindset Eliezer is looking for—the mindset where you figure those things out unprompted, without an Eliezer there to correct you.
I find this comment kind of aggravating.
I’ll claim that the very mindset you mention starts with not taking Eliezer at face value when he half-implies he’s the only person producing useful alignment research on earth, an that his ability to write an angry rant about hopeless it all is proves that everyone else is a follower drone because they didn’t write the rant first.
Like, I think Eliezer deserves a lot of respect, and I’m aware I’m caricaturing him a bit, but… not that much?
I don’t even think I disagree with you in substance. The mindset of thinking for yourself is useful, etc. But part of that mindset is to not unironically quote everything Eliezer says about how smart he is.
This was fun to read, but also a little awkward. This feels less like “The world if everyone was an economist” and more “The world if everyone agreed with Eliezer Yudkowsky about everything”.
Some thoughts:
I don’t care how strong your social norms are, you’re not enforcing that pornography ban. Forget computers, it’s unworkable as long as people have paper.
Same thing with sad people not reproducing. People would go “fuck social norms” and have kids anyway. People who respect the norms would be pushed out of the gene pool. I don’t see how you could enforce those norms without totalitarian violence.
I don’t see how you could have both a self-repairing culture of transparency and also a completely secret conspiracy that suppresses technological development (in a free market with its own evolutionary pressures) without anyone realizing it. The company that makes the fastest computers drives everyone else out of business. You can only stop Moore’s law if everyone coordinates to not build better computers, but that’s not subtle.
I’m not sure EY missed that (the guy is usually really good with this stuff), so maybe the joke is that an AGI already took over their world or something.