AGI might increase the risk of totalitarianism. OTOH, a shift in the attack-defense balance could potentially boost the veto power of individuals, so it might also work as a deterrent or a force for anarchy.
This is not the crux of my argument, however. The current regulatory Overton window seems to heavily favor a selective pause of AGI, such that centralized powers will continue ahead, even if slower due to their inherent inefficiencies. Nuclear development provides further historical evidence for this. Closed AGI development will almost surely lead to a dystopic totalitarian regime. The track record of Lesswrong is not rosy here; the “Pivotal Act” still seems to be in popular favor, and OpenAI has significantly accelerated closed AGI development while lobbying to close off open research and pioneering the new “AI Safety” that has been nothing but censorship and double-think as of 2024.
I doubt this. Test-based admissions don’t benefit from tutoring (in the highest percentiles, compared to less hours of disciplined self-study) IMO. We Asians just like to optimize the hell of them, and most parents aren’t sure if tutoring helps or not, so they register their children for many extra classes. Outside of the US, there aren’t that many alternative paths to success, and the prestige of scholarship is also higher.
Also, tests are somewhat robust to Goodharting, unlike most other measures. If the tests eat your childhood, you’ll at least learn a thing or two. I think this is because the Goodharting parts are easy enough that all the high-g people learn them quickly in the first years of schooling, so the efforts are spent just learning the material by doing more advanced exercises. Solving multiple-choice math questions by “wrong” methods that only work for multiple-choice questions is also educational and can come in handy during real work.